mir

module
v0.0.0-...-c1f947c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 13, 2015 License: GPL-3.0

README

This is Mir, the web indexing and search engine written in Go. To compile the sprinter, use the go building tool like such (in cmd/sprinter) go build ., or use the Makefile by issuing make sprinter in the main directory. To build this you obviously need the Go compiler. Release packages can be found at Github. Once the Go compiler is installed and the Go path set, you can install this package.

To benchmarks, the sprinter program accepts a -bench flag, but it needs a MongoDB testing backend in the specific location of ../storage/mongo_test. This mongo_test folder can be found in the storage folder, and because these benchmarks use functions that exist in the TestDB source, this location is hardcoded. When not using this flag, this error won't be encountered. This means that two extra programs have to be installed, MongoDB and supervisor, to start the benchmarking.

To print the sprinter output, use the -verbose flag.

Currently, the sprinter command also shows the flags for the testing command, because of importing this in the main package. The only two sprinter flags are -verbose and -bench.

A target in the main Makefile can be found, named make archive, that builds the program and creates a directory structure so that the benchmarking can be run. This folder is called sprinter and resides in cmd/sprinter, or you can unpack it from ./sprinter.tar.gz.

In short, the requirements to run all benchmarks are:

  • The Go compiler
  • MongoDB
  • Supervisor If not running a benchmark, but just performing a 2000 request crawl, only the Go compiler is needed.

Directories

Path Synopsis
cmd
sink
The sink is a program that acts as a testing backend to the crawler, so we don't have to pester the internet.
The sink is a program that acts as a testing backend to the crawler, so we don't have to pester the internet.
sprinter
The Mir Sprinter command.
The Mir Sprinter command.
Package sprinter implements our fast web crawler.
Package sprinter implements our fast web crawler.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL