torture
Multi FTP crawler and file search. Written for CCC events.
torture exists of two seperate components: The crawler and the frontend. Both are interconnected via the Elasticsearch backend.
Setup
It is always a good idea to setup a GOPATH.
Dependencies are managed using dep.
General
- Install and setup Elasticsearch
- Go into
crawler/
and run go build
- Go into
frontend/
and run go build
Crawler
- See crawler/README.md
Frontend
- Run the frontend
Docker
Setting up elasticsearch, crawler and frontend is significantly easier using Docker:
cp crawler/config.json .
docker-compose up -d
Reload the crawler's config:
docker-compose kill -s USR1 crawler
Authors
See AUTHORS. Do not forget to add yourself!
License
See LICENSE