crawler

command
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 17, 2020 License: MIT Imports: 11 Imported by: 0

README

crawler

This package contains a basic crawler built on top of Bokchoy.

Usage

Producer
go run main.go -run producer -url https://golang.org/

By default the depth is 1, you can override it with -depth flag.

Worker
go run main.go -run worker

By default the concurrency is 1, you can override it with -concurrency flag.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL