Documentation
¶
Index ¶
- func JsonToFile(filename, indent string, object interface{}) error
- func RawEach(s *goquery.Selection) (a []*goquery.Selection)
- func RecursiveChildFiltered(s *goquery.Selection, filters ...string) *goquery.Selection
- func URLJoin(baseURL string, paths ...string) (string, error)
- type Scraper
- func (s *Scraper) DoRaw(urlS, method string, data []byte) (body []byte, _ error)
- func (s *Scraper) DownloadFile(urlS, outdir string) (suggested, filename, newURL string, _ error)
- func (s *Scraper) Get(urlS, sel string) (_ *goquery.Selection, newURL string, _ error)
- func (s *Scraper) GetRaw(urlS string) ([]byte, error)
- type ScraperOpts
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func JsonToFile ¶
writes object to file as json
func RawEach ¶
let's play dumb (unavail/-exported newSingleSelection), redoing things for nth time already
func RecursiveChildFiltered ¶
Types ¶
type Scraper ¶
type Scraper struct {
Cookies []*network.CookieParam // required: Name, Value, Domain: ".ope.ee"
Timeout time.Duration // 0: disabled
// Only works after InitScraper()
InitExtraAllocatorOpts []chromedp.ExecAllocatorOption
InitGlobalConcurrentLimit int
Ctx context.Context
// contains filtered or unexported fields
}
use InitScraper to initialize internal values
func InitScraper ¶
populates internal values on Scraper
func (*Scraper) DownloadFile ¶
based on https://github.com/chromedp/examples/blob/3384adb2158f6df7e6a48458875a3a5f24aea0c3/download_file/main.go timeout: 0 to disable
type ScraperOpts ¶
type ScraperOpts struct {
Executable string
}
Source Files
Click to show internal directories.
Click to hide internal directories.