crawler

package module
v0.0.0-...-1b07a7a Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2018 License: MIT Imports: 5 Imported by: 0

README

crawler Build Status

网络爬虫程序

  • client 从网页抓取数据
  • document 简单实现文档查询
  • example/meituan 解析美团数据,目前只实现美食部分

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client

type Client struct {
	Header http.Header
	// contains filtered or unexported fields
}

func NewClient

func NewClient() *Client

func (*Client) Get

func (this *Client) Get(url string) (*Document, error)

func (*Client) Head

func (this *Client) Head(url string) error

func (*Client) SetProxy

func (this *Client) SetProxy(proxyURL string) error

设置代理

type Document

type Document struct {
	// contains filtered or unexported fields
}

func NewDocument

func NewDocument(root *html.Node) *Document

func (*Document) Find

func (this *Document) Find(pattern string) ([]string, bool)

Directories

Path Synopsis
example

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL