README
¶
goscreenscraper
Go Screen Scrapper is library for screen scraping. It's support get and post method.
INSTALL
go get github.com/yogyrahmawan/goscreenscraper
USAGE
import (
"fmt"
"github.com/yogyrahmawan/goscreenscraper"
)
func main() {
//example call without redirect, post method
m := make(map[string]string)
m["log"] = "shyrin"
m["pwd"] = "your_own_password"
m["wp-submit"] = "Log In"
m["redirect_to"] = "http://www.tourismjournal.net/wp-admin/"
m["testcookie"] = "1"
req2 := screenscraper.NewRequest("POST","http://www.tourismjournal.net","/wp-login.php","http://www.tourismjournal.net/wp-login.php?2yniv1rfyct10sdlt1d1r",m,nil)
resp2,_ := req2.CallWithoutRedirect()
fmt.Println(resp2.Cookies)
//example call and redirect, get method
req3 := screenscraper.NewRequest("GET","http://www.tourismjournal.net","/wp-admin/","",nil,resp2.Cookies)
resp3,_ := req3.CallAndRedirect()
fmt.Println(string(resp3.RawContent))
}
Release Notes
See Notes on release.txt
Author
Yogy Rahmawan
Documentation
¶
Index ¶
Constants ¶
View Source
const ( METHOD_POST = "POST" METHOD_GET = "GET" )
constant for method post and get
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type HTTPRequest ¶
type HTTPRequest struct {
// contains filtered or unexported fields
}
*
- HTTP Request will be used for creating request
- method : method POST or GET
- postContents : contents that will be sent through request
- baseUrl : base url of site (ex : http://www.yoursite.com) . It must be start with http://
- referer : the previous url
- cookies : response cookies from previous page/login page
*
func NewRequest ¶
func NewRequest(method string, baseUrl string, targetUrl string, referer string, formData map[string]string, cookies []*http.Cookie) *HTTPRequest
initialise request
func (*HTTPRequest) CallAndRedirect ¶
func (httpRequest *HTTPRequest) CallAndRedirect() (*HttpReponse, error)
*
- CallAndRedirect use client for send request.
- It is implement cookieJar so if you want to send request using cookies, Use this method
- It is support redirect page *
func (*HTTPRequest) CallWithoutRedirect ¶
func (httpRequest *HTTPRequest) CallWithoutRedirect() (*HttpReponse, error)
*
- CallWithoutRedirect method using roundtrip transport for sending data
- This method doesn't allow cookies when sending data
- It's also doesn't support redirect. Plese refer doc of http.Transport roundtrip
- I usually use this method to get cookie after successfull login *
type HttpReponse ¶
*
- HTTP Response
- RawContent : array of byte from resp.Body
- lastUrl : landing page(redirect page)
- Cookies : cookies from response
*
Source Files
Click to show internal directories.
Click to hide internal directories.