httpext

Go Extended Http-Client

Installation:

Tor have to run on your machine if you want to send Request over the onion network.

➡ how to install tor-service

➡ go get github.com/lostbrokenkey/httpext

client := httpext.New()

// socks5://127.0.0.1:9050 is default
client.enableTor()

// in other cases simply set:
proxy := Proxy{"127.0.0.1", 9050}
client.EnableTorProxy(proxy)

// default user-agent
// regular:     Go-httpext-client/1.0
// tor enabled: a random user-agent is generated

// to set a custom user-agent:
client.Header.UserAgent = "my custom user-agent"

Examples:

GET Html

package main

import "github.com/lostbrokenkey/httpext"

func main() {

	client := httpext.New()
	client.EnableTor()
	res, _ := client.Get("http://example.com/")
	doc, _ := res.HTML()
	node := doc.Find("h1")
	println(node.Text())

}

more information about goquery: https://github.com/PuerkitoBio/goquery

GET JSON

package main

import "github.com/lostbrokenkey/httpext"

type r struct {
	IP string `json:"ip"`
}

func main() {

	client := httpext.New()
	client.EnableTor()
	res, _ := client.Get("https://api.ipify.org?format=json")
	var response r
	_ = res.JSON(&response)
	println(response.IP)

}

GET Feed

package main

import (
	"fmt"

	"github.com/lostbrokenkey/httpext"
)

func main() {

	client := httpext.New()
	url := "https://www.nasa.gov/rss/dyn/breaking_news.rss"
	res, err := client.Get(url)
	feed, _ := res.Feed()
	if err == nil {
		fmt.Printf("%s", feed.Description)
	}

}

POST Json

package main

import (
	"fmt"

	"github.com/lostbrokenkey/httpext"
)

type Test struct {
	Field string `json:"the-field"`
}

func main() {

	t := Test{Field: "hello world"}

	client := httpext.New()
	url := "https://webhook.site/xxxxxxxx-xxxxxxxxx-xxxxxxxxx"
	res, _ := client.PostJSON(url, t)
	str, _ := res.String()
	fmt.Println(str)

}

Dependencies

• goquery - HTML Parser
• gofeed - Feed Parser
• msgpack - MessagePack encoding

have fun 😉