url

package module
v1.2.10 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 14, 2023 License: MIT Imports: 9 Imported by: 0

README

url

The url package is a simple url standardizer that will parse a url into constituant parts and will set url.IP flag if/when the hostname is an IPv4|6 and sets the url.IDNA flag when domain converted to the punycode form from UTF-8 characters. Parse confirms host has the basic look of a host. The package also resets itself each time url.Parse("blah") is called. It also provides a safe and simple wrapper for extracting the public suffix based root hostnames.

Note: It may be necessary to recompile the publicsuffix package periodically to keep it current since it uses an intenally compressed reference set.

var u url.URL
// ...
if u.Parse("http://blah.example.com/path/page?q=1#123") {
    // valid, use it
    if !u.IP {
        eTLD, _ := u.EffectiveTLDPlusOne()
        fmt.Println(u.Host,eTLD,u.IDNA)
    }
    fmt.Println(u.String())
}

// struct values after calling u.Parse(url)
//
// u.Host blah.example.com
// u.Path /path/page
// u.IP   false
// u.INDA false
// eTLD  example.com

fp := u.Fingerprint(u)
// fp[0].Host blah.example.com
// fp[0].FP   1cbec737f863e4922cee63cc2ebbfaafcd1cff8b790d8cfd2e6a5d550b648afa
// fp[1].Host example.com
// fp[1].FP   1790d8cfd2cbc2ebbfaafcdeec737f863e492d2cee63c6a5d550b648afacff8b

Why not just use the net/url package?

Because sometimes you just need to write your own package for brevity and easy of use.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FPHex64

func FPHex64(u *URL) string

FPHex64 is a xxhash64 fingerprint generator utility using u.Host from url.URL

func FPUint64

func FPUint64(u *URL) uint64

FPUint64 is a xxhash64 fingerprint generator utility using u.Host from url.URL

func Fingerprint

func Fingerprint(u *URL) (fp []fingerprint)

Fingerprint is a sha256 fingerprint generator utility for url.URL

fp[0] is u.Host
fp[1] is eTLD+1 and only present when u.Host != eTLD+1

sha256: 1cbec737f863e4922cee63cc2ebbfaafcd1cff8b790d8cfd2e6a5d550b648afa

func IsPrivate added in v1.2.2

func IsPrivate(ip interface{}) (ok bool)

IsPrivate utility vefifies that an ip representation is not in a reserved range; supports url.URL, net.IP, and string types

func RemoveScheme added in v1.2.3

func RemoveScheme(url string) string

RemoveScheme helper removes all text before ://

func Segmentizer added in v1.2.9

func Segmentizer(u *URL) (segment struct {
	Label []string
	Host  string
	Path  []string
})

Types

type URL

type URL struct {
	Host, Port, Path string
	IP, IDNA         bool
}

URL parse and validate url with type detections for IP|IDNA

func (*URL) EffectiveTLDPlusOne

func (u *URL) EffectiveTLDPlusOne() (string, error)

EffectiveTLDPlusOne is a wrapper around public suffix version that will convert the u.Host to the EffectiveTLDPlusOne version safely

func (*URL) HasPort added in v1.2.10

func (u *URL) HasPort() bool

reports if a port is present

func (*URL) Parse

func (u *URL) Parse(url string) bool

Parse the url into consitituate parts. Set u.IP flag if hostname is IPv4|6 and u.IDNA flag when domain converted

func (*URL) Parser

func (u *URL) Parser(r io.Reader) func() (string, bool, error)

Parser reads off an io.Reader and returns a function that returns only valid domain|IPv4|IPv6 with the url.IP flag

func (*URL) String

func (u *URL) String() string

String representation of the *URL components rebuit that will and always ensure no trailing slash is present when no path is present

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL