README
¶
Mozilla Public Suffix List database
MPSL is a domain parser based on Mozilla's public suffix list.
Domain to parse may contain the following parts:
- Top level domain (TLD) - the last part of the domain that doesn't contain a dots (example: com, org, ...).
- Effective TLD (eTLD) - the last part that contains arbitrary count of the dots (example: co.uk, cdn.prod.atlassian-dev.net, ...).
- Effective TLD plus one (eTLD+1) - eTLD + domain name (or first subdomain).
- ICANN flag - is TLD/eTLD provided by public ICANN list.
This package provides a possibility to get all parts at once or get them separately by call corresponding methods (see below).
Installation
The package requires access to pre-downloaded public suffix database or URL to the list file to fetch it on the fly (preferred way is to give URL to full PSL database).
There is three ways to initialize the database:
db, err := mpsl.New(hasher)
// way 1: load from local file
err = db.Load("/path/to/psl/file")
// way 2: fetch from remote URL
err = db.Fetch("http://url/to/psl/file") // or FetchFull() to fetch official PSL file.
// way 3: load from local file (if file's mod time is less than expire) or fetch from remote URL (and save result to the local file).
err = db.LoadOrFetchIf("/path/to/psl/file", "http://url/to/psl/file", time.Hour * 24) // or LoadOrFetchFullIf(...).
All PSL data will be stored in special storage optimized for fast access and minimal pointers count (three pointers in fact).
Usage
After initialization database will ready to parse the domains:
domain := "a.b.c.amazon.co.uk"
tld, icann := db.GetTLDStr(domain) // "uk", true
etld := db.GetEffectiveTLDStr(domain) // "co.uk"
etld1 := db.GetEffectiveTLDPlusOneStr(domain) // "amazon.co.uk"
// or get all parts at once
tld, etld, etld1, icann := db.ParseStr(domain) // the same data ...
Documentation
¶
Index ¶
- Constants
- Variables
- type DB
- func (db *DB) Fetch(dbURL string) (err error)
- func (db *DB) FetchFull() error
- func (db *DB) GetETLD(hostname []byte) []byte
- func (db *DB) GetETLD1(hostname []byte) []byte
- func (db *DB) GetETLD1Str(hostname string) string
- func (db *DB) GetETLDStr(hostname string) string
- func (db *DB) GetEffectiveTLD(hostname []byte) (etld []byte)
- func (db *DB) GetEffectiveTLDPlusOne(hostname []byte) (etld1 []byte)
- func (db *DB) GetEffectiveTLDPlusOneStr(hostname string) (etld1 string)
- func (db *DB) GetEffectiveTLDStr(hostname string) (etld string)
- func (db *DB) GetTLD(hostname []byte) (tld []byte, icann bool)
- func (db *DB) GetTLDStr(hostname string) (tld string, icann bool)
- func (db *DB) Load(dbFile string) (err error)
- func (db *DB) LoadOrFetchFullIf(dbFile string, expire time.Duration) error
- func (db *DB) LoadOrFetchIf(dbFile, dbURL string, expire time.Duration) error
- func (db *DB) Parse(hostname []byte) (tld, etld, etld1 []byte, icann bool)
- func (db *DB) ParseStr(hostname string) (tld, etld, etld1 string, icann bool)
- func (db *DB) Reset()
Constants ¶
const (
// FullURL of full known PSL data.
FullURL = "https://raw.githubusercontent.com/publicsuffix/list/master/public_suffix_list.dat"
)
Variables ¶
var ( ErrBadDB = errors.New("cache uninitialized, use New()") ErrNoHasher = errors.New("no hasher provided") )
Functions ¶
This section is empty.
Types ¶
type DB ¶
type DB struct {
// contains filtered or unexported fields
}
DB is an implementation of Mozilla Public Suffix List database.
func (*DB) GetETLD1Str ¶
GetETLD1Str is a shorthand alias of GetEffectiveTLDPlusOneStr.
func (*DB) GetETLDStr ¶
GetETLDStr is a shorthand alias of GetEffectiveTLDStr.
func (*DB) GetEffectiveTLD ¶
GetEffectiveTLD returns only eTLD part of hostname.
func (*DB) GetEffectiveTLDPlusOne ¶
GetEffectiveTLDPlusOne return only eTLD1 part of hostname.
func (*DB) GetEffectiveTLDPlusOneStr ¶
GetEffectiveTLDPlusOneStr return only eTLD1 part of string hostname.
func (*DB) GetEffectiveTLDStr ¶
GetEffectiveTLDStr returns only eTLD part of string hostname.
func (*DB) LoadOrFetchFullIf ¶
LoadOrFetchFullIf is similar to LoadOrFetchIf but loads data from full URL.
func (*DB) LoadOrFetchIf ¶
LoadOrFetchIf loads PSL data from local file (if file is younger than expire param) or load data from given URL.
After fetching from URL raw PSL data will be stored to local file (if possible).
Source Files