README
¶
(+)go-url-normalize[RFC 3986]
Normalize URL | Count unique normalized URL | Count unique normalized URL per TLD
Installation
Install the package by doing go get:
go get github.com/slimdestro/gourlnormalizer
Three exported methods that you can use in your modules are:
NormalizeURL(url string)
CountUniqueNormalizedUrls(url []string)
CountUniqueNormalizedUrlsPerTopLevelDomain(url []string)
Example
package main
import (
"fmt"
"github.com/slimdestro/gourlnormalizer"
)
func main() {
normalizedUrl,_ := normalizer.NormalizeURL("https://example.com?b=2&a=1")
fmt.Println(normalizedUrl)
// compare both url to see what has changed
urls__ := []string{"https://example.com?a=1&b=2", "https://example.com?b=2&a=1"}
fmt.Println(normalizer.CountUniqueNormalizedUrls(urls__))
urls___ := []string{"https://tempo.com", "https://example.com", "https://subdomain.example.com"}
fmt.Println(normalizer.CountUniqueNormalizedUrlsPerTopLevelDomain(urls___))
}
Author
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var (
Ports = map[string]int{
"http": 80,
"https": 443,
"ftp": 21,
}
)
Functions ¶
func CountUniqueNormalizedUrls ¶
@ CountUniqueNormalizedUrls counts unique normalized urls @ calls NormalizeURL() @ Ex: ["https://example.com?a=1&b=2", "https://example.com?b=2&a=1"] returns 1 @ As these 2 urls are same after they are normalized
func CountUniqueNormalizedUrlsPerTopLevelDomain ¶
@ CountUniqueNormalizedUrls counts unique normalized url per TLD @ calls NormalizeURL() @ Ex: ["https://example.com", "https://subdomain.example.com"] returns map["example.com" => 2]
func NormalizeURL ¶
@ NormalizeURL() returns RFC-3986 formatted string @ this method is also being used as helper: @ CountUniqueNormalizedUrlsPerTopLevelDomain && CountUniqueNormalizedUrls
Types ¶
This section is empty.
Source Files
Click to show internal directories.
Click to hide internal directories.