loadfavicon

package module
v2.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 29, 2023 License: MIT Imports: 20 Imported by: 1

README

loadfavicon

loadfavicon gets and downloads favicons of a given website, written in go.

The version 2 has been fully rewritten.

Features

  • look up all favicons referenced in the of a website, plus the favicon.ico file itself at the root of the website.
  • returns only valid image files and urls. removes duplicates.
  • sluggify website names to make a valid disk filename (eg. to put them in cache for example)
  • get all favicons or only a single one according to choosen options.

Usage

You can install it and run it with the loadfavicon command line

go install github.com/lolorenzo777/loadfavicon

or you can use it in your own code

go get github.com/lolorenzo777/loadfavicon

Requests or bugs?

https://github.com/lolorenzo777/loadfavicon/issues

Licence

Copyright @lolorenzo777 - 2023

MIT Licence

References

Documentation

Overview

loadfavicon module includes a package and a command tool to downloads favicons of a given website.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func DownloadAll added in v2.1.0 

func DownloadAll(client *http.Client, websiteURL string, toDir string, onlymissing bool) (n int, err error)

Download downloads favicons related to a website and save them locally to the 'toDir' directory. The directory is created if it does not exist. The written file is suffixed with the icon file name. Download does not overwrite existing file on disk if onlymissing parameter is turned on.

You need to provide an http.Client for example with a timeout like this : 

client := &http.Client{Timeout: time.Second * 5}

Returns the number of successfull downloads.

func DownloadOne added in v2.1.0 

func DownloadOne(client *http.Client, websiteURL string, toDir string, onlymissing bool) (iconfilename string, err error)

DownloadOne downloads the favicon of a website and save it locally to the 'toDir' directory. The directory is created if it does not exist. DownloadOne downloads the svg version if it exists otherwize looks for one with the highest resolution. The written file is not suffixed with the icon file name. DownloadOne does not overwrite existing file on disk if onlymissing parameter is turned on.

You need to provide an http.Client for example with a timeout like this : 

client := &http.Client{Timeout: time.Second * 5}

Returns the filename of the downloaded icon. If none icons are found the returned filename is empty with no error.

func Slugify added in v2.2.0 

func Slugify(prefix1 string, prefix2 string, filename string, ext string) string

Slugify returns the slugified name base on a simple filename with an extension, and a prefix. The returned string is ready to be used as a filename.

The returned string pattern is: 

[{prefix1}+][{prefix2}+][{filename}].{ext}

where each part is slugified. Any query and fragment are ignored.

Returns only the file name, not an absolute path.

func ToPixels added in v2.1.0 

func ToPixels(size string) int64

ToPixels converts size string in number of pixels. Expected size pattern is {width}x{height}|svg. Returns MaxInt for SVG. Returns -1 if unable to get the image size or if the image is not loaded yet.

Types

type Favicon 

type Favicon struct {
	WebsiteURL *url.URL // The absolute URL of the favicon's host website
	WebIconURL *url.URL // The URL of the favicon's file

	Image    []byte // The loaded raw image
	Color    string // Color specfications if any specified in the <link> node
	Size     string // Size specfications if any specified in the <link> node
	MimeType string // The mimetype detected when loading the image.
}

func Download 

func Download(client *http.Client, websiteURL string, toDir string, size string, onlymissing bool, suffix bool) (favicons []Favicon, err error)

Download downloads favicons related to a website and save it locally to the 'toDir' directory. The directory is created if it does not exist. Download downloads the svg version if it exists otherwize looks for one in the given size. If no favicon exists in te given size, DownloadOne downloads the closest favicon with an upper number of pixels.

Parameters: 

size = {width}x{height} or maxres // If the size is empty all favicons found are downloaded. If the size is 'maxres' only the favicon with the maximum resolution is downloaded.
onlymissing = true // downloads if the file is missing on the disk, do not overwrite.
suffix = true // written file name will be suffixed with the icon file name.

You need to provide an http.Client for example with a timeout like this : 

client := &http.Client{Timeout: time.Second * 5}

Returns the slice of downloaded Favicons. If none icons were found the returned slice is empty with no error.

Example 
verbose.IsOn = false
os.RemoveAll("./examples")

client := &http.Client{Timeout: time.Second * 5}
DownloadOne(client, "https://lolorenzo777.github.io/website4tests-2", "./examples", false)
DownloadAll(client, "https://laurent.lourenco.pro", "./examples", false)
DownloadOne(client, "https://web.archive.org/", "./examples", false)
DownloadOne(client, "https://wikipedia.org", "./examples", false)

files, _ := os.ReadDir("./examples")
for _, file := range files {
	fmt.Println(file.Name())
}

Output:

laurent-lourenco-pro+16x16+favicon-16x16.png
laurent-lourenco-pro+180x180+apple-touch-icon.png
laurent-lourenco-pro+32x32+favicon-32x32.png
lolorenzo777-github-io+32x32+.png
web-archive-org+32x32+.ico
wikipedia-org+160x160+.png

func GetFaviconLinks(client *http.Client, websiteURL string) (favicons []Favicon, errX error)

GetFaviconLinks returns a list of favicons urls for websiteURL.

GetFaviconLinks extracts the list of links declared in the <head> section of the site that may correspond to favicon. favicon.ico is added to the list if the list is empty.

Color and Size favicon properties are filled with data found in the <head> items. The Size and the MimeType will be updated when loading the Image.

You need to provide an http.Client for example with a timeout like this : 

client := &http.Client{Timeout: time.Second * 5}

func Read 

func Read(client *http.Client, websiteURL string, sz ...string) (favicons []Favicon, err error)

Read reads favicons of a website and returns them in a slice. The returned slice is sorted from the highest icon resolution to the lowest one, starting with SVG ones if any.

The size parameter must follow this pattern: 

size = {width}x{height} or maxres

If the size is empty all favicons found are read. If the size is 'maxres' only the favicon with the maximum resolution is downloaded.

You need to provide an http.Client for example with a timeout like this : 

client := &http.Client{Timeout: time.Second * 5}

The returned slice contains valid images only. (see Favicon.ReadImage)

Example 
client := &http.Client{Timeout: time.Second * 5}
for _, w := range _gwebsitesOK {
	fmt.Println("Read:", w)
	icons, err := Read(client, w)
	if err != nil {
		fmt.Println(err)
	} else {
		printlist(icons)
	}
	fmt.Println()
}

Output:

Read: https://lolorenzo777.github.io/website4tests-2
https://lolorenzo777.github.io/website4tests-2  --  https://lolorenzo777.github.io/website4tests-2/test-32x32.png

Read: https://laurent.lourenco.pro
https://laurent.lourenco.pro  --  https://laurent.lourenco.pro/apple-touch-icon.png
https://laurent.lourenco.pro  --  https://laurent.lourenco.pro/favicon-32x32.png
https://laurent.lourenco.pro  --  https://laurent.lourenco.pro/favicon-16x16.png

Read: https://go.dev/
https://go.dev/  --  https://go.dev/images/favicon-gopher.svg

Read: https://brave.com/
https://brave.com/  --  https://brave.com/static-assets/images/cropped-brave_appicon_release-192x192.png
https://brave.com/  --  https://brave.com/static-assets/images/cropped-brave_appicon_release-180x180.png
https://brave.com/  --  https://brave.com/static-assets/images/brave-favicon.png
https://brave.com/  --  https://brave.com/static-assets/images/cropped-brave_appicon_release-32x32.png

Read: https://github.com/
https://github.com/  --  https://github.githubassets.com/pinned-octocat.svg

Read: https://www.sncf.com/
https://www.sncf.com/  --  https://www.sncf.com/themes/sncfcom/img/favicon.png
https://www.sncf.com/  --  https://www.sncf.com/themes/sncfcom/img/apple-touch-icon.png
https://www.sncf.com/  --  https://www.sncf.com/themes/sncfcom/img/favicon.ico
https://www.sncf.com/  --  https://www.sncf.com/themes/sncfcom/img/favicon-32x32.png
https://www.sncf.com/  --  https://www.sncf.com/themes/sncfcom/img/favicon-16x16.png

Read: https://protonmail.com/
https://protonmail.com/  --  https://proton.me/favicons/safari-pinned-tab.svg

Read: https://getbootstrap.com/
https://getbootstrap.com/  --  https://getbootstrap.com/docs/5.3/assets/img/favicons/safari-pinned-tab.svg

Read: https://www.cloudflare.com/
https://www.cloudflare.com/  --  https://www.cloudflare.com/favicon.ico

Read: https://www.docker.com/
https://www.docker.com/  --  https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-192x192.png
https://www.docker.com/  --  https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-180x180.png
https://www.docker.com/  --  https://www.docker.com/wp-content/uploads/2023/04/cropped-Docker-favicon-32x32.png
Example (Second) 
// verbose.IsOn = true
// verbose.IsDebugging = true

client := &http.Client{Timeout: time.Second * 5}
for _, w := range _gwebsitesTricky {
	fmt.Println("Read:", w)
	icons, err := Read(client, w)
	if err != nil {
		fmt.Println(err)
	} else {
		printlist(icons)
	}
	fmt.Println()
}

Output:

Read: https://bitcoin.org/bitcoin.pdf
https://bitcoin.org  --  https://bitcoin.org/img/icons/logo_ios.png?1687792074
https://bitcoin.org  --  https://bitcoin.org/favicon.png?1687792074

Read: https://twitter.com/
https://twitter.com/  --  https://twitter.com/favicon.ico

Read: https://mail.proton.me/u/0/inbox/
https://mail.proton.me/u/0/inbox/  --  https://mail.proton.me/assets/favicon.782cda472f79b5eed726.svg

Read: https://www.amazon.com/
https://www.amazon.com/  --  https://www.amazon.com/favicon.ico

Read: https://getemoji.com/
https://getemoji.com/  --  https://getemoji.com/favicon.ico

Read: https://www.linkedin.com/in/laurentlourenco
https://www.linkedin.com/in/laurentlourenco  --  https://www.linkedin.com/favicon.ico

Read: https://mail.google.com/mail/u/0/#inbox
https://mail.google.com/mail/u/0/#inbox  --  https://mail.google.com/favicon.ico

func (Favicon) IsFaviconIco added in v2.2.1 

func (icon Favicon) IsFaviconIco() bool

IsFaviconIco true if the icon file name is "favicon.ico"

func (Favicon) IsSVG 

func (icon Favicon) IsSVG() bool

IsSVG returns true if the icon file name extension is ".svg"

func (Favicon) Pixels 

func (icon Favicon) Pixels() int64

Pixels returns the total number of pixels. Returns MaxInt for SVG. Returns -1 if unable to get the image size or if the image is not loaded yet.

func (*Favicon) ReadImage 

func (icon *Favicon) ReadImage(client *http.Client) error

func (Favicon) Slugify added in v2.2.0 

func (icon Favicon) Slugify(iconpath bool) string

Slugify returns a slugified file name based on the WebsiteURL, the Size and WebIconURL.

The DiskFileName pattern is: 

{WebsiteURL.Host}+{size}[+{base WebIconURL}].{ext}

where each part is slugified. Any query and fragment are ignored.

Returns only the file name, not an absolute path.

Example 
type Test struct {
	web  string
	icon string
	sz   string
}

data := []Test{
	{web: "https://www.dummy.com",
		icon: "favicon.ico",
	},
	{web: "https://www.dummy.com",
		icon: "favicon.ico",
		sz:   "16X16",
	},
	{web: "https://www.dummy.com/website/",
		icon: "https://cdn.com/dummy/favicon.png",
		sz:   "64x64",
	},
	{web: "https://www.dummy.com/website/",
		icon: "/assets/favicon.png",
		sz:   "128X128",
	},
	{web: "https://www.dummy.com/website/",
		icon: "./assets/favicon.png?1234567",
		sz:   "256x256",
	},
}

for _, d := range data {
	for f := 0; f <= 1; f++ {
		icon := new(Favicon)
		icon.WebsiteURL, _ = url.Parse(d.web)
		icon.WebIconURL, _ = url.Parse(d.icon)
		icon.Size = d.sz
		fmt.Println(icon.Slugify(f == 1))
	}
}

Output:

www-dummy-com+.ico
www-dummy-com+favicon.ico
www-dummy-com+16x16+.ico
www-dummy-com+16x16+favicon.ico
www-dummy-com+64x64+.png
www-dummy-com+64x64+favicon.png
www-dummy-com+128x128+.png
www-dummy-com+128x128+favicon.png
www-dummy-com+256x256+.png
www-dummy-com+256x256+favicon.png

func (Favicon) String 

func (icon Favicon) String() string

Source Files

View all Source files

Directories

Path Synopsis
cmd
loadfavicon
pkg
svg

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.