nullitics

package module
v0.0.0-...-3cfbd4a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 13, 2022 License: MIT Imports: 22 Imported by: 0

README

NULLITICS

Build Status GoDoc GoReportCard example Docker Image Size (latest by date) Buy me a coffee

Zero-effort web analytics. This is a self-hosted open-source version of Nullitics. Check out https://nullitics.com for the world cheapest web analytics in the cloud.

Features

  • Privacy-focused (no cookies, fully anonymized).
  • Easy to set up (no databases, no external dependencies).
  • Meaningful, stylish dashboard.
  • Easy to understand metrics (unique visitors, page views, referrers, countries, device types).
  • Very lightweight (Docker image is under 10MB, same is the size of the executable).
  • Compliant with GDPR, ePrivacy, PECR, CCPA, and COPPA.
  • Fast (can handle 35K req/sec on my humble personal server).
  • By-pass ad-blockers.
  • Can be used as a standalone service or as a Go library.
  • You own your data.

How to use it?

Nullitics comes in several flavours:

  • Embedded (a Go library to use in your Go projects).
  • Self-hosted (a static binary or a Docker container to run on your server).
  • Cloud (the world cheapest web analytics, that also respect your users' privacy).
Library API

The package API is designed to be as simple as possible:

mux := http.NewServeMux()
...
// Create a new Nullitics collector to collect analytics
c := nullitics.New()
// Register a report endpoint to see the dashboard
mux.Handle("/_/stats/", c.Report(nil))
// Wrap mux so that every request would be recorded
http.ListenAndServe(":"+port, c.Collect(mux))

Of course, there's plenty of room for customization, see GoDoc for further details.

Also you may try out the ./cmd/example to see how Nullitics work as library.

Self-hosted container

Running container on a personal server is even simpler:

docker run -p 8080:8080 zserge/nullitics \
	-url mydomain.com \
	-dir nullitics-data \
	-loc Europe/Berlin \

You may check ./cmd/pixel to see how the standalone version works.

Of course, you can still build it yourself and run as a Linux service instead of a Docker container, if you like.

License

Code is distributed under MIT license, feel free to use it in your proprietary projects as well.

Documentation

Index

Constants

View Source 
const (
	// Mobile device type
	Mobile = "mobile"
	// Desktop device type
	Desktop = "desktop"
)

Variables

View Source 
var (
	// MaxPathLength is the longest possible URI or event name length.
	MaxPathLength = 200
	// MaxRefLength is the longest possible referrer length. Typically, domain
	// names are no longer than 63 bytes.
	MaxRefLength = 64
	// MaxCountryLength is the longest possible country code. Nullitics uses ISO
	// codes, so 2 bytes should be enough.
	MaxCountryLength = 2
)
View Source 
var (
	// IPHeaders are request headers, containing the real user IP address
	IPHeaders = []string{"X-Real-IP", "X-Forwarded-For"}
	// MobileUAs are user-agent substrings, typical only for mobile devices
	MobileUAs = []string{"iPhone", "iPad", "Android"}
	// MobileBreakpoint is the maximum screen width for mobile devices, 992px is taken from Bootstrap.
	MobileBreakpoint = 992
	// SkipSubdomains is a list of common subdomains to skip in referrers
	SkipSubdomains = []string{"www.", "www1.", "www2.", "www3.", "www4.", "m.", "l.", "lm.", "i.", "old."}
	// BotAgents is a list of substrings commonly met in bot/crawler User-Agent strings
	BotAgents = []string{"bot", "crawler", "spider", "spyder", "search", "worm", "fetch", "nutch", "http://", "https://"}
)
View Source 
var Now = time.Now

Now is a timestamp generator. It is made public so that the caller could mock it or replace with their own implementation.

View Source 
var (

	// ReportTemplate is a http/template.Template for the default dashboard UI.
	// Feel free to customize it to your own needs by providing the {{extra_head}},
	// {{header}} or {{footer}} sections.
	ReportTemplate = template.Must(template.ParseFS(report, "report/*.*", "report/components/*.*"))
)

Functions

func RandomString 

func RandomString(n int) string

RandomString is a helper utility to generate random string IDs, salts etc.

Types

type Appender 

type Appender struct {
	// contains filtered or unexported fields
}

Appender is an append-only log writer.

func NewAppender 

func NewAppender(filename string, truncate bool) (*Appender, error)

NewAppender creates an Appender for the given log filename. It may optionally truncate the log file before using it.

func (*Appender) Append 

func (ap *Appender) Append(hit *Hit) error

Append write hit data to the end of the log file.

func (*Appender) Close 

func (ap *Appender) Close() error

Close shuts down the appender.

func (*Appender) StartTime 

func (ap *Appender) StartTime() time.Time

StartTime returns the timestamp of the first hit in the log.

type Collector 

type Collector struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Collector is an abstracton that records Hits and provides collected Stats.

func New 

func New(options ...Option) *Collector

New creates a collector instance with the given options.

func (*Collector) Add 

func (c *Collector) Add(r *http.Request) error

Add allows to collect a hit caused by the given request.

func (*Collector) Close 

func (c *Collector) Close() error

Close shuts down the collector.

func (*Collector) Collect 

func (c *Collector) Collect(h http.Handler) http.Handler

Collect is a middleware that wraps an existing handler and collects every hit/request.

func (*Collector) Hit 

func (c *Collector) Hit(hit *Hit) error

Hit records a single hit data.

func (*Collector) Report 

func (c *Collector) Report(extra interface{}) http.Handler

Report returns a handler that renders the dashboard report for the collected stats

func (*Collector) ServeHTTP 

func (c *Collector) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP makes Collector implement a http.Handler interface. This handler acts as an API and allows to collect stats via a tracking pixel or POST API.

func (*Collector) Stats 

func (c *Collector) Stats() (*Stats, *Stats, error)

Stats returns the daily and overall statistic for the given collector. Daily stats have hourly precision, total stats have daily precision.

type Frame 

type Frame struct {
	Rows []Row
	// contains filtered or unexported fields
}

Frame is a matrix-shaped data frame, used to store time series of named integer values.

func (*Frame) Delete 

func (f *Frame) Delete(name string) bool

Delete removes a row from the frame

func (*Frame) Grow 

func (f *Frame) Grow(size int)

Grow resizes all the rows of the frame to the given width. Same width will be also applied to the new rows that will be created in this frame.

func (*Frame) Len 

func (f *Frame) Len() int

Len returns the current length of the row time series (i.e. row width) within the frame.

func (*Frame) Row 

func (f *Frame) Row(name string) *Row

Row returns an existing row by its name or inserts one. It ensures that rows are sorted alphabetically within the data frame.

type GeoFinder 

type GeoFinder interface {
	Find(ip string) string
}

GeoFinder is an interface, that can find the country ISO code by the IP address. 

var GeoDB GeoFinder

GeoDB is a file location for a GeoLite CSV database. By default the location is set to $GEODB environment variable, but can be changed if needed.

func NewGeoDB 

func NewGeoDB(zipfile string) (GeoFinder, error)

NewGeoDB reads a GeoLite CSV zip file and returns the geo database, or an error.

type Hit 

type Hit struct {
	Timestamp time.Time
	URI       string
	Session   string
	Ref       string
	Country   string
	Device    string
}

Hit is a basic data type describing a single page visit or event.

type Option 

type Option func(c *Collector)

Option is a function option data type for Collector.

func Blacklist 

func Blacklist(b func(string) bool) Option

Blacklist ignores certain URIs using a custom filtering function.

func BlacklistPrefix 

func BlacklistPrefix(blacklist ...string) Option

BlacklistPrefix ignores certain URIs by prefix.

func Dir 

func Dir(dir string) Option

Dir sets the collector working directory.

func Location 

func Location(loc *time.Location) Option

Location set the collector time zone.

func Salt 

func Salt(salt string) Option

Salt initializes the collector salt for hashes. By default the salt is a random string.

type Row 

type Row struct {
	Name   string
	Values []int
}

Row is a single record for names time series in the stats data frame. Row has a unique name and a series of integer values, one for each time slot of the data frame.

func (*Row) Get 

func (r *Row) Get(i int) int

Get is a helper method to safely read Row values without taking care about row bounds.

func (*Row) Last 

func (r *Row) Last(n int) (sum int)

Last returns the sum of the last n elements in the Row

type Stats 

type Stats struct {
	Start     time.Time
	Interval  time.Duration
	URIs      Frame
	Sessions  Frame
	Refs      Frame
	Countries Frame
	Devices   Frame
}

Stats is an aggregated data from the various site-related statistics over a given time period.

func ParseAppendLog 

func ParseAppendLog(filename string, location *time.Location) (*Stats, error)

ParseAppendLog read the log file, assuming the timestamps are in the given time zone, and returns a Stats object with hourly precision.

func ParseStatsCSV 

func ParseStatsCSV(s string) (*Stats, error)

ParseStatsCSV returns a Stats instance from the given CSV text string.

func (*Stats) CSV 

func (stats *Stats) CSV() string

CSV returns a CSV-formatted text stats representation.

Source Files

View all Source files

Directories

Path Synopsis
cmd
example
pixel

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.