Documentation
¶
Index ¶
- Constants
- Variables
- func CheckpointDB(duration string, s chan int)
- func ConfigureLogging(debug bool, w io.Writer)
- func CreateMapVar(n string)
- func CreateStringVar(n, v string)
- func DestroyLink(l *Link)
- func EditMode(s string) bool
- func ExtractUser(r *http.Request) string
- func FormatRequest(r *http.Request) string
- func GetCheckMode(r *http.Request) bool
- func GetExpireTime(start time.Time, duration string) (time.Time, error)
- func GetPrettyBehaviorString(b int) string
- func GetURL(url string, mapLookups map[string]string, mappings map[string]string, ...) (string, bool, error)
- func IndexSearchDB(interval string, s chan int)
- func IsValidKeyword(s string) bool
- func ListenURL() *url.URL
- func NewLinkID(id string) int
- func PrintList(ll ListOfLinks)
- func PruneExpiringLinks(c chan int)
- func RotateSlice(s []string, val string) []string
- func RunFailoverMonitor(updates chan *LinkDatabase)
- func SanitizeURL(u string) string
- func SearchDB(term string, maxresults int, s chan int) []string
- func SendUpdates(ldb *LinkDatabase)
- func Shutdown(d *LinkDatabase, s chan int)
- func Similar(reference, comparison string) float64
- func Synchronize()
- type ByAtime
- type ByClicks
- type ByLinkClicks
- type ByMtime
- type Config
- type EditRecord
- type ExtractionCapture
- type GoRequest
- type Gpath
- type InternalURL
- type Keyword
- type Link
- type LinkDatabase
- func (d *LinkDatabase) CommitNewLink(l *Link) (int, error)
- func (d *LinkDatabase) Couple(ll *ListOfLinks, linkObj *Link)
- func (d *LinkDatabase) Decouple(ll *ListOfLinks, linkObj *Link)
- func (d *LinkDatabase) Export(fh io.Writer, s chan int) error
- func (d *LinkDatabase) ExportNetwork() error
- func (d *LinkDatabase) GetLink(id int, url string) *Link
- func (d *LinkDatabase) Import(fh io.Reader, s chan int) error
- func (d *LinkDatabase) IndexKeywords()
- func (d *LinkDatabase) LinksByAtime(count int) []*Link
- func (d *LinkDatabase) LinksByClicks(count int) []*Link
- func (d *LinkDatabase) LinksByMtime(count int) []*Link
- func (d *LinkDatabase) Prune()
- func (d *LinkDatabase) TopLists(count int) []*ListOfLinks
- type ListOfLinks
- func (ll *ListOfLinks) CheckTag(inputTag string) string
- func (ll *ListOfLinks) ClickSort() []*Link
- func (ll *ListOfLinks) GetRedirectURL() string
- func (ll *ListOfLinks) GetTag(i int) []string
- func (ll *ListOfLinks) GetTagString(i int, delimiter string) string
- func (ll *ListOfLinks) GetUsages(linkid int) []string
- func (ll *ListOfLinks) ModifyLogging(setting bool)
- type Metadata
- type Trie
- type TrieNode
- type UserVariables
Constants ¶
const ( RedirectToList = -1 RedirectToFreshest = -2 // the default when new lists are created RedirectToTop = -3 RedirectToRandom = -4 )
This is an enum for different list of links redirect behaviors. Named cases get negative integers. Zero is going to be "unset" (should never be seen) One or greater will be a link ID.
const (
TRIE_SIZE = 255
)
lower case letters and digits
Variables ¶
var ActiveStandbySeed = generateSeed()
higher number wins and becomes active
var BurnTime = time.Date(1, 1, 1, 1, 1, 1, 1, time.UTC)
used to indicate links to be 'burned after reading', date set well in the past
var ExternalAddress string // NAT/external address where users interface
var ExternalPort int // should be optional
var ExternalProto string // http or https
var FailoverLocal string
var FailoverPeer string
var GodbFileName string
var IsActiveRedirector = true
enum/label determining if this is the active or standby We start active until we lose a dice roll.
var LevDistRatio float64 // ratio of lev distance to term length
var LinkDataBase = MakeNewLinkDatabase()
var LinkLog = make(map[Keyword][]string)
LinkLog structure for special link usages on all special keywords This grows to the size of len(keywords) in the link database. Each array of strings is a list of most recent to oldest usages of that keyword. This is thrown away when the redirector shuts down.
var LinkLogCapacity int
var LinkLogNewKeywords bool
var LinkZero = newEmptyLink(LinkDataBase, "127.0.0.1", "This is link zero.", "link zero!")
var ListenAddress string // address redirector process should listen on
var ListenPort int // port redirector process should listen on
var LogFile string
var Never = time.Date(2081, 7, 17, 7, 12, 0, 0, time.UTC)
used for Mtime, date set ridiculously far in the future
var NewListBehavior string // default behavior for new lists
var PruneInterval string // number of seconds to wait between link burnings
var RedirectorMetadata = MakeNewMetadata()
var RedirectorName string // defaults to 'go2'
var SYNC = make(chan int, 1)
goroutine sync mechanism for linkdatabase interactions
var SearchKeywordsData = make(map[string]string)
Keywords with smooshed strings of their interesting string data
var SearchKeywordsTrie = MakeNewTrie()
Functions ¶
func CheckpointDB ¶
CheckpointDB saves a copy of the link database at a provided interval (a time duration string). This also syncs the DB to the failover peer through a TCP connection at the same interval.
func ConfigureLogging ¶
ConfigureLogging will set debug logging up with the -d flag when this program is run.
func CreateMapVar ¶
func CreateMapVar(n string)
func CreateStringVar ¶
func CreateStringVar(n, v string)
func DestroyLink ¶
func DestroyLink(l *Link)
destroyLink will remove a link object, then decouple it from all lists that link is a member of.
func EditMode ¶
Return true if a string indicates 'edit mode', meaning the user wanted to force an edit page for something.
func ExtractUser ¶
Extract the user login name from any cookies presented in their requests. The cookie name will be 'redirectorlogin' and the value is their login name.
func FormatRequest ¶
FormatRequest generates ascii representation of a request It useful for printing incoming requests on the console.
func GetCheckMode ¶
GetCheckMode reads an HTTP request object and returns a boolean indicating whether or not check mode is enabled on the request. The only indicator of check mode is a URL parameter called "check" with a value of "true" or "false".
func GetExpireTime ¶
GetExpireTime returns a date used to set the delete time on a link. The only reason this exists is to add error checking around parsing of the duration string.
func GetPrettyBehaviorString ¶
This returns a human-readable behavior or a link title if direct is selected as the behavior.
func GetURL ¶
func GetURL(url string, mapLookups map[string]string, mappings map[string]string, lv map[string]string, check chan<- string) (string, bool, error)
GetURL takes a URL string with substitutions and returns a final URL with all substitutions performed. The "complete" boolean indicates whether no more substitutions need to be done.
The {1} is still supposed to be the entire parameter incoming from the user's request., so we still need to substitute it into the URL where specified. TODO: make it impossible to make a variable named "1" to eliminate confusion
func IndexSearchDB ¶
Populate easily-searchable structures with information from the real linkDB. This is meant to be run in a goroutine every 30 seconds or so.
func IsValidKeyword ¶
Check whether a string would create a valid keyword
func ListenURL ¶
Return a URL of this redirector, used for redirects back to index If ExternalPort and ListenPort are different in the configuration, return only the ExternalAddress. Otherwise, return the explicit listen address/port.
func PruneExpiringLinks ¶
func PruneExpiringLinks(c chan int)
pruneExpiringLinks will look through the link database and delete links which have a Dtime in the past.
func RotateSlice ¶
rotateSlice adds val at s[0], rotating all existing elements 1 position rightward This limits capacity of s to LinkLogCapacity, as defined in the config file.
func RunFailoverMonitor ¶
func RunFailoverMonitor(updates chan *LinkDatabase)
This handles incoming updates from the active redirector peer.
func SanitizeURL ¶
Clean up anomalies in a URL string due to omissions or user inputs
func SearchDB ¶
search the entire link database using a string as input This can be used both for suggestions and full search This search algorithm weights the results based on how much of a match we find. Results are ordered from most to least relevant in the returned array.
func SendUpdates ¶
func SendUpdates(ldb *LinkDatabase)
This sends an entire link database out on the wire. It needs to be improved to only send incremental updates.
func Shutdown ¶
func Shutdown(d *LinkDatabase, s chan int)
Shutdown is meant to handle graceful termination of the redirector.
Ctrl+C/sigterm is the signal this runs on.
func Synchronize ¶
func Synchronize()
Types ¶
type ByClicks ¶
type ByClicks []*ListOfLinks
ByClicks implements sorting on an array of ListOfLinks by click count (descending).
type ByLinkClicks ¶
type ByLinkClicks []*Link
ByLinkClicks implements sorting by click count on each link.
func (ByLinkClicks) Len ¶
func (a ByLinkClicks) Len() int
func (ByLinkClicks) Less ¶
func (a ByLinkClicks) Less(i, j int) bool
func (ByLinkClicks) Swap ¶
func (a ByLinkClicks) Swap(i, j int)
type Config ¶
type Config struct {
LocalListenAddress string `json:"local_listen_address"`
LocalListenPort int `json:"local_listen_port"`
ExternalAddress string `json:"external_address"`
ExternalPort int `json:"external_port"`
ExternalProto string `json:"external_proto"`
GodbFilename string `json:"godb_filename"`
RedirectorName string `json:"redirector_name"`
PruneInterval string `json:"prune_interval"`
NewListBehavior string `json:"new_list_behavior"`
LinkLogNewKeywords bool `json:"link_log_new_keywords"`
LinkLogCapacity int `json:"link_log_capacity"`
LevDistRatio float64 `json:"levenshtein_distance_ratio"`
LogFile string `json:"log_file"`
FailoverPeer string `json:"failover_peer"`
FailoverLocal string `json:"failover_local"`
}
func RenderConfig ¶
RenderConfig parses config.json off the disk and returns a Config struct with an err value.
type EditRecord ¶
type EditRecord struct {
EditDate time.Time `json:"edit_date"`
EditMsg string `json:"edit_msg"`
EditUser string `json:"edit_user"`
}
EditRecord holds what amounts to a line in a log file. It contains the date things were done, the person who did it, and the message.
func PrependEdit ¶
func PrependEdit(e []*EditRecord, val *EditRecord) []*EditRecord
type ExtractionCapture ¶
type GoRequest ¶
type GoRequest struct {
WantsCheck bool // keyword starts with 'check'
CheckMode bool // request has check=true URL parameter
EditMode bool // true if keyword starts with '.'
Path Gpath // internal path representation
Valid bool // Is the keyword valid?
User string // pulled from the cookie their browser sent
}
func MakeNewGoRequest ¶
Parse an incoming request for go2 semantics and any errors doing so
func (*GoRequest) StringPath ¶
We need the full path they entered for reconstructing internal links to redirect to. Main use case is redirecting to the check interface using the url param check=true
type Gpath ¶
Gpath holds a Keyword, a Tag, and an array of any Params supplied by the user. This is used to structure their input to the redirector (from their browser URL bar).
type InternalURL ¶
type InternalURL string
custom internal url type
func (InternalURL) Valid ¶
func (i InternalURL) Valid() bool
Is the URL fully formed? Can it be rendered as it currently stands in a browser?
func (InternalURL) VarCount ¶
func (i InternalURL) VarCount() int
How many {variable} blocks are in the URL string?
type Keyword ¶
type Keyword string
Keyword is a string representing a list name.
func MakeNewKeyword ¶
MakeNewKeyword is constructor and sanitizer for keywords so they can be used safely in lookups. valid characters are ALPHA, DIGIT, or any of "-_ ~" (note that space is in there) All keywords successfully created will be lower-cased because case-sensitive would drive people NUTS.
type Link ¶
type Link struct {
ID int // This is the one value users can never change.
URL, Title string
Lists []Keyword
Ctime, Mtime, Atime, Dtime time.Time
LinkVariables map[string]string
Clicks int
}
Links are identified by their global LinkID.
The URL is a string because it might have substitutions within, not being a valid URL while stored here. Ctime == created, Mtime == modified, Atime == last time clicked/redirected LinkVariables keys are variable named capture groups. Values are an enum which defines their defaults.
func TopLink ¶
func TopLink(ll ListOfLinks) *Link
type LinkDatabase ¶
type LinkDatabase struct {
Lists map[Keyword]*ListOfLinks
Links map[int]*Link
Variables *UserVariables
NextLinkID int
}
func MakeNewLinkDatabase ¶
func MakeNewLinkDatabase() *LinkDatabase
NewLinkDatabase is an exported constructor for making that first links db
func (*LinkDatabase) CommitNewLink ¶
func (d *LinkDatabase) CommitNewLink(l *Link) (int, error)
CommitNewLink adds a Link object to the database. We only advance the linkid counter if we are actually about to commit a link.
func (*LinkDatabase) Couple ¶
func (d *LinkDatabase) Couple(ll *ListOfLinks, linkObj *Link)
Couple a an existing link's pointer to a list of links. The list can be existing or will be committed here if new. When you combine a list and a link, the list gets this link included and the link gets its memberships updated.
func (*LinkDatabase) Decouple ¶
func (d *LinkDatabase) Decouple(ll *ListOfLinks, linkObj *Link)
Decouple a list of links and a specific link. If the removal of a link from the list results in a zero-length list, the list is deleted. When the link is removed, its tagbinding entry is removed as well. Any nil arguments are checked and the function returns without doing anything.
func (*LinkDatabase) Export ¶
func (d *LinkDatabase) Export(fh io.Writer, s chan int) error
Export will marshal the current LinkDataBase into JSON and write it to the provided io.Writer.
func (*LinkDatabase) ExportNetwork ¶
func (d *LinkDatabase) ExportNetwork() error
func (*LinkDatabase) GetLink ¶
func (d *LinkDatabase) GetLink(id int, url string) *Link
Get a link object by ID or URL.
func (*LinkDatabase) Import ¶
func (d *LinkDatabase) Import(fh io.Reader, s chan int) error
Import will read data from the provided io.Reader into memory at the global LinkDataBase variable.
func (*LinkDatabase) IndexKeywords ¶
func (d *LinkDatabase) IndexKeywords()
IndexKeywords is meant to be run at a regular interval. It maintains a data structure full of keywords and interesting strings associated with these keywords. This is useful for search functions, so they don't have to iterate through the entire linkDB themselves.
func (*LinkDatabase) LinksByAtime ¶
func (d *LinkDatabase) LinksByAtime(count int) []*Link
Sort by access time (Atime)
func (*LinkDatabase) LinksByClicks ¶
func (d *LinkDatabase) LinksByClicks(count int) []*Link
func (*LinkDatabase) LinksByMtime ¶
func (d *LinkDatabase) LinksByMtime(count int) []*Link
Sort by modification time (Mtime)
func (*LinkDatabase) Prune ¶
func (d *LinkDatabase) Prune()
Iterate through the link database and remove links which have Dtime(s) before 'now'.
This is a destructive operation, meant to keep the DB free of links which are expired.
func (*LinkDatabase) TopLists ¶
func (d *LinkDatabase) TopLists(count int) []*ListOfLinks
TopLists returns a collection of lists of links, sorted by click count. The desired length of the returned result is the input. Use -1 to return all lists in the linkDB, sorted by click count.
type ListOfLinks ¶
type ListOfLinks struct {
Keyword Keyword
Links map[int]*Link
Behavior int // negative IDs are special cases
Clicks int
Usage string
Logging bool
TagBindings map[int][]string
Extractions map[int]ExtractionCapture // int == link ID, ExtractionCapture == param example and regex
}
ListOfLinks most notably contains a map of [int]*link referring to all links coupled with this list. Bindings keep track of link ID-to-code word mappings. A link in a given list of links can have a different code, because each link can be thought of like its own context
func MakeNewList ¶
func MakeNewList(keyword Keyword) *ListOfLinks
Combine a keyword and a link pointer to generate a new listofLinks func MakeNewList(keyword Keyword, linkobj *Link) *ListOfLinks {
func (*ListOfLinks) CheckTag ¶
func (ll *ListOfLinks) CheckTag(inputTag string) string
Check a tag on a list of links and return a string describing any problems (if any). Currently, the only problem users can create is a duplicate tag in a list.
func (*ListOfLinks) ClickSort ¶
func (ll *ListOfLinks) ClickSort() []*Link
ClickSort will sort a list of links by each link's click count. It will not return a modified list, but an array of *Link pointers. This is used mostly for list display purposes.
func (*ListOfLinks) GetRedirectURL ¶
func (ll *ListOfLinks) GetRedirectURL() string
GetRedirectURL will return a URL string for given keyword based on its current behavior.
func (*ListOfLinks) GetTag ¶
func (ll *ListOfLinks) GetTag(i int) []string
Return a tag []string for a given link ID in this list of links.
func (*ListOfLinks) GetTagString ¶
func (ll *ListOfLinks) GetTagString(i int, delimiter string) string
func (*ListOfLinks) GetUsages ¶
func (ll *ListOfLinks) GetUsages(linkid int) []string
GetUsages is run in the templates to provide all possible usage strings for a given link in this list.
func (*ListOfLinks) ModifyLogging ¶
func (ll *ListOfLinks) ModifyLogging(setting bool)
Set link logging to on(true) or off(false) for the provided keyword. true == empty linklog is created for the list false == linklog entry for the keyword is deleted, can be used to clear out history The reason this exists is the user has the right to be forgotten. They should be able to both record and delete recordings of usages of keywords.
type Metadata ¶
type Metadata struct {
ListEdits map[Keyword][]*EditRecord
LinkEdits map[int][]*EditRecord
}
Metadata holds all list/link edits. It can be marshaled to JSON and saved to disk.
func MakeNewMetadata ¶
func MakeNewMetadata() *Metadata
This is called to initialize the in-memory metadata for all lists and links.
type Trie ¶
type Trie struct {
// contains filtered or unexported fields
}
func (*Trie) Insert ¶
Insert a word into the trie, returning an error if the index exceeds the TRIE_SIZE
func (*Trie) Startswith ¶
StartsWith is a prefix match of any length input
type UserVariables ¶
type UserVariables struct {
Strings map[string]string `json:"strings"`
Maps map[string]map[string]string `json:"maps"`
Uses map[string][]*Link `json:"uses"`
}
Users can set variables globally which can be looked up and used in URL replacement operations.
This data is held in memory while running and exported to disk when shutting down. These variables sit in the link database alongside the lists and links. Uses is a structure holding variable names and arrays of link IDs employing those variables.
Source Files