dbapi

package
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 28, 2017 License: GPL-3.0 Imports: 19 Imported by: 5

Documentation

Overview

Package dbapi contains code wrapped around an SQL(ite3) DB. It is used for inserting, updating and retrieving lexical entries from a pronunciation lexicon database. A lexical entry is represented by the dbapi.Entry struct, that mirrors entries of the entry database table, along with associated tables such as transcription and lemma.

Index

Constants

This section is empty.

Variables

View Source 
var Schema = `` /* 6625-byte string literal not displayed */

Schema is a string containing the SQL definition of the lexicon database

Functions

func ImportLexiconFile 

func ImportLexiconFile(db *sql.DB, lexiconName lex.LexName, logger Logger, lexiconFileName string, validator *validation.Validator) error

ImportLexiconFile is intended for 'clean' imports. It doesn't check whether the words already exist and so on. It does not do any sanity checks whatsoever of the transcriptions before they are added. If the validator parameter is initialized, each entry will be validated before import, and the validation result will be added to the db.

func RemoveEmptyStrings 

func RemoveEmptyStrings(ss []string) []string

RemoveEmptyStrings does that

func Sqlite3WithRegex 

func Sqlite3WithRegex()

Sqlite3WithRegex registers an Sqlite3 driver with regexp support. (Unfortunately quite slow regexp matching)

func ToLower 

func ToLower(ss []string) []string

ToLower lower-cases its input strings

func UpdateSchema 

func UpdateSchema(dbFile string) error

UpdateSchema migrates a 'live' pronlex db to a new schema version. The dbFile argument is the path to an Sqlite db file.

func ValidateLexiconFile 

func ValidateLexiconFile(logger Logger, lexiconFileName string, validator *validation.Validator, printMode PrintMode) error

ValidateLexiconFile validates the input file and prints any validation errors to the specified logger.

Types

type DBMQuery 

type DBMQuery struct {
	LexRefs []lex.LexRef
	Query   Query
}

DBMQuery is a query used by the DBManager, containing lexicon referenes (db+lex name) and a dbapi.Query

type DBManager 

type DBManager struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

DBManager is used by external services (i.e., lexserver) to cache sql database instances along with their names

func NewDBManager 

func NewDBManager() *DBManager

NewDBManager creates a new DBManager instance with empty cache

func (*DBManager) AddDB 

func (dbm *DBManager) AddDB(dbRef lex.DBRef, db *sql.DB) error

AddDB is used to add a database to the cached map of available databases. It does NOT create the database on disk. To create AND add the database, use DefineSqliteDB instead.

func (*DBManager) CloseDB 

func (dbm *DBManager) CloseDB(dbRef lex.DBRef) error

CloseDB is used to close the specified database

func (*DBManager) ContainsDB 

func (dbm *DBManager) ContainsDB(dbRef lex.DBRef) bool

ContainsDB checks if the input database reference exists

func (*DBManager) DefineLexicon 

func (dbm *DBManager) DefineLexicon(lexRef lex.LexRef, symbolSetName string) error

DefineLexicon saves the name of a new lexicon to the db.

func (*DBManager) DefineLexicons 

func (dbm *DBManager) DefineLexicons(dbRef lex.DBRef, symbolSetName string, lexes ...lex.LexName) error

DefineLexicons saves the names of the new lexicons to the db.

func (*DBManager) DefineSqliteDB 

func (dbm *DBManager) DefineSqliteDB(dbRef lex.DBRef, dbPath string) error

DefineSqliteDB is used to define a new sqlite3 database and add it to the DB manager cache.

func (*DBManager) DeleteLexicon 

func (dbm *DBManager) DeleteLexicon(lexRef lex.LexRef) error

DeleteLexicon deletes the lexicon from the associated lexicon database. Returns an error if the lexicon doesn't exist, or if the lexicon is not empty.

func (*DBManager) EntryCount 

func (dbm *DBManager) EntryCount(lexRef lex.LexRef) (int64, error)

EntryCount counts the number of entries in a lexicon

func (*DBManager) GetLexicon 

func (dbm *DBManager) GetLexicon(lexRef lex.LexRef) (lex.LexRefWithInfo, error)

GetLexicon returns a information (LexRefWithInfo) matching a lexicon name in the db. Returns error if no such lexicon name in db

func (*DBManager) ImportLexiconFile 

func (dbm *DBManager) ImportLexiconFile(lexRef lex.LexRef, logger Logger, lexiconFileName string, validator *validation.Validator) error

ImportLexiconFile is intended for 'clean' imports. It doesn't check whether the words already exist and so on. It does not do any sanity checks whatsoever of the transcriptions before they are added. If the validator parameter is initialized, each entry will be validated before import, and the validation result will be added to the db.

func (*DBManager) InsertEntries 

func (dbm *DBManager) InsertEntries(lexRef lex.LexRef, entries []lex.Entry) ([]int64, error)

InsertEntries saves a list of Entries and associates them to the lexicon

func (*DBManager) LexiconExists 

func (dbm *DBManager) LexiconExists(lexRef lex.LexRef) (bool, error)

LexiconExists is used to check if the specified lexicon exists in the specified database

func (*DBManager) LexiconStats 

func (dbm *DBManager) LexiconStats(lexRef lex.LexRef) (LexStats, error)

LexiconStats calls the specified database a number of times, gathering different numbers, e.g. on how many entries there are in a lexicon.

func (*DBManager) ListAllEntryStatuses 

func (dbm *DBManager) ListAllEntryStatuses(lexRef lex.LexRef) ([]string, error)

ListAllEntryStatuses returns a list of all names EntryStatuses, also those that are not 'current' (i.e., the most recent status). In other words, this list potentially includes statuses not in use, but that have been used before.

func (*DBManager) ListCurrentEntryStatuses 

func (dbm *DBManager) ListCurrentEntryStatuses(lexRef lex.LexRef) ([]string, error)

ListCurrentEntryStatuses returns a list of all names EntryStatuses marked 'current' (i.e., the most recent status).

func (*DBManager) ListDBNames 

func (dbm *DBManager) ListDBNames() ([]lex.DBRef, error)

ListDBNames lists all database names in the cached map of available databases. It does NOT verify what databases are actually existing on disk.

func (*DBManager) ListLexicons 

func (dbm *DBManager) ListLexicons() ([]lex.LexRefWithInfo, error)

ListLexicons returns a list of defined lexicons, including database name, lexicon name, and symbol set name

func (*DBManager) LookUp 

func (dbm *DBManager) LookUp(q DBMQuery, out lex.EntryWriter) error

LookUp takes a DBMQuery, searches the specified lexicon for the included search query. The result is written to a lex.EntryWriter.

func (*DBManager) LookUpIntoMap 

func (dbm *DBManager) LookUpIntoMap(q DBMQuery) (map[lex.DBRef][]lex.Entry, error)

LookUpIntoMap is a wrapper around LookUp, returning a map of Entries

func (*DBManager) LookUpIntoSlice 

func (dbm *DBManager) LookUpIntoSlice(q DBMQuery) ([]lex.Entry, error)

LookUpIntoSlice is a wrapper around LookUp, returning a slice of Entries

func (*DBManager) MoveNewEntries 

func (dbm *DBManager) MoveNewEntries(dbRef lex.DBRef, fromLex, toLex lex.LexName, newSource, newStatus string) (MoveResult, error)

MoveNewEntries moves lexical entries from the lexicon named fromLexicon to the lexicon named toLexicon. The 'newSource' string is the name of the new source of the entries to be moved, and 'newStatus' is the name of the new status to set on the moved entries. Currently, source and/or status may not be the empty string. TODO: Maybe it should be possible to skip source and status values?

Only "new" entries are moved, i.e., entries with lex.Entry.Strn values found in fromLexicon but *not* found in toLexicon. The rationale behind this function is to first create a small additional lexicon with new entries (the fromLexicon), that can later be appended to the master lexicon (the toLexicon).

func (*DBManager) RemoveDB 

func (dbm *DBManager) RemoveDB(dbRef lex.DBRef) error

RemoveDB is used to remove a database from the cached map of available databases. It does NOT remove from the database from disk.

func (*DBManager) SuperDeleteLexicon 

func (dbm *DBManager) SuperDeleteLexicon(lexRef lex.LexRef) error

SuperDeleteLexicon deletes the lexicon from the associated lexicon database, and also whipes all associated entries out of existence. Returns an error if the lexicon doesn't exist, TODO Send progress message to client over websocket (it takes some time)

func (*DBManager) UpdateEntry 

func (dbm *DBManager) UpdateEntry(e lex.Entry) (lex.Entry, bool, error)

UpdateEntry wraps call to UpdateEntryTx with a transaction, and returns the updated entry, fresh from the db

func (*DBManager) Validate 

func (dbm *DBManager) Validate(lexRef lex.LexRef, logger Logger, vd validation.Validator, q Query) (ValStats, error)

Validate all entries given the specified lexRef and search query. Updates validation stats in db, and returns these.

func (*DBManager) ValidationStats 

func (dbm *DBManager) ValidationStats(lexRef lex.LexRef) (ValStats, error)

ValidationStats returns existing validation stats for the specified lexRef

type LexStats 

type LexStats struct {
	Lexicon string `json:"lexicon"`
	// The number of entries in the lexicon corresponding to database id LexiconID
	Entries int64 `json:"entries"`

	// Status frequencies, as strings: StatusName<TAB>Frequency
	// TODO better structure for status/freq (string/int)
	StatusFrequencies []string `json:"statusFrequencies"`

	ValStats ValStats
}

LexStats holds the result of a call to the dbapi.LexiconStats function.

type Logger 

type Logger interface {
	Progress(string)
	Write(string)
	LogInterval() int
}

Logger is an interface for logging progress and other messages

type MoveResult 

type MoveResult struct {
	N int64
}

MoveResult is returned from the MoveNewEntries function. TODO Since it only contains a single int64, this struct is probably not needed. Only useful if more info is to be returned.

type PrintMode 

type PrintMode int

PrintMode specified the type of output to print (all/valid/invalid) 

const (
	// PrintAll prints all entries, valid or invalid
	PrintAll PrintMode = iota

	// PrintValid prints valid entries only
	PrintValid

	// PrintInvalid prints invalid entries only
	PrintInvalid
)

type Query 

type Query struct {
	// list of words to get corresponding entries for
	Words []string `json:"words"`
	// a 'like' db search expression matching words
	WordLike   string `json:"wordLike"`
	WordRegexp string `json:"wordRegexp"`

	WordParts       []string `json:"wordParts"`
	WordPartsLike   string   `json:"wordPartsLike"`
	WordPartsRegexp string   `json:"wordPartsRegexp"`

	// a slice of Entry.IDs to search for
	EntryIDs []int64 `json:"entryIds"`
	// a 'like' db search expression matching transcriptions
	TranscriptionLike   string `json:"transcriptionLike"`
	TranscriptionRegexp string `json:"transcriptionRegexp"`
	// a 'like' db search expression matching part of speech strings
	PartOfSpeechLike   string `json:"partOfSpeechLike"`
	PartOfSpeechRegexp string `json:"partOfSpeechRegexp"`
	// list of lemma forms to get corresponding entries for
	Lemmas []string `json:"lemmas"`
	// an SQL 'like' expression to match lemma forms
	LemmaLike   string `json:"lemmaLike"`
	LemmaRegexp string `json:"lemmaRegexp"`
	// an SQL 'like' expression to match lemma readings
	ReadingLike   string `json:"readingLike"`
	ReadingRegexp string `json:"readingRegexp"`
	// an SQL 'like' expression to match lemma paradigms
	ParadigmLike   string `json:"paradigmLike"`
	ParadigmRegexp string `json:"paradigmRegexp"`

	// A list of entry statuses to match
	EntryStatus []string `json:"entryStatus"`

	// Select entries with one or more EntryValidations
	HasEntryValidation bool `json:"hasEntryValidation"`

	// the page returned by the SQL query's 'LIMIT' (starts at 1)
	Page int64 `json:"page"`
	// the page length of the SQL query's 'LIMIT'
	PageLength int64 `json:"pageLength"`
}

Query represents an sql search query to the lexicon database TODO Change to list(s) of search critieria. TODO add boolean for include/exclude (i.e., "NOT" in the generated SQL).

func NewQuery 

func NewQuery() Query

NewQuery returns a Query instance where PageLength: 25

func (Query) Empty 

func (q Query) Empty() bool

Empty returns true if there are not search criteria values This is no longer a sane way to do it, since the number of search criteria has grown.

type QueryStats 

type QueryStats struct {
	Query   Query `json:"query"`
	Entries int64 `json:"entries"`
}

QueryStats holds the result of a call to the dbapi.LexiconStats function. TODO add fields for additional stats

type SilentLogger 

type SilentLogger struct {
}

SilentLogger is a muted logger, used for testing to skip too much confusing test output

func (SilentLogger) LogInterval 

func (l SilentLogger) LogInterval() int

LogInterval speficies logging interval (to be used by the calling process)

func (SilentLogger) Progress 

func (l SilentLogger) Progress(s string)

Progress logs progress info

func (SilentLogger) Write 

func (l SilentLogger) Write(s string)

Write logs a message string

type StderrLogger 

type StderrLogger struct {
	LogIntervalVar int
}

StderrLogger is a logger for printing messages to standard error. Implements the dbapi.Logger interface.

func (StderrLogger) LogInterval 

func (l StderrLogger) LogInterval() int

LogInterval speficies logging interval (to be used by the calling process)

func (StderrLogger) Progress 

func (l StderrLogger) Progress(s string)

Progress logs progress info

func (StderrLogger) Write 

func (l StderrLogger) Write(s string)

Write logs a message string

type StdoutLogger 

type StdoutLogger struct {
	LogIntervalVar int
}

StdoutLogger is a logger for printing messages to standard out. Implements the dbapi.Logger interface.

func (StdoutLogger) LogInterval 

func (l StdoutLogger) LogInterval() int

LogInterval speficies logging interval (to be used by the calling process)

func (StdoutLogger) Progress 

func (l StdoutLogger) Progress(s string)

Progress logs progress info

func (StdoutLogger) Write 

func (l StdoutLogger) Write(s string)

Write logs a message string

type Symbol 

type Symbol struct {
	LexiconID   int64  `json:"lexiconId"`
	Symbol      string `json:"symbol"`
	Category    string `json:"category"`
	Description string `json:"description"`
	IPA         string `json:"ipa"`
}

Symbol corresponds to the symbol db table, and holds a phonetic symbol

type ValStats 

type ValStats struct {
	// TotalEntries is the total entries to be validated
	TotalEntries int

	// ValidatedEntries is the total validated entries so far
	ValidatedEntries int

	// TotalValidations is the total number of validation messages so far
	TotalValidations int

	// InvalidEntries is the number of invalid entries so far
	InvalidEntries int

	Levels map[string]int `json:"levels"`
	Rules  map[string]int `json:"rules"`
}

ValStats is used to incrementally give statistics during a validation process, or to just represent a final validation statistics.

func Validate 

func Validate(db *sql.DB, lexNames []lex.LexName, logger Logger, vd validation.Validator, q Query) (ValStats, error)

Validate all entries given the specified lexRef and search query. Updates validation stats in db, and returns these.

type WebSockLogger 

type WebSockLogger struct {
	LogIntervalVar int
	// contains filtered or unexported fields
}

WebSockLogger is a logger for printing messages to a web socket. Implements the dbapi.Logger interface.

func NewWebSockLogger 

func NewWebSockLogger(websock *websocket.Conn) WebSockLogger

NewWebSockLogger creates a new websock logger using the input connection

func (WebSockLogger) LogInterval 

func (l WebSockLogger) LogInterval() int

LogInterval speficies logging interval (to be used by the calling process)

func (WebSockLogger) Progress 

func (l WebSockLogger) Progress(msg string)

Progress logs progress info

func (WebSockLogger) Write 

func (l WebSockLogger) Write(msg string)

Write logs a message string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.