Documentation ¶
Overview ¶
Package core proposes a generic framework that executes online clustering algorithm.
Index ¶
- Constants
- Variables
- func IsFinished(finishing Finishing, model OCModel) bool
- func Par(process func(int, int, int), size int, degree int)
- func PrepareConf(conf Conf) (err error)
- func WaitTimeout(finishing Finishing, duration time.Duration, ocm OCModel) error
- type Algo
- func (algo *Algo) Batch() (err error)
- func (algo *Algo) CanNeverFinish(finishing Finishing, timeout time.Duration) bool
- func (algo *Algo) Centroids() (centroids Clust)
- func (algo *Algo) Conf() Conf
- func (algo *Algo) Copy(conf Conf, space Space) (oc OnlineClust, err error)
- func (algo *Algo) Impl() Impl
- func (algo *Algo) Init() error
- func (algo *Algo) Pause() (err error)
- func (algo *Algo) Play() (err error)
- func (algo *Algo) Predict(elemt Elemt) (pred Elemt, label int, dist float64)
- func (algo *Algo) Push(elemt Elemt) (err error)
- func (algo *Algo) RuntimeFigures() (figures RuntimeFigures)
- func (algo *Algo) Space() Space
- func (algo *Algo) Status() OCStatus
- func (algo *Algo) Stop() (err error)
- func (algo *Algo) Wait(finishing Finishing, timeout time.Duration) (err error)
- type AndFinishing
- type Buffer
- type Clust
- func (c *Clust) Assign(elemt Elemt, space Space) (centroid Elemt, label int, dist float64)
- func (c *Clust) Initializer(int, []Elemt, Space, *rand.Rand) (centroids Clust, err error)
- func (c *Clust) MapLabel(elemts []Elemt, space Space) (labels []int, dists []float64)
- func (c *Clust) ParMapLabel(elemts []Elemt, space Space, degree int) (labels []int, dists []float64)
- func (c *Clust) ParReduceDBA(elemts []Elemt, space Space, degree int) (Clust, []int)
- func (c *Clust) ParReduceDBAForLabels(elemts []Elemt, labels []int, space Space, degree int) ([]Elemt, []int)
- func (c *Clust) ParReduceLoss(elemts []Elemt, space Space, norm float64, degree int) ([]float64, []int)
- func (c *Clust) ParReduceLossForLabels(elemts []Elemt, labels []int, space Space, norm float64, degree int) ([]float64, []int)
- func (c *Clust) ParTotalLoss(elemts []Elemt, space Space, norm float64, degree int) float64
- func (c *Clust) ReduceDBA(elemts []Elemt, space Space) (centroids Clust, cards []int)
- func (c *Clust) ReduceDBAForLabels(elemts []Elemt, labels []int, space Space) (means []Elemt, cards []int)
- func (c *Clust) ReduceLoss(elemts []Elemt, space Space, norm float64) ([]float64, []int)
- func (c *Clust) ReduceLossForLabels(elemts []Elemt, labels []int, space Space, norm float64) ([]float64, []int)
- func (c *Clust) TotalLoss(elemts []Elemt, space Space, norm float64) float64
- type ClustStatus
- type Conf
- type CtrlConf
- type DataBuffer
- type Elemt
- type Finishing
- type Impl
- type Initializer
- type IterFinishing
- type OCCtrl
- type OCModel
- type OCStatus
- type OnlineClust
- type OrFinishing
- type PartitionProcess
- type RuntimeFigures
- type SimpleOCModel
- type Space
- type SpaceConf
- type StatusFinishing
- type StatusNotifier
- type Timeout
Constants ¶
const ( // Iterations is the number of iterations Iterations = "iterations" // PushedData is the number of pushed data PushedData = "pushedData" // Duration is algo duration Duration = "duration" // LastDataTime is the last pushed data time LastDataTime = "lastDataTime" )
Variables ¶
var ErrAlreadyCreated = errors.New("Algorithm is already created")
ErrAlreadyCreated raised if algorithm is already created
var ErrElapsedIter = errors.New("amount of iterations done")
ErrElapsedIter raised when amont of iterations is done
var ErrFinished = errors.New("algorithm is finished")
ErrFinished raised when algorithm is finished
var ErrIdle = errors.New("Algorithm is idle")
ErrIdle raised if algo is idle
var ErrInitializing = errors.New("Algorithm is initializing")
ErrInitializing raised while algorithm status equals Initializing
var ErrNeverFinish = errors.New("algorithm can not finish. Specify core.Conf.Iter, core.Conf.IterPerData, core.Conf.DataPerIter or core.Conf.Finishing for allowing your algorithm to sleep")
ErrNeverFinish raised when wait method is called while the algorithm will never finish
var ErrNotAlive = errors.New("algorithm is not alive")
ErrNotAlive raised when algo is not alive
var ErrNotIdle = errors.New("Algorithm is not idle")
ErrNotIdle idle status is asked and not setted
var ErrNotIterate = errors.New("algorithm can not iterate. Check iterations and dataPerIter conditions")
ErrNotIterate raised when play is called while algo can not iterate
var ErrNotRunning = errors.New("Algorithm is not running")
ErrNotRunning raised while algorithm status equals Created, Ready or Failed
var ErrReconfiguring = errors.New("Algorithm is reconfiguring")
ErrReconfiguring raised if algo is reconfiguring
var ErrRunning = errors.New("Algorithm is running")
ErrRunning raised while algorithm status equals Running, Idle or Sleeping
var ErrSleeping = errors.New("Algorithm is sleeping")
ErrSleeping raised while algorithm status equals Sleeping, Idle or Sleeping
var ErrTimeout = errors.New("algorithm timed out")
ErrTimeout is returned when an error occurs
Functions ¶
func IsFinished ¶ added in v0.2.0
IsFinished input finishing on input OCModel
func Par ¶
Par runs a function in parallel over data partitions given data size and degree of parallelism
func PrepareConf ¶ added in v0.2.0
PrepareConf before using it in algo
Types ¶
type Algo ¶
type Algo struct {
// contains filtered or unexported fields
}
Algo in charge of algorithm execution with both implementation and user configuration
func (*Algo) CanNeverFinish ¶ added in v0.2.0
CanNeverFinish true if finishing is nil, timeout is negative, conf.Timeout, conf.Iter and conf.IterPerData equal 0
func (*Algo) Copy ¶
func (algo *Algo) Copy(conf Conf, space Space) (oc OnlineClust, err error)
Copy make a copy of this algo with new conf and space
func (*Algo) RuntimeFigures ¶
func (algo *Algo) RuntimeFigures() (figures RuntimeFigures)
RuntimeFigures returns specific algo properties
type AndFinishing ¶ added in v0.2.0
type AndFinishing struct {
Finishings []Finishing
}
AndFinishing applies and rule over finishings
func (AndFinishing) IsFinished ¶ added in v0.2.0
func (and AndFinishing) IsFinished(ocm OCModel) (cond bool)
IsFinished is the And Convergence implementation
type Buffer ¶
Buffer interface
func NewDataBuffer ¶
NewDataBuffer creates a fixed size buffer if given size > 0. Otherwise creates an infinite size buffer.
type Clust ¶
type Clust []Elemt
Clust type is an abbrevation for centroids indexed by labels.
func (*Clust) Assign ¶
Assign returns the element nearest centroid, its label and the distance to the centroid
func (*Clust) Initializer ¶
Initializer that always returns the centroids
func (*Clust) ParMapLabel ¶
func (c *Clust) ParMapLabel(elemts []Elemt, space Space, degree int) (labels []int, dists []float64)
ParMapLabel assigns elements to centroids in parallel
func (*Clust) ParReduceDBA ¶
ParReduceDBA computes centroids and cardinality of each clusters for given elements in parallel.
func (*Clust) ParReduceDBAForLabels ¶
func (c *Clust) ParReduceDBAForLabels(elemts []Elemt, labels []int, space Space, degree int) ([]Elemt, []int)
ParReduceDBAForLabels computes loss and cardinality in each cluster for the given labels in parallel
func (*Clust) ParReduceLoss ¶
func (c *Clust) ParReduceLoss(elemts []Elemt, space Space, norm float64, degree int) ([]float64, []int)
ParReduceLoss computes loss and cardinality in each cluster for the given elements in parallel
func (*Clust) ParReduceLossForLabels ¶
func (c *Clust) ParReduceLossForLabels(elemts []Elemt, labels []int, space Space, norm float64, degree int) ([]float64, []int)
ParReduceLossForLabels computes loss and cardinality in each cluster for the given labels in parallel
func (*Clust) ParTotalLoss ¶
ParTotalLoss computes loss from distances between elements and their nearest centroid in parallel
func (*Clust) ReduceDBA ¶
ReduceDBA computes centroids and cardinality of each clusters for given elements.
func (*Clust) ReduceDBAForLabels ¶
func (c *Clust) ReduceDBAForLabels(elemts []Elemt, labels []int, space Space) (means []Elemt, cards []int)
ReduceDBAForLabels computes loss and cardinality in each cluster for the given labels
func (*Clust) ReduceLoss ¶
ReduceLoss computes loss and cardinality in each cluster for the given elements
type ClustStatus ¶
type ClustStatus int64
ClustStatus integer type
const ( Created ClustStatus = iota Initializing // initializing Ready // ready to run Running // used when algorithm is playing Idle // paused by user Finished // when the algorithm has finished )
ClustStatus const values
func (ClustStatus) String ¶
func (clustStatus ClustStatus) String() string
String display value message
type Conf ¶
type Conf interface { Verify() error Ctrl() *CtrlConf // get controller configuration SetDefaultValues() }
Conf is implementation configuration interface
type CtrlConf ¶ added in v0.2.0
type CtrlConf struct { Iter int // minimal number of iteration before sleeping. Default unlimited IterFreq float64 // maximal number of iteration per seconds Timeout time.Duration // minimal number of nanoseconds before stopping the algorithm. 0 is infinite (default) DataPerIter int // minimal pushed data number before iterating IterPerData int // minimal iterations per `DataPerIter` data StatusNotifier StatusNotifier // algo execution notifier Finishing Finishing // algo convergence matcher }
CtrlConf specific to algo controller
func (*CtrlConf) SetDefaultValues ¶ added in v0.2.0
func (conf *CtrlConf) SetDefaultValues()
SetDefaultValues set
type DataBuffer ¶
type DataBuffer struct {
// contains filtered or unexported fields
}
DataBuffer that stores data. In synchronous mode, when pushed() is called data are stored. In asynchronous mode, when pushed() is called data are staged. Staged data are stored when apply() is called.
func (*DataBuffer) Apply ¶
func (b *DataBuffer) Apply() (err error)
Apply all staged data in asynchronous mode, otherwise do nothing
type Elemt ¶
type Elemt interface{}
Elemt interface that can be used in a clustering algorithm
type Finishing ¶ added in v0.2.0
Finishing defines
func NewAndFinishing ¶ added in v0.2.0
NewAndFinishing returns And
func NewOrFinishing ¶ added in v0.2.0
NewOrFinishing returns Or
type Impl ¶
type Impl interface { // initialize the algorithm Init(OCModel) (Clust, error) // process one algorithm iteration Iterate(OCModel) (Clust, RuntimeFigures, error) // push a data. The second argument is the model Push(Elemt, OCModel) error // Get a copy of impl Copy(OCModel) (Impl, error) }
Impl concrete algorithms
type Initializer ¶
type Initializer func(k int, elemts []Elemt, space Space, src *rand.Rand) (centroids Clust, err error)
Initializer function initializes k centroids from the given elements.
type IterFinishing ¶ added in v0.2.0
IterFinishing is a finishing with iterations such as parameter
func NewIterFinishing ¶ added in v0.2.0
func NewIterFinishing(iter int, iterPerData int) IterFinishing
NewIterFinishing returns new instance
func (IterFinishing) IsFinished ¶ added in v0.2.0
func (i IterFinishing) IsFinished(ocm OCModel) (cond bool)
IsFinished is Iterations Convergence implementation
type OCCtrl ¶ added in v0.2.0
type OCCtrl interface { Init() error // initialize algo centroids with impl strategy Play() error // play the algorithm Pause() error // pause the algorithm (idle) Wait(Finishing, time.Duration) error // wait for finishing condition and maximal duration. By default, finishing is ready/idle/finished status, and duration is infinite Stop() error // stop the algorithm Push(Elemt) error // add element Predict(elemt Elemt) (Elemt, int, float64) // input elemt centroid/label with distance to closest centroid Batch() error // batch mode (stop, play, wait then stop) Copy(Conf, Space) (OnlineClust, error) // make a copy of this algo with new configuration and space }
OCCtrl online clustring controller
type OCModel ¶ added in v0.2.0
type OCModel interface { Centroids() Clust // clustering result Conf() Conf // algo conf Impl() Impl // algo impl Space() Space // data space Status() OCStatus // algo status RuntimeFigures() RuntimeFigures // clustering figures }
OCModel online clustering model
func NewSimpleOCModel ¶ added in v0.2.0
func NewSimpleOCModel(conf Conf, space Space, status OCStatus, runtimeFigures RuntimeFigures, centroids Clust) OCModel
NewSimpleOCModel creates a simple oc model
type OCStatus ¶ added in v0.2.0
type OCStatus struct { Value ClustStatus Error error }
OCStatus describes Online Clustering status with ClustStatus and error
func NewOCStatus ¶ added in v0.2.0
func NewOCStatus(status ClustStatus) OCStatus
NewOCStatus returns ocstatus with specific cluststatus
func NewOCStatusError ¶ added in v0.2.0
NewOCStatusError returns new errored ocstatus
type OnlineClust ¶
OnlineClust interface When a prediction is made, the element can be pushed to the model. A prediction consists in a centroid and a label.
type OrFinishing ¶ added in v0.2.0
type OrFinishing struct {
Finishings []Finishing
}
OrFinishing applies or rule over finishings
func (OrFinishing) IsFinished ¶ added in v0.2.0
func (or OrFinishing) IsFinished(ocm OCModel) (cond bool)
IsFinished is the Or Convergence implementation
type PartitionProcess ¶
type PartitionProcess = func(start, end, rank int)
PartitionProcess represents a function that runs in parallel over a data partitions
type RuntimeFigures ¶ added in v0.2.0
RuntimeFigures are meta values given by respective impl
type SimpleOCModel ¶ added in v0.2.0
type SimpleOCModel struct {
// contains filtered or unexported fields
}
SimpleOCModel simple for fast run execution
func (SimpleOCModel) Centroids ¶ added in v0.2.0
func (model SimpleOCModel) Centroids() Clust
Centroids for simpleocmodel
func (SimpleOCModel) Conf ¶ added in v0.2.0
func (model SimpleOCModel) Conf() Conf
Conf for simpleocmodel
func (SimpleOCModel) Impl ¶ added in v0.2.0
func (model SimpleOCModel) Impl() Impl
Impl for simpleocmodel
func (SimpleOCModel) RuntimeFigures ¶ added in v0.2.0
func (model SimpleOCModel) RuntimeFigures() RuntimeFigures
RuntimeFigures for simpleocmodel
func (SimpleOCModel) Space ¶ added in v0.2.0
func (model SimpleOCModel) Space() Space
Space for simpleocmodel
func (SimpleOCModel) Status ¶ added in v0.2.0
func (model SimpleOCModel) Status() OCStatus
Status for simpleocmodel
type Space ¶
type Space interface { Dist(elemt1, elemt2 Elemt) float64 Combine(elemt1 Elemt, weight1 int, elemt2 Elemt, weight2 int) Elemt Copy(elemt Elemt) Elemt Dim(data []Elemt) int }
Space operations needed for clustering a set of elements
type StatusFinishing ¶ added in v0.2.0
type StatusFinishing struct { Status []ClustStatus // Specific status finish condition Error bool // if true and ocm failed, finish }
StatusFinishing compare OCModel status
func NewStatusFinishing ¶ added in v0.2.0
func NewStatusFinishing(error bool, status ...ClustStatus) StatusFinishing
NewStatusFinishing returns new instance
func (StatusFinishing) IsFinished ¶ added in v0.2.0
func (sf StatusFinishing) IsFinished(ocm OCModel) (cond bool)
IsFinished StatusFinishing finish condition
type StatusNotifier ¶
type StatusNotifier = func(OnlineClust, OCStatus)
StatusNotifier for being notified by Online clustering change status