core

package
v0.2.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 27, 2020 License: MIT Imports: 7 Imported by: 1

Documentation

Overview

Package core proposes a generic framework that executes online clustering algorithm.

Index

Constants

View Source
const (
	// Iterations is the number of iterations
	Iterations = "iterations"
	// PushedData is the number of pushed data
	PushedData = "pushedData"
	// Duration is algo duration
	Duration = "duration"
	// LastDataTime is the last pushed data time
	LastDataTime = "lastDataTime"
)

Variables

View Source
var ErrAlreadyCreated = errors.New("Algorithm is already created")

ErrAlreadyCreated raised if algorithm is already created

View Source
var ErrElapsedIter = errors.New("amount of iterations done")

ErrElapsedIter raised when amont of iterations is done

View Source
var ErrFinished = errors.New("algorithm is finished")

ErrFinished raised when algorithm is finished

View Source
var ErrIdle = errors.New("Algorithm is idle")

ErrIdle raised if algo is idle

View Source
var ErrInitializing = errors.New("Algorithm is initializing")

ErrInitializing raised while algorithm status equals Initializing

View Source
var ErrNeverFinish = errors.New("algorithm can not finish. Specify core.Conf.Iter, core.Conf.IterPerData, core.Conf.DataPerIter or core.Conf.Finishing for allowing your algorithm to sleep")

ErrNeverFinish raised when wait method is called while the algorithm will never finish

View Source
var ErrNotAlive = errors.New("algorithm is not alive")

ErrNotAlive raised when algo is not alive

View Source
var ErrNotIdle = errors.New("Algorithm is not idle")

ErrNotIdle idle status is asked and not setted

View Source
var ErrNotIterate = errors.New("algorithm can not iterate. Check iterations and dataPerIter conditions")

ErrNotIterate raised when play is called while algo can not iterate

View Source
var ErrNotRunning = errors.New("Algorithm is not running")

ErrNotRunning raised while algorithm status equals Created, Ready or Failed

View Source
var ErrReconfiguring = errors.New("Algorithm is reconfiguring")

ErrReconfiguring raised if algo is reconfiguring

View Source
var ErrRunning = errors.New("Algorithm is running")

ErrRunning raised while algorithm status equals Running, Idle or Sleeping

View Source
var ErrSleeping = errors.New("Algorithm is sleeping")

ErrSleeping raised while algorithm status equals Sleeping, Idle or Sleeping

View Source
var ErrTimeout = errors.New("algorithm timed out")

ErrTimeout is returned when an error occurs

Functions

func IsFinished added in v0.2.0

func IsFinished(finishing Finishing, model OCModel) bool

IsFinished input finishing on input OCModel

func Par

func Par(process func(int, int, int), size int, degree int)

Par runs a function in parallel over data partitions given data size and degree of parallelism

func PrepareConf added in v0.2.0

func PrepareConf(conf Conf) (err error)

PrepareConf before using it in algo

func WaitTimeout

func WaitTimeout(finishing Finishing, duration time.Duration, ocm OCModel) error

WaitTimeout process. Return true if timedout

Types

type Algo

type Algo struct {
	// contains filtered or unexported fields
}

Algo in charge of algorithm execution with both implementation and user configuration

func NewAlgo

func NewAlgo(conf Conf, impl Impl, space Space) (algo *Algo)

NewAlgo creates a new algorithm instance

func (*Algo) Batch

func (algo *Algo) Batch() (err error)

Batch executes the algorithm in batch mode

func (*Algo) CanNeverFinish added in v0.2.0

func (algo *Algo) CanNeverFinish(finishing Finishing, timeout time.Duration) bool

CanNeverFinish true if finishing is nil, timeout is negative, conf.Timeout, conf.Iter and conf.IterPerData equal 0

func (*Algo) Centroids

func (algo *Algo) Centroids() (centroids Clust)

Centroids Get the centroids currently found by the algorithm

func (*Algo) Conf

func (algo *Algo) Conf() Conf

Conf returns configuration

func (*Algo) Copy

func (algo *Algo) Copy(conf Conf, space Space) (oc OnlineClust, err error)

Copy make a copy of this algo with new conf and space

func (*Algo) Impl

func (algo *Algo) Impl() Impl

Impl returns impl

func (*Algo) Init

func (algo *Algo) Init() error

Init initialize centroids and set status to Ready

func (*Algo) Pause

func (algo *Algo) Pause() (err error)

Pause the algorithm and set status to idle

func (*Algo) Play

func (algo *Algo) Play() (err error)

Play the algorithm in online mode

func (*Algo) Predict

func (algo *Algo) Predict(elemt Elemt) (pred Elemt, label int, dist float64)

Predict the cluster for a new observation

func (*Algo) Push

func (algo *Algo) Push(elemt Elemt) (err error)

Push a new observation in the algorithm

func (*Algo) RuntimeFigures

func (algo *Algo) RuntimeFigures() (figures RuntimeFigures)

RuntimeFigures returns specific algo properties

func (*Algo) Space

func (algo *Algo) Space() Space

Space returns space

func (*Algo) Status

func (algo *Algo) Status() OCStatus

Status returns algorithm status and failed error

func (*Algo) Stop

func (algo *Algo) Stop() (err error)

Stop the algorithm

func (*Algo) Wait

func (algo *Algo) Wait(finishing Finishing, timeout time.Duration) (err error)

Wait for online finishing predicate

type AndFinishing added in v0.2.0

type AndFinishing struct {
	Finishings []Finishing
}

AndFinishing applies and rule over finishings

func (AndFinishing) IsFinished added in v0.2.0

func (and AndFinishing) IsFinished(ocm OCModel) (cond bool)

IsFinished is the And Convergence implementation

type Buffer

type Buffer interface {
	Push(elemt Elemt, running bool) error
	Data() []Elemt
	Apply() error
}

Buffer interface

func NewDataBuffer

func NewDataBuffer(data []Elemt, size int) Buffer

NewDataBuffer creates a fixed size buffer if given size > 0. Otherwise creates an infinite size buffer.

type Clust

type Clust []Elemt

Clust type is an abbrevation for centroids indexed by labels.

func (*Clust) Assign

func (c *Clust) Assign(elemt Elemt, space Space) (centroid Elemt, label int, dist float64)

Assign returns the element nearest centroid, its label and the distance to the centroid

func (*Clust) Initializer

func (c *Clust) Initializer(int, []Elemt, Space, *rand.Rand) (centroids Clust, err error)

Initializer that always returns the centroids

func (*Clust) MapLabel

func (c *Clust) MapLabel(elemts []Elemt, space Space) (labels []int, dists []float64)

MapLabel assigns elements to centroids

func (*Clust) ParMapLabel

func (c *Clust) ParMapLabel(elemts []Elemt, space Space, degree int) (labels []int, dists []float64)

ParMapLabel assigns elements to centroids in parallel

func (*Clust) ParReduceDBA

func (c *Clust) ParReduceDBA(elemts []Elemt, space Space, degree int) (Clust, []int)

ParReduceDBA computes centroids and cardinality of each clusters for given elements in parallel.

func (*Clust) ParReduceDBAForLabels

func (c *Clust) ParReduceDBAForLabels(elemts []Elemt, labels []int, space Space, degree int) ([]Elemt, []int)

ParReduceDBAForLabels computes loss and cardinality in each cluster for the given labels in parallel

func (*Clust) ParReduceLoss

func (c *Clust) ParReduceLoss(elemts []Elemt, space Space, norm float64, degree int) ([]float64, []int)

ParReduceLoss computes loss and cardinality in each cluster for the given elements in parallel

func (*Clust) ParReduceLossForLabels

func (c *Clust) ParReduceLossForLabels(elemts []Elemt, labels []int, space Space, norm float64, degree int) ([]float64, []int)

ParReduceLossForLabels computes loss and cardinality in each cluster for the given labels in parallel

func (*Clust) ParTotalLoss

func (c *Clust) ParTotalLoss(elemts []Elemt, space Space, norm float64, degree int) float64

ParTotalLoss computes loss from distances between elements and their nearest centroid in parallel

func (*Clust) ReduceDBA

func (c *Clust) ReduceDBA(elemts []Elemt, space Space) (centroids Clust, cards []int)

ReduceDBA computes centroids and cardinality of each clusters for given elements.

func (*Clust) ReduceDBAForLabels

func (c *Clust) ReduceDBAForLabels(elemts []Elemt, labels []int, space Space) (means []Elemt, cards []int)

ReduceDBAForLabels computes loss and cardinality in each cluster for the given labels

func (*Clust) ReduceLoss

func (c *Clust) ReduceLoss(elemts []Elemt, space Space, norm float64) ([]float64, []int)

ReduceLoss computes loss and cardinality in each cluster for the given elements

func (*Clust) ReduceLossForLabels

func (c *Clust) ReduceLossForLabels(elemts []Elemt, labels []int, space Space, norm float64) ([]float64, []int)

ReduceLossForLabels computes loss and cardinality in each cluster for the given labels

func (*Clust) TotalLoss

func (c *Clust) TotalLoss(elemts []Elemt, space Space, norm float64) float64

TotalLoss computes loss from distances between elements and their nearest centroid

type ClustStatus

type ClustStatus int64

ClustStatus integer type

const (
	Created      ClustStatus = iota
	Initializing             // initializing
	Ready                    // ready to run
	Running                  // used when algorithm is playing
	Idle                     // paused by user
	Finished                 // when the algorithm has finished
)

ClustStatus const values

func (ClustStatus) String

func (clustStatus ClustStatus) String() string

String display value message

type Conf

type Conf interface {
	Verify() error
	Ctrl() *CtrlConf // get controller configuration
	SetDefaultValues()
}

Conf is implementation configuration interface

type CtrlConf added in v0.2.0

type CtrlConf struct {
	Iter           int            // minimal number of iteration before sleeping. Default unlimited
	IterFreq       float64        // maximal number of iteration per seconds
	Timeout        time.Duration  // minimal number of nanoseconds before stopping the algorithm. 0 is infinite (default)
	DataPerIter    int            // minimal pushed data number before iterating
	IterPerData    int            // minimal iterations per `DataPerIter` data
	StatusNotifier StatusNotifier // algo execution notifier
	Finishing      Finishing      // algo convergence matcher
}

CtrlConf specific to algo controller

func (*CtrlConf) Ctrl added in v0.2.0

func (conf *CtrlConf) Ctrl() *CtrlConf

Ctrl Get pointer to algoconf

func (*CtrlConf) SetDefaultValues added in v0.2.0

func (conf *CtrlConf) SetDefaultValues()

SetDefaultValues set

func (*CtrlConf) Verify added in v0.2.0

func (conf *CtrlConf) Verify() (err error)

Verify conf parameters

type DataBuffer

type DataBuffer struct {
	// contains filtered or unexported fields
}

DataBuffer that stores data. In synchronous mode, when pushed() is called data are stored. In asynchronous mode, when pushed() is called data are staged. Staged data are stored when apply() is called.

func (*DataBuffer) Apply

func (b *DataBuffer) Apply() (err error)

Apply all staged data in asynchronous mode, otherwise do nothing

func (*DataBuffer) Data

func (b *DataBuffer) Data() (data []Elemt)

Data returns buffer data

func (*DataBuffer) Push

func (b *DataBuffer) Push(elmt Elemt, running bool) (err error)

Push stores or stages an element depending on synchronous / asynchronous mode.

type Elemt

type Elemt interface{}

Elemt interface that can be used in a clustering algorithm

func DBA

func DBA(elemts []Elemt, space Space) (dba Elemt, err error)

DBA returns the averaged element

func WeightedDBA

func WeightedDBA(elemts []Elemt, weights []int, space Space) (dba Elemt, err error)

WeightedDBA computes the average of given elements with given weights

type Finishing added in v0.2.0

type Finishing interface {
	IsFinished(OCModel) bool
}

Finishing defines

func NewAndFinishing added in v0.2.0

func NewAndFinishing(finishings ...Finishing) Finishing

NewAndFinishing returns And

func NewOrFinishing added in v0.2.0

func NewOrFinishing(finishings ...Finishing) Finishing

NewOrFinishing returns Or

type Impl

type Impl interface {
	// initialize the algorithm
	Init(OCModel) (Clust, error)
	// process one algorithm iteration
	Iterate(OCModel) (Clust, RuntimeFigures, error)
	// push a data. The second argument is the model
	Push(Elemt, OCModel) error
	// Get a copy of  impl
	Copy(OCModel) (Impl, error)
}

Impl concrete algorithms

type Initializer

type Initializer func(k int, elemts []Elemt, space Space, src *rand.Rand) (centroids Clust, err error)

Initializer function initializes k centroids from the given elements.

type IterFinishing added in v0.2.0

type IterFinishing struct {
	Iter        int
	IterPerData int
}

IterFinishing is a finishing with iterations such as parameter

func NewIterFinishing added in v0.2.0

func NewIterFinishing(iter int, iterPerData int) IterFinishing

NewIterFinishing returns new instance

func (IterFinishing) IsFinished added in v0.2.0

func (i IterFinishing) IsFinished(ocm OCModel) (cond bool)

IsFinished is Iterations Convergence implementation

type OCCtrl added in v0.2.0

type OCCtrl interface {
	Init() error                               // initialize algo centroids with impl strategy
	Play() error                               // play the algorithm
	Pause() error                              // pause the algorithm (idle)
	Wait(Finishing, time.Duration) error       // wait for finishing condition and maximal duration. By default, finishing is ready/idle/finished status, and duration is infinite
	Stop() error                               // stop the algorithm
	Push(Elemt) error                          // add element
	Predict(elemt Elemt) (Elemt, int, float64) // input elemt centroid/label with distance to closest centroid
	Batch() error                              // batch mode (stop, play, wait then stop)
	Copy(Conf, Space) (OnlineClust, error)     // make a copy of this algo with new configuration and space
}

OCCtrl online clustring controller

type OCModel added in v0.2.0

type OCModel interface {
	Centroids() Clust               // clustering result
	Conf() Conf                     // algo conf
	Impl() Impl                     // algo impl
	Space() Space                   // data space
	Status() OCStatus               // algo status
	RuntimeFigures() RuntimeFigures // clustering figures
}

OCModel online clustering model

func NewSimpleOCModel added in v0.2.0

func NewSimpleOCModel(conf Conf, space Space, status OCStatus, runtimeFigures RuntimeFigures, centroids Clust) OCModel

NewSimpleOCModel creates a simple oc model

type OCStatus added in v0.2.0

type OCStatus struct {
	Value ClustStatus
	Error error
}

OCStatus describes Online Clustering status with ClustStatus and error

func NewOCStatus added in v0.2.0

func NewOCStatus(status ClustStatus) OCStatus

NewOCStatus returns ocstatus with specific cluststatus

func NewOCStatusError added in v0.2.0

func NewOCStatusError(err error) OCStatus

NewOCStatusError returns new errored ocstatus

func (OCStatus) Alive added in v0.2.0

func (status OCStatus) Alive() bool

Alive check if status is running without error

func (OCStatus) Running added in v0.2.0

func (status OCStatus) Running() bool

Running check if status is running or idle

type OnlineClust

type OnlineClust interface {
	OCCtrl
	OCModel
}

OnlineClust interface When a prediction is made, the element can be pushed to the model. A prediction consists in a centroid and a label.

type OrFinishing added in v0.2.0

type OrFinishing struct {
	Finishings []Finishing
}

OrFinishing applies or rule over finishings

func (OrFinishing) IsFinished added in v0.2.0

func (or OrFinishing) IsFinished(ocm OCModel) (cond bool)

IsFinished is the Or Convergence implementation

type PartitionProcess

type PartitionProcess = func(start, end, rank int)

PartitionProcess represents a function that runs in parallel over a data partitions

type RuntimeFigures added in v0.2.0

type RuntimeFigures map[string]float64

RuntimeFigures are meta values given by respective impl

type SimpleOCModel added in v0.2.0

type SimpleOCModel struct {
	// contains filtered or unexported fields
}

SimpleOCModel simple for fast run execution

func (SimpleOCModel) Centroids added in v0.2.0

func (model SimpleOCModel) Centroids() Clust

Centroids for simpleocmodel

func (SimpleOCModel) Conf added in v0.2.0

func (model SimpleOCModel) Conf() Conf

Conf for simpleocmodel

func (SimpleOCModel) Impl added in v0.2.0

func (model SimpleOCModel) Impl() Impl

Impl for simpleocmodel

func (SimpleOCModel) RuntimeFigures added in v0.2.0

func (model SimpleOCModel) RuntimeFigures() RuntimeFigures

RuntimeFigures for simpleocmodel

func (SimpleOCModel) Space added in v0.2.0

func (model SimpleOCModel) Space() Space

Space for simpleocmodel

func (SimpleOCModel) Status added in v0.2.0

func (model SimpleOCModel) Status() OCStatus

Status for simpleocmodel

type Space

type Space interface {
	Dist(elemt1, elemt2 Elemt) float64
	Combine(elemt1 Elemt, weight1 int, elemt2 Elemt, weight2 int) Elemt
	Copy(elemt Elemt) Elemt
	Dim(data []Elemt) int
}

Space operations needed for clustering a set of elements

type SpaceConf

type SpaceConf interface{}

SpaceConf is a space configuration interface

type StatusFinishing added in v0.2.0

type StatusFinishing struct {
	Status []ClustStatus // Specific status finish condition
	Error  bool          // if true and ocm failed, finish
}

StatusFinishing compare OCModel status

func NewStatusFinishing added in v0.2.0

func NewStatusFinishing(error bool, status ...ClustStatus) StatusFinishing

NewStatusFinishing returns new instance

func (StatusFinishing) IsFinished added in v0.2.0

func (sf StatusFinishing) IsFinished(ocm OCModel) (cond bool)

IsFinished StatusFinishing finish condition

type StatusNotifier

type StatusNotifier = func(OnlineClust, OCStatus)

StatusNotifier for being notified by Online clustering change status

type Timeout

type Timeout interface {
	Disable()
	Enabled() bool
}

Timeout interface for managing timeout algorithm

func InterruptionTimeout

func InterruptionTimeout(duration time.Duration, interruption func(error) error) (result Timeout)

InterruptionTimeout process

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL