package infer

import "bitbucket.org/dtolpin/infergo/infer"

Package infer contains inference algorithms: maximum likelihood estimation by gradient descent and approximation of the posterior by Markov Chain Monte Carlo methods (notably Hamiltonian Monte Carlo family of algorithms).

Index ¶

• func FuncGrad(m model.Model) ( Func func(x []float64) float64, Grad func(grad, x []float64), )
• type Adam
• func (opt *Adam) Step( m model.Model, x []float64, ) ( ll float64, grad []float64, )
• type DepthAdapter
• func (da *DepthAdapter) Adapt( nuts *NUTS, samples <-chan []float64, nIter int, )
• type DualAveraging
• func (da *DualAveraging) Step(t, x, gradSum float64) float64
• type Grad
• type HMC
• func (hmc *HMC) Sample( m model.Model, x []float64, samples chan []float64, )
• func (s *HMC) Stop()
• type MCMC
• type Momentum
• func (opt *Momentum) Step( m model.Model, x []float64, ) ( ll float64, grad []float64, )
• type NUTS
• func (nuts *NUTS) MeanDepth() float64
• func (nuts *NUTS) Sample( m model.Model, x []float64, samples chan []float64, )
• func (s *NUTS) Stop()

Package Files ¶

adapt.go gonum.go grad.go infer.go mcmc.go

func FuncGrad ¶ Uses

func FuncGrad(m model.Model) (
    Func func(x []float64) float64,
    Grad func(grad, x []float64),
)

FuncGrad returns the function to minimize and the gradient, suitable as fields for gonum optimize.Problem, corresponding to maximization of the model's log-likelihood.

type Adam ¶ Uses

type Adam struct {
    Rate  float64 // learning rate
    Beta1 float64 // first momentum factor
    Beta2 float64 // second momentum factor
    Eps   float64 // stabilizer
    // contains filtered or unexported fields
}

Adam (https://arxiv.org/abs/1412.6980).

func (*Adam) Step ¶ Uses

func (opt *Adam) Step(
    m model.Model,
    x []float64,
) (
    ll float64,
    grad []float64,
)

Step implements the Optimizer interface.

type DepthAdapter ¶ Uses

type DepthAdapter struct {
    DualAveraging
    Depth   float64
    NAdpt   int
    MinGrad float64
}

Parameters of adaptation to the target depth

func (*DepthAdapter) Adapt ¶ Uses

func (da *DepthAdapter) Adapt(
    nuts *NUTS,
    samples <-chan []float64,
    nIter int,
)

Adapt adapts NUTS sampler to the target depth. At most nIter iterations are run.

type DualAveraging ¶ Uses

type DualAveraging struct {
    Rate float64
}

Parameters of dual averaging.

func (*DualAveraging) Step ¶ Uses

func (da *DualAveraging) Step(t, x, gradSum float64) float64

Step implements Nesterov's primal-dual averaging, oversimplified.

chi = -gradSum/math.Sqrt(t)
eta = Rate/t
x = eta*chi + (1-eta)*x

type Grad ¶ Uses

type Grad interface {
    Step(m model.Model, x []float64) (ll float64, grad []float64)
}

Grad is the interface of gradient-based optimizers. Step makes a single step over parameters in the gradient direction.

type HMC ¶ Uses

type HMC struct {

    // Parameters
    L   int     // number of leapfrog steps
    Eps float64 // leapfrog step size
    // contains filtered or unexported fields
}

Vanilla Hamiltonian Monte Carlo Sampler.

func (*HMC) Sample ¶ Uses

func (hmc *HMC) Sample(
    m model.Model,
    x []float64,
    samples chan []float64,
)

func (*HMC) Stop ¶ Uses

func (s *HMC) Stop()

Stop stops a sampler gracefully, using the samples channel for synchronization. Stop must be called before further calls to differentiated code. A part of the MCMC interface.

type MCMC ¶ Uses

type MCMC interface {
    Sample(
        m model.Model,
        x []float64,
        samples chan []float64,
    )
    Stop()
}

MCMC is the interface of MCMC samplers.

type Momentum ¶ Uses

type Momentum struct {
    Rate  float64 //learning rate
    Decay float64 // rate decay
    Gamma float64 // gradient momentum factor
    // contains filtered or unexported fields
}

Gradient ascent with momentum (https://www.nature.com/articles/323533a0). If the momentum factor is not set, and thus 0, reduces to vanilla gradient ascent.

func (*Momentum) Step ¶ Uses

func (opt *Momentum) Step(
    m model.Model,
    x []float64,
) (
    ll float64,
    grad []float64,
)

Step implements the Optimizer interface.

type NUTS ¶ Uses

type NUTS struct {

    // Parameters
    Eps      float64 // step size
    Delta    float64 // lower bound on energy for doubling
    MaxDepth int     // maximum depth
    // Statistics
    // Depth belief is encoded as a vector of beta-bernoulli
    // distributions. If the depth is greater than the element's
    // index i, Depth[i][0] is incremented; for index depth,
    // Depth[depth][1] is incremented.
    Depth [][2]float64 // depth belief
    // contains filtered or unexported fields
}

No U-Turn Sampler (https://arxiv.org/abs/1111.4246).

func (*NUTS) MeanDepth ¶ Uses

func (nuts *NUTS) MeanDepth() float64

MeanDepth returns the average observed depth.

func (*NUTS) Sample ¶ Uses

func (nuts *NUTS) Sample(
    m model.Model,
    x []float64,
    samples chan []float64,
)

func (*NUTS) Stop ¶ Uses

func (s *NUTS) Stop()

Stop stops a sampler gracefully, using the samples channel for synchronization. Stop must be called before further calls to differentiated code. A part of the MCMC interface.