experiments

package module
v0.0.0-...-d47a9d6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 7, 2023 License: Apache-2.0 Imports: 13 Imported by: 0

README

Statistical experiments with stock data

Build Status GoDoc

The purpose of this repository is to hold and document a series of statistical experiments with real financial data, primarily over the US stock market daily price data from 1998 to mid-2022. The code and the necessary configurations are provided, so the experiments can be repeated by anyone with access to the data and some basic understanding of how to compile and run a command-line app.

A few random conventions throughout the study:

  • Even though I'm a single person writing the study, I use the pronoun "we" to refer to myself and my readers, implying that "we're doing this together" (and not "We, Louis XIV"). Whenever I make a personal choice (without the reader), I use the pronoun "I" indicating that it's me who's doing it, it's my personal choice. The distinction is somewhat arbitrary, but it explains why sometimes I write "we" and sometimes "I".

  • Important definitions or key moments will be in italics, while math and code fragments will be either an inlined code or in a

multi-line
code block.

Most plots and other types of experimental results are accompanied by a corresponding config to reproduce the plot or experiment. See the usage section for details on how to use such configs with the tool.

Table of Contents

Installation

Requirements:

git clone https://github.com/stockparfait/experiments.git
cd experiments
make init
make install

This installs an executable experiments in your ${GOPATH}/bin, where GOPATH=`go env GOPATH` .

Quick start

  • Subscribe to the data source on Nasdaq Data Link; most of these experiments use only the equities prices.
  • Download the data by running parfait-sharadar - see stockparfait/stockparfait for details.
  • Copy stockparfait/stockparfait/js folder contents to a separate working directory, for example ~/plots/; here I'll refer to it as ${PLOTS}.
  • Run an experiment.

An example of these steps together:

${GOPATH}/bin/parfait-sharadar
mkdir -p ${PLOTS}
cp stockparfait/stockparfait/js/* ${PLOTS}
experiments -conf ${CONFIG}.json -js ${PLOTS}/data.js

where ${CONFIG}.json is the config of your choice from one of the experiments, or your own. The schema of such configs is in config/config.go.

  • Open ${PLOTS}/plot.html in your browser to see the resulting plots.

Contributing to Stock Parfait Experiments

Pull requests are welcome. We suggest to contact us beforehand to coordinate your code contributions.

Having said that, this repository serves primarily as the documentation of my own research into the behavior of the US stock market, and also as an example and the purpose and motivation behind the core libraries in stockparfait/stockparfait.

Documentation

Overview

Package experiments implements the common Experiment API.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddValue 

func AddValue(ctx context.Context, prefix, key, value string) error

AddValue adds (or overwrites) a <prefix key>:value pair to the Values in the context.

func AnalyticalDistribution 

func AnalyticalDistribution(ctx context.Context, c *config.AnalyticalDistribution) (dist stats.Distribution, distName string, err error)

AnalyticalDistribution instantiates a distribution from config.

func Compound 

func Compound(ctx context.Context, d stats.Distribution, n int, compType string, c *stats.ParallelSamplingConfig) (dist stats.DistributionWithHistogram, err error)

Compound the distribution d; that is, return the distribution of the sum of n samples of d. The compounding is performed according to compType: "direct" (n samples per 1 compounded sample), "fast" (sliding window sum) or "biased" (based on Monte Carlo integration with an appropriate variable substitution), and the configuration of parallel sampling.

func CompoundDistribution 

func CompoundDistribution(ctx context.Context, c *config.CompoundDistribution) (dist stats.Distribution, distName string, err error)

CompoundDistribution instantiates a compounded distribution from config. When c.N=1, the source distribution is passed through as is.

func DeriveAlpha 

func DeriveAlpha(h *stats.Histogram, mean, MAD float64, c *config.DeriveAlpha) float64

DeriveAlpha estimates the degrees of freedom parameter for a Student's T distribution with the given mean and MAD that most closely corresponds to the sample distribution given as a histogram h.

func DistributionDistance 

func DistributionDistance(h *stats.Histogram, d stats.Distribution, ignoreCounts int) float64

DistributionDistance computes a measure between the sample distribution given by h and an analytical distribution d in xs points corresponding to h's buckets, ignoring the buckets with less than ignoreCounts samples. The leftmost and rightmost buckets are always ignored, as they are catch-all buckets and may not accurately represent the p.d.f. value.

func FindMin 

func FindMin(f func(float64) float64, min, max, epsilon float64, maxIter int) float64

FindMin is a generic search for a function minimum within [min..max] interval. Stop when the search interval is less than epsilon, or the number of iterations exceeds maxIter.

For correct functionality assumes min < max, epsilon > 0, maxIter >= 1, and f to be continuous and monotone around a single minimum in [min..max].

func LeastSquares 

func LeastSquares(xs, ys []float64) (incline float64, intercept float64, err error)

LeastSquares computes 1-D linear regression for Y = incline*X + intercept based on the given data. The number of elements in xs and ys must be the same. It is possible for the incline to be +Inf (when all xs are the same), which is not an error.

func PlotDistribution 

func PlotDistribution(ctx context.Context, dh stats.DistributionWithHistogram, c *config.DistributionPlot, prefix, legend string) error

PlotDistribution dh, specifically its p.d.f. as approximated by dh.Histogram(), and related plots according to the config c.

func PlotScatter 

func PlotScatter(ctx context.Context, xs, ys []float64, c *config.ScatterPlot, prefix, legend, yLabel string) error

PlotScatter plots the unordered points given as xs and ys as a scatter plot, according to the config.

func Prefix 

func Prefix(prefix, s string) string

Prefix adds a space-separated prefix to s, unless prefix is empty.

func Source 

func Source(ctx context.Context, c *config.Source) (iterator.IteratorCloser[LogProfits], error)

Source generates log-profit sequence according to the config. Please remember to close the resulting iterator.

func SourceMap 

func SourceMap[T any](ctx context.Context, c *config.Source, f func([]LogProfits) T) (iterator.IteratorCloser[T], error)

SourceMap generates log-profit sequences according to the config, processes them with f in batches and returns an iterator of f([]LogProfits). The advantage over Source() followed by Map or ParallelMap is that f() is called in the same parallel worker that processes each batch of tickers, thus reducing inter-process communications.

Please remember to close the resulting iterator.

func SourceMapPrices 

func SourceMapPrices[T any](ctx context.Context, c *config.Source, f func([]Prices) T) (iterator.IteratorCloser[T], error)

func Stability 

func Stability(length int, f func(low, high int) float64, c *config.StabilityPlot) []float64

Stability returns a series of deviations of the statistic f over a Timeseries of size `length`, as specified by the config.

Here f computes the statistic for the given range [low..high) (includes low, excludes high).

func UseValues 

func UseValues(ctx context.Context, v Values) context.Context

UseValues injects Values into the context, to be used by AddValue.

Types

type CumulativeStatistic 

type CumulativeStatistic struct {
	Xs          []float64
	Ys          []float64
	Percentiles [][]float64
	Expected    float64 // expected value of the statistic
	// contains filtered or unexported fields
}

CumulativeStatistic tracks the value of a statistic as more samples arrive. It is intended to be plotted as a graph of the statistic as a function of the number of samples.

The idea is to evaluate visually the noisiness of the statistic as the number of samples increase.

func NewCumulativeStatistic 

func NewCumulativeStatistic(cfg *config.CumulativeStatistic) *CumulativeStatistic

NewCumulativeStatistic initializes an empty CumulativeStatistic object.

func (*CumulativeStatistic) AddDirect 

func (c *CumulativeStatistic) AddDirect(y float64)

AddDirect adds y as the direct value of the statistic at the next sample. The caller is responsible for computing the statistic from the current and all of the preceding samples.

func (*CumulativeStatistic) AddToAverage 

func (c *CumulativeStatistic) AddToAverage(y float64)

AddToAverage updates a statistic computed as the average of y(x) values. This is useful e.g. for tracking a mean.

func (*CumulativeStatistic) Map 

func (c *CumulativeStatistic) Map(f func(float64) float64)

Map applies f to all the resulting point values (the statistic and its percentiles).

This is useful e.g. for the standard deviation: accumulate variance as the average of (y - mean)^2, and compute the square root using Map.

func (*CumulativeStatistic) Plot 

func (c *CumulativeStatistic) Plot(ctx context.Context, yLabel, legend string) error

Plot the accumulated statistic values, percentiles and the expected value, as configured.

func (*CumulativeStatistic) SetExpected 

func (c *CumulativeStatistic) SetExpected(y float64)

SetExpected value of the statistic, for visual reference on the graph.

func (*CumulativeStatistic) Skip 

func (c *CumulativeStatistic) Skip()

Skip the next sample from the statistic, but advance the sample and point counters.

type Experiment 

type Experiment interface {
	Prefix(s string) string
	AddValue(ctx context.Context, key, value string) error
	Run(ctx context.Context, cfg config.ExperimentConfig) error
}

Experiment is a generic interface for a single experiment.

type LogProfits 

type LogProfits struct {
	Ticker     string
	Timeseries *stats.Timeseries
}

type Prices 

type Prices struct {
	Ticker string
	Rows   []db.PriceRow
}

type TestExperiment 

type TestExperiment struct {
	// contains filtered or unexported fields
}

TestExperiment is a fake experiment used in tests. Define actual experiments in their own subpackages.

func (*TestExperiment) AddValue 

func (t *TestExperiment) AddValue(ctx context.Context, k, v string) error

func (*TestExperiment) Prefix 

func (t *TestExperiment) Prefix(s string) string

func (*TestExperiment) Run 

func (t *TestExperiment) Run(ctx context.Context, cfg config.ExperimentConfig) error

type Values 

type Values = map[string]string

Values is a key:value map populated by implementations of Experiment to be printed on the terminal at the end of the run. It is typically used to print various values of interest not suitable for graphical plots.

func GetValues 

func GetValues(ctx context.Context) Values

GetValues previously injected by UseValues, or nil.

Source Files

View all Source files

Directories

Path Synopsis
apps
experiments
Package autocorr is an experiment with auto-correlation of log-profit series.
 Package autocorr is an experiment with auto-correlation of log-profit series.
Package beta is an experiment with cross-correlation between stocks.
 Package beta is an experiment with cross-correlation between stocks.
Package config implements configuration schema for all the experiments.
 Package config implements configuration schema for all the experiments.
Package distribution is an experiment plotting distributions of log-profits.
 Package distribution is an experiment plotting distributions of log-profits.
Package hold implements a "buy and hold" experiment.
 Package hold implements a "buy and hold" experiment.
Package portfolio is a tool for analyzing the performance of an existing portfolio, implemented in a form of an "experiment".
 Package portfolio is a tool for analyzing the performance of an existing portfolio, implemented in a form of an "experiment".
Package powerdist is an experiment to study analytical power distributions.
 Package powerdist is an experiment to study analytical power distributions.
Package simulator implements various trading / investment strategies.
 Package simulator implements various trading / investment strategies.
Package trading is an experiment in exploiting volatility for profit.
 Package trading is an experiment in exploiting volatility for profit.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.