simulation

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 18, 2019 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Overview

Package simulation implements a simulation framework for any state machine built on the SDK which utilizes auth.

It is primarily intended for fuzz testing the integration of modules. It will test that the provided operations are interoperable, and that the desired invariants hold. It can additionally be used to detect what the performance benchmarks in the system are, by using benchmarking mode and cpu / mem profiling. If it detects a failure, it provides the entire log of what was ran.

The simulator takes as input: a random seed, the set of operations to run, the invariants to test, and additional parameters to configure how long to run, and misc. parameters that affect simulation speed.

It is intended that every module provides a list of Operations which will randomly create and run a message / tx in a manner that is interesting to fuzz, and verify that the state transition was executed as exported. Each module should additionally provide methods to assert that the desired invariants hold.

Then to perform a randomized simulation, select the set of desired operations, the weightings for each, the invariants you want to test, and how long to run it for. Then run simulation.Simulate! The simulator will handle things like ensuring that validators periodically double signing, or go offline.

Index

Constants

View Source
const (
	BeginBlockEntryKind = "begin_block"
	EndBlockEntryKind   = "end_block"
	MsgEntryKind        = "msg"
	QueuedMsgEntryKind  = "queued_msg"
)

entry kinds for use within OperationEntry

View Source
const (

	// Simulation parameter constants
	SendEnabled              = "send_enabled"
	MaxMemoChars             = "max_memo_characters"
	TxSigLimit               = "tx_sig_limit"
	TxSizeCostPerByte        = "tx_size_cost_per_byte"
	SigVerifyCostED25519     = "sig_verify_cost_ed25519"
	SigVerifyCostSECP256K1   = "sig_verify_cost_secp256k1"
	DepositParamsMinDeposit  = "deposit_params_min_deposit"
	VotingParamsVotingPeriod = "voting_params_voting_period"
	TallyParamsQuorum        = "tally_params_quorum"
	TallyParamsThreshold     = "tally_params_threshold"
	TallyParamsVeto          = "tally_params_veto"
	UnbondingTime            = "unbonding_time"
	MaxValidators            = "max_validators"
	SignedBlocksWindow       = "signed_blocks_window"
	MinSignedPerWindow       = "min_signed_per_window"
	DowntimeJailDuration     = "downtime_jail_duration"
	SlashFractionDoubleSign  = "slash_fraction_double_sign"
	SlashFractionDowntime    = "slash_fraction_downtime"
	InflationRateChange      = "inflation_rate_change"
	Inflation                = "inflation"
	InflationMax             = "inflation_max"
	InflationMin             = "inflation_min"
	GoalBonded               = "goal_bonded"
	CommunityTax             = "community_tax"
	BaseProposerReward       = "base_proposer_reward"
	BonusProposerReward      = "bonus_proposer_reward"
)

nolint

Variables

View Source
var (

	// ModuleParamSimulator defines module parameter value simulators. All
	// values simulated should be within valid acceptable range for the given
	// parameter.
	ModuleParamSimulator = map[string]func(r *rand.Rand) interface{}{
		SendEnabled: func(r *rand.Rand) interface{} {
			return r.Int63n(2) == 0
		},
		MaxMemoChars: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 100, 200))
		},
		TxSigLimit: func(r *rand.Rand) interface{} {
			return uint64(r.Intn(7) + 1)
		},
		TxSizeCostPerByte: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 5, 15))
		},
		SigVerifyCostED25519: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 500, 1000))
		},
		SigVerifyCostSECP256K1: func(r *rand.Rand) interface{} {
			return uint64(RandIntBetween(r, 500, 1000))
		},
		DepositParamsMinDeposit: func(r *rand.Rand) interface{} {
			return sdk.Coins{sdk.NewInt64Coin(sdk.DefaultBondDenom, int64(RandIntBetween(r, 1, 1e3)))}
		},
		VotingParamsVotingPeriod: func(r *rand.Rand) interface{} {
			return time.Duration(RandIntBetween(r, 1, 2*60*60*24*2)) * time.Second
		},
		TallyParamsQuorum: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(RandIntBetween(r, 334, 500)), 3)
		},
		TallyParamsThreshold: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(RandIntBetween(r, 450, 550)), 3)
		},
		TallyParamsVeto: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(RandIntBetween(r, 250, 334)), 3)
		},
		UnbondingTime: func(r *rand.Rand) interface{} {
			return time.Duration(RandIntBetween(r, 60, 60*60*24*3*2)) * time.Second
		},
		MaxValidators: func(r *rand.Rand) interface{} {
			return uint16(r.Intn(250) + 1)
		},
		SignedBlocksWindow: func(r *rand.Rand) interface{} {
			return int64(RandIntBetween(r, 10, 1000))
		},
		MinSignedPerWindow: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(r.Intn(10)), 1)
		},
		DowntimeJailDuration: func(r *rand.Rand) interface{} {
			return time.Duration(RandIntBetween(r, 60, 60*60*24)) * time.Second
		},
		SlashFractionDoubleSign: func(r *rand.Rand) interface{} {
			return sdk.NewDec(1).Quo(sdk.NewDec(int64(r.Intn(50) + 1)))
		},
		SlashFractionDowntime: func(r *rand.Rand) interface{} {
			return sdk.NewDec(1).Quo(sdk.NewDec(int64(r.Intn(200) + 1)))
		},
		InflationRateChange: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(r.Intn(99)), 2)
		},
		Inflation: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(int64(r.Intn(99)), 2)
		},
		InflationMax: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(20, 2)
		},
		InflationMin: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(7, 2)
		},
		GoalBonded: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(67, 2)
		},
		CommunityTax: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(1, 2).Add(sdk.NewDecWithPrec(int64(r.Intn(30)), 2))
		},
		BaseProposerReward: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(1, 2).Add(sdk.NewDecWithPrec(int64(r.Intn(30)), 2))
		},
		BonusProposerReward: func(r *rand.Rand) interface{} {
			return sdk.NewDecWithPrec(1, 2).Add(sdk.NewDecWithPrec(int64(r.Intn(30)), 2))
		},
	}
)

TODO explain transitional matrix usage

Functions

func DeriveRand

func DeriveRand(r *rand.Rand) *rand.Rand

Derive a new rand deterministically from a rand. Unlike rand.New(rand.NewSource(seed)), the result is "more random" depending on the source and state of r. NOTE: not crypto safe.

func GetMemberOfInitialState

func GetMemberOfInitialState(r *rand.Rand, weights []int) int

GetMemberOfInitialState takes an initial array of weights, of size n. It returns a weighted random number in [0,n).

func PeriodicInvariants

func PeriodicInvariants(invariants []sdk.Invariant, period, offset int) []sdk.Invariant

PeriodicInvariants returns an array of wrapped Invariants. Where each invariant function is only executed periodically defined by period and offset.

func RandIntBetween

func RandIntBetween(r *rand.Rand, min, max int) int

RandIntBetween returns a random int between two numbers inclusively.

func RandPositiveInt

func RandPositiveInt(r *rand.Rand, max sdk.Int) (sdk.Int, error)

get a rand positive sdk.Int

func RandStringOfLength

func RandStringOfLength(r *rand.Rand, n int) string

shamelessly copied from https://stackoverflow.com/questions/22892120/how-to-generate-a-random-string-of-a-fixed-length-in-golang#31832326 Generate a random string of a particular length

func RandTimestamp

func RandTimestamp(r *rand.Rand) time.Time

RandTimestamp generates a random timestamp

func RandomAmount

func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int

Generate a random amount Note: The range of RandomAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomDecAmount

func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec

RandomDecAmount generates a random decimal amount Note: The range of RandomDecAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomRequestBeginBlock

func RandomRequestBeginBlock(r *rand.Rand, params Params,
	validators mockValidators, pastTimes []time.Time,
	pastVoteInfos [][]abci.VoteInfo,
	event func(route, op, evResult string), header abci.Header) abci.RequestBeginBlock

RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction

Types

type Account

type Account struct {
	PrivKey crypto.PrivKey
	PubKey  crypto.PubKey
	Address sdk.AccAddress
}

Account contains a privkey, pubkey, address tuple eventually more useful data can be placed in here. (e.g. number of coins)

func RandomAcc

func RandomAcc(r *rand.Rand, accs []Account) Account

RandomAcc pick a random account from an array

func RandomAccounts

func RandomAccounts(r *rand.Rand, n int) []Account

RandomAccounts generates n random accounts

func (Account) Equals

func (acc Account) Equals(acc2 Account) bool

are two accounts equal

type AppParams

type AppParams map[string]json.RawMessage

TODO add description

func (AppParams) GetOrGenerate

func (sp AppParams) GetOrGenerate(cdc *codec.Codec, key string, ptr interface{}, r *rand.Rand, ps ParamSimulator)

GetOrGenerate attempts to get a given parameter by key from the AppParams object. If it exists, it'll be decoded and returned. Otherwise, the provided ParamSimulator is used to generate a random value.

type AppStateFn

type AppStateFn func(
	r *rand.Rand, accs []Account,
) (appState json.RawMessage, accounts []Account, chainId string, genesisTimestamp time.Time)

AppStateFn returns the app state json bytes, the genesis accounts, and the chain identifier

type DummyLogWriter

type DummyLogWriter struct{}

_____________________ dummy log writter

func (*DummyLogWriter) AddEntry

func (lw *DummyLogWriter) AddEntry(_ OperationEntry)

do nothing

func (*DummyLogWriter) PrintLogs

func (lw *DummyLogWriter) PrintLogs()

do nothing

type EventStats

type EventStats map[string]map[string]map[string]int

EventStats defines an object that keeps a tally of each event that has occurred during a simulation.

func NewEventStats

func NewEventStats() EventStats

NewEventStats creates a new empty EventStats object

func (EventStats) ExportJSON

func (es EventStats) ExportJSON(path string)

ExportJSON saves the event stats as a JSON file on a given path

func (EventStats) Print

func (es EventStats) Print(w io.Writer)

Print the event stats in JSON format.

func (EventStats) Tally

func (es EventStats) Tally(route, op, evResult string)

Tally increases the count of a simulation event.

type FutureOperation

type FutureOperation struct {
	BlockHeight int
	BlockTime   time.Time
	Op          Operation
}

FutureOperation is an operation which will be ran at the beginning of the provided BlockHeight. If both a BlockHeight and BlockTime are specified, it will use the BlockHeight. In the (likely) event that multiple operations are queued at the same block height, they will execute in a FIFO pattern.

type LogWriter

type LogWriter interface {
	AddEntry(OperationEntry)
	PrintLogs()
}

log writter

func NewLogWriter

func NewLogWriter(testingmode bool) LogWriter

LogWriter - return a dummy or standard log writer given the testingmode

type Operation

type Operation func(r *rand.Rand, app *baseapp.BaseApp,
	ctx sdk.Context, accounts []Account) (
	OperationMsg OperationMsg, futureOps []FutureOperation, err error)

Operation runs a state machine transition, and ensures the transition happened as exported. The operation could be running and testing a fuzzed transaction, or doing the same for a message.

For ease of debugging, an operation returns a descriptive message "action", which details what this fuzzed state machine transition actually did.

Operations can optionally provide a list of "FutureOperations" to run later These will be ran at the beginning of the corresponding block.

type OperationEntry

type OperationEntry struct {
	EntryKind string          `json:"entry_kind" yaml:"entry_kind"`
	Height    int64           `json:"height" yaml:"height"`
	Order     int64           `json:"order" yaml:"order"`
	Operation json.RawMessage `json:"operation" yaml:"operation"`
}

OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)

func BeginBlockEntry

func BeginBlockEntry(height int64) OperationEntry

BeginBlockEntry - operation entry for begin block

func EndBlockEntry

func EndBlockEntry(height int64) OperationEntry

EndBlockEntry - operation entry for end block

func MsgEntry

func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry

MsgEntry - operation entry for standard msg

func NewOperationEntry

func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry

NewOperationEntry creates a new OperationEntry instance

func QueuedMsgEntry

func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry

QueuedMsgEntry creates an operation entry for a given queued message.

func (OperationEntry) MustMarshal

func (oe OperationEntry) MustMarshal() json.RawMessage

MustMarshal marshals the operation entry, panic on error.

type OperationMsg

type OperationMsg struct {
	Route   string          `json:"route" yaml:"route"`
	Name    string          `json:"name" yaml:"name"`
	Comment string          `json:"comment" yaml:"comment"`
	OK      bool            `json:"ok" yaml:"ok"`
	Msg     json.RawMessage `json:"msg" yaml:"msg"`
}

OperationMsg - structure for operation output

func NewOperationMsg

func NewOperationMsg(msg sdk.Msg, ok bool, comment string) OperationMsg

NewOperationMsg - create a new operation message from sdk.Msg

func NewOperationMsgBasic

func NewOperationMsgBasic(route, name, comment string, ok bool, msg []byte) OperationMsg

NewOperationMsgBasic creates a new operation message from raw input.

func NoOpMsg

func NoOpMsg(route string) OperationMsg

NoOpMsg - create a no-operation message

func (OperationMsg) LogEvent

func (om OperationMsg) LogEvent(eventLogger func(route, op, evResult string))

LogEvent adds an event for the events stats

func (OperationMsg) MustMarshal

func (om OperationMsg) MustMarshal() json.RawMessage

MustMarshal Marshals the operation msg, panic on error

func (OperationMsg) String

func (om OperationMsg) String() string

log entry text for this operation msg

type OperationQueue

type OperationQueue map[int][]Operation

OperationQueue defines an object for a queue of operations

func NewOperationQueue

func NewOperationQueue() OperationQueue

NewOperationQueue creates a new OperationQueue instance.

type ParamSimulator

type ParamSimulator func(r *rand.Rand)

TODO add description

type Params

type Params struct {
	PastEvidenceFraction      float64
	NumKeys                   int
	EvidenceFraction          float64
	InitialLivenessWeightings []int
	LivenessTransitionMatrix  TransitionMatrix
	BlockSizeTransitionMatrix TransitionMatrix
}

Simulation parameters

func RandomParams

func RandomParams(r *rand.Rand) Params

Return random simulation parameters

func SimulateFromSeed

func SimulateFromSeed(
	tb testing.TB, w io.Writer, app *baseapp.BaseApp,
	appStateFn AppStateFn, seed int64,
	ops WeightedOperations, invariants sdk.Invariants,
	initialHeight, numBlocks, exportParamsHeight, blockSize int,
	exportStatsPath string,
	exportParams, commit, lean, onOperation, allInvariants bool,
	blackListedAccs map[string]bool,
) (stopEarly bool, exportedParams Params, err error)

SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided seed. TODO: split this monster function up

type StandardLogWriter

type StandardLogWriter struct {
	OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
}

log writter

func (*StandardLogWriter) AddEntry

func (lw *StandardLogWriter) AddEntry(opEntry OperationEntry)

add an entry to the log writter

func (*StandardLogWriter) PrintLogs

func (lw *StandardLogWriter) PrintLogs()

PrintLogs - print the logs to a simulation file

type TransitionMatrix

type TransitionMatrix struct {
	// contains filtered or unexported fields
}

TransitionMatrix is _almost_ a left stochastic matrix. It is technically not one due to not normalizing the column values. In the future, if we want to find the steady state distribution, it will be quite easy to normalize these values to get a stochastic matrix. Floats aren't currently used as the default due to non-determinism across architectures

func CreateTransitionMatrix

func CreateTransitionMatrix(weights [][]int) (TransitionMatrix, error)

CreateTransitionMatrix creates a transition matrix from the provided weights. TODO: Provide example usage

func (TransitionMatrix) NextState

func (t TransitionMatrix) NextState(r *rand.Rand, i int) int

NextState returns the next state randomly chosen using r, and the weightings provided in the transition matrix.

type WeightedOperation

type WeightedOperation struct {
	Weight int
	Op     Operation
}

WeightedOperation is an operation with associated weight. This is used to bias the selection operation within the simulator.

type WeightedOperations

type WeightedOperations []WeightedOperation

WeightedOperations is the group of all weighted operations to simulate.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL