cosmos-sdk: github.com/cosmos/cosmos-sdk/x/simulation Index | Files

package simulation

import "github.com/cosmos/cosmos-sdk/x/simulation"

Package simulation implements a full fledged Cosmos SDK application used for executing simulation test suites.

Simulation App

The SimApp type defines an application used for running extensive simulation testing suites. It contains all core modules, including governance, staking, slashing, and distribution.

Simulation is executed with various inputs including the number of blocks to simulate, the block size, whether the app should commit or not, the invariant checking period, and a seed which is used as a source of pseudo-randomness.

In addition to the various inputs, simulation runs mainly in three modes:

1. Completely random where the initial state, module parameters and simulation parameters are pseudo-randomly generated.

2. From a genesis file where the initial state and the module parameters are defined. This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested.

3. From a params file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated.

The simulation test suite also supports testing determinism and import/export functionality.

Randomness

Currently, simulation uses a single seed (integer) as a source for a PRNG by which all random operations are executed from. Any call to the PRNG changes all future operations as the internal state of the PRNG is modified. For example, if a new message type is created and needs to be simulated, the new introduced PRNG call will change all subsequent operations.

This may can often be problematic when testing fixes to simulation faults. One current solution to this is to use a params file as mentioned above. In the future the simulation suite is expected to support a series of PRNGs that can be used uniquely per module and simulation component so that they will not effect each others state execution outcome.

Usage

To execute a completely pseudo-random simulation:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-v -timeout 24h

To execute simulation from a genesis file:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-Genesis=/path/to/genesis.json \
 	-v -timeout 24h

To execute simulation from a simulation params file:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
	-run=TestFullAppSimulation \
	-Enabled=true \
	-NumBlocks=100 \
	-BlockSize=200 \
	-Commit=true \
	-Seed=99 \
	-Period=5 \
	-Params=/path/to/params.json \
	-v -timeout 24h

To export the simulation params to a file at a given block height:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportParamsPath=/path/to/params.json \
	-ExportParamsHeight=50 \
	 -v -timeout 24h

To export the simulation app state (i.e genesis) to a file:

 $ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
 	-run=TestFullAppSimulation \
 	-Enabled=true \
 	-NumBlocks=100 \
 	-BlockSize=200 \
 	-Commit=true \
 	-Seed=99 \
 	-Period=5 \
	-ExportStatePath=/path/to/genesis.json \
	 v -timeout 24h

Params

Params that are provided to simulation from a JSON file are used to used to set both module parameters and simulation parameters. See sim_test.go for the full set of parameters that can be provided.

Index

Package Files

account.go config.go doc.go event_stats.go log.go mock_tendermint.go operation.go params.go rand_util.go simulate.go transition_matrix.go util.go

Constants

const (
    BeginBlockEntryKind = "begin_block"
    EndBlockEntryKind   = "end_block"
    MsgEntryKind        = "msg"
    QueuedMsgEntryKind  = "queued_msg"
)

entry kinds for use within OperationEntry

func DeriveRand Uses

func DeriveRand(r *rand.Rand) *rand.Rand

DeriveRand derives a new Rand deterministically from another random source. Unlike rand.New(rand.NewSource(seed)), the result is "more random" depending on the source and state of r.

NOTE: not crypto safe.

func GetMemberOfInitialState Uses

func GetMemberOfInitialState(r *rand.Rand, weights []int) int

GetMemberOfInitialState takes an initial array of weights, of size n. It returns a weighted random number in [0,n).

func RandIntBetween Uses

func RandIntBetween(r *rand.Rand, min, max int) int

RandIntBetween returns a random int between two numbers inclusively.

func RandPositiveInt Uses

func RandPositiveInt(r *rand.Rand, max sdk.Int) (sdk.Int, error)

RandPositiveInt get a rand positive sdk.Int

func RandStringOfLength Uses

func RandStringOfLength(r *rand.Rand, n int) string

RandStringOfLength generates a random string of a particular length

func RandSubsetCoins Uses

func RandSubsetCoins(r *rand.Rand, coins sdk.Coins) sdk.Coins

returns random subset of the provided coins will return at least one coin unless coins argument is empty or malformed i.e. 0 amt in coins

func RandTimestamp Uses

func RandTimestamp(r *rand.Rand) time.Time

RandTimestamp generates a random timestamp

func RandomAmount Uses

func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int

RandomAmount generates a random amount Note: The range of RandomAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomDecAmount Uses

func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec

RandomDecAmount generates a random decimal amount Note: The range of RandomDecAmount includes max, and is, in fact, biased to return max as well as 0.

func RandomFees Uses

func RandomFees(r *rand.Rand, ctx sdk.Context, spendableCoins sdk.Coins) (sdk.Coins, error)

RandomFees returns a random fee by selecting a random coin denomination and amount from the account's available balance. If the user doesn't have enough funds for paying fees, it returns empty coins.

func RandomRequestBeginBlock Uses

func RandomRequestBeginBlock(r *rand.Rand, params Params,
    validators mockValidators, pastTimes []time.Time,
    pastVoteInfos [][]abci.VoteInfo,
    event func(route, op, evResult string), header abci.Header) abci.RequestBeginBlock

RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction

type Account Uses

type Account struct {
    PrivKey crypto.PrivKey
    PubKey  crypto.PubKey
    Address sdk.AccAddress
}

Account contains a privkey, pubkey, address tuple eventually more useful data can be placed in here. (e.g. number of coins)

func FindAccount Uses

func FindAccount(accs []Account, address sdk.Address) (Account, bool)

FindAccount iterates over all the simulation accounts to find the one that matches the given address

func RandomAcc Uses

func RandomAcc(r *rand.Rand, accs []Account) (Account, int)

RandomAcc picks and returns a random account from an array and returs its position in the array.

func RandomAccounts Uses

func RandomAccounts(r *rand.Rand, n int) []Account

RandomAccounts generates n random accounts

func (Account) Equals Uses

func (acc Account) Equals(acc2 Account) bool

Equals returns true if two accounts are equal

type AppParams Uses

type AppParams map[string]json.RawMessage

AppParams defines a flat JSON of key/values for all possible configurable simulation parameters. It might contain: operation weights, simulation parameters and flattened module state parameters (i.e not stored under it's respective module name).

func (AppParams) GetOrGenerate Uses

func (sp AppParams) GetOrGenerate(cdc *codec.Codec, key string, ptr interface{}, r *rand.Rand, ps ParamSimulator)

GetOrGenerate attempts to get a given parameter by key from the AppParams object. If it exists, it'll be decoded and returned. Otherwise, the provided ParamSimulator is used to generate a random value or default value (eg: in the case of operation weights where Rand is not used).

type AppStateFn Uses

type AppStateFn func(r *rand.Rand, accs []Account, config Config) (
    appState json.RawMessage, accounts []Account, chainId string, genesisTimestamp time.Time,
)

AppStateFn returns the app state json bytes and the genesis accounts

type Config Uses

type Config struct {
    GenesisFile string // custom simulation genesis file; cannot be used with params file
    ParamsFile  string // custom simulation params file which overrides any random params; cannot be used with genesis

    ExportParamsPath   string // custom file path to save the exported params JSON
    ExportParamsHeight int    //height to which export the randomly generated params
    ExportStatePath    string //custom file path to save the exported app state JSON
    ExportStatsPath    string // custom file path to save the exported simulation statistics JSON

    Seed               int64  // simulation random seed
    InitialBlockHeight int    // initial block to start the simulation
    NumBlocks          int    // number of new blocks to simulate from the initial block height
    BlockSize          int    // operations per block
    ChainID            string // chain-id used on the simulation

    Lean   bool // lean simulation log output
    Commit bool // have the simulation commit

    OnOperation   bool // run slow invariants every operation
    AllInvariants bool // print all failed invariants if a broken invariant is found
}

Config contains the necessary configuration flags for the simulator

type ContentSimulatorFn Uses

type ContentSimulatorFn func(r *rand.Rand, ctx sdk.Context, accs []Account) govtypes.Content

ContentSimulatorFn defines a function type alias for generating random proposal content.

type DummyLogWriter Uses

type DummyLogWriter struct{}

_____________________ dummy log writter

func (*DummyLogWriter) AddEntry Uses

func (lw *DummyLogWriter) AddEntry(_ OperationEntry)

do nothing

func (*DummyLogWriter) PrintLogs Uses

func (lw *DummyLogWriter) PrintLogs()

do nothing

type EventStats Uses

type EventStats map[string]map[string]map[string]int

EventStats defines an object that keeps a tally of each event that has occurred during a simulation.

func NewEventStats Uses

func NewEventStats() EventStats

NewEventStats creates a new empty EventStats object

func (EventStats) ExportJSON Uses

func (es EventStats) ExportJSON(path string)

ExportJSON saves the event stats as a JSON file on a given path

func (EventStats) Print Uses

func (es EventStats) Print(w io.Writer)

Print the event stats in JSON format.

func (EventStats) Tally Uses

func (es EventStats) Tally(route, op, evResult string)

Tally increases the count of a simulation event.

type FutureOperation Uses

type FutureOperation struct {
    BlockHeight int
    BlockTime   time.Time
    Op          Operation
}

FutureOperation is an operation which will be ran at the beginning of the provided BlockHeight. If both a BlockHeight and BlockTime are specified, it will use the BlockHeight. In the (likely) event that multiple operations are queued at the same block height, they will execute in a FIFO pattern.

type LogWriter Uses

type LogWriter interface {
    AddEntry(OperationEntry)
    PrintLogs()
}

log writter

func NewLogWriter Uses

func NewLogWriter(testingmode bool) LogWriter

LogWriter - return a dummy or standard log writer given the testingmode

type Operation Uses

type Operation func(r *rand.Rand, app *baseapp.BaseApp,
    ctx sdk.Context, accounts []Account, chainID string) (
    OperationMsg OperationMsg, futureOps []FutureOperation, err error)

Operation runs a state machine transition, and ensures the transition happened as expected. The operation could be running and testing a fuzzed transaction, or doing the same for a message.

For ease of debugging, an operation returns a descriptive message "action", which details what this fuzzed state machine transition actually did.

Operations can optionally provide a list of "FutureOperations" to run later These will be ran at the beginning of the corresponding block.

type OperationEntry Uses

type OperationEntry struct {
    EntryKind string          `json:"entry_kind" yaml:"entry_kind"`
    Height    int64           `json:"height" yaml:"height"`
    Order     int64           `json:"order" yaml:"order"`
    Operation json.RawMessage `json:"operation" yaml:"operation"`
}

OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)

func BeginBlockEntry Uses

func BeginBlockEntry(height int64) OperationEntry

BeginBlockEntry - operation entry for begin block

func EndBlockEntry Uses

func EndBlockEntry(height int64) OperationEntry

EndBlockEntry - operation entry for end block

func MsgEntry Uses

func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry

MsgEntry - operation entry for standard msg

func NewOperationEntry Uses

func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry

NewOperationEntry creates a new OperationEntry instance

func QueuedMsgEntry Uses

func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry

QueuedMsgEntry creates an operation entry for a given queued message.

func (OperationEntry) MustMarshal Uses

func (oe OperationEntry) MustMarshal() json.RawMessage

MustMarshal marshals the operation entry, panic on error.

type OperationMsg Uses

type OperationMsg struct {
    Route   string          `json:"route" yaml:"route"`     // msg route (i.e module name)
    Name    string          `json:"name" yaml:"name"`       // operation name (msg Type or "no-operation")
    Comment string          `json:"comment" yaml:"comment"` // additional comment
    OK      bool            `json:"ok" yaml:"ok"`           // success
    Msg     json.RawMessage `json:"msg" yaml:"msg"`         // JSON encoded msg
}

OperationMsg - structure for operation output

func NewOperationMsg Uses

func NewOperationMsg(msg sdk.Msg, ok bool, comment string) OperationMsg

NewOperationMsg - create a new operation message from sdk.Msg

func NewOperationMsgBasic Uses

func NewOperationMsgBasic(route, name, comment string, ok bool, msg []byte) OperationMsg

NewOperationMsgBasic creates a new operation message from raw input.

func NoOpMsg Uses

func NoOpMsg(route string) OperationMsg

NoOpMsg - create a no-operation message

func (OperationMsg) LogEvent Uses

func (om OperationMsg) LogEvent(eventLogger func(route, op, evResult string))

LogEvent adds an event for the events stats

func (OperationMsg) MustMarshal Uses

func (om OperationMsg) MustMarshal() json.RawMessage

MustMarshal Marshals the operation msg, panic on error

func (OperationMsg) String Uses

func (om OperationMsg) String() string

log entry text for this operation msg

type OperationQueue Uses

type OperationQueue map[int][]Operation

OperationQueue defines an object for a queue of operations

func NewOperationQueue Uses

func NewOperationQueue() OperationQueue

NewOperationQueue creates a new OperationQueue instance.

type ParamChange Uses

type ParamChange struct {
    Subspace string
    Key      string
    SimValue SimValFn
}

ParamChange defines the object used for simulating parameter change proposals

func NewSimParamChange Uses

func NewSimParamChange(subspace, key string, simVal SimValFn) ParamChange

NewSimParamChange creates a new ParamChange instance

func (ParamChange) ComposedKey Uses

func (spc ParamChange) ComposedKey() string

ComposedKey creates a new composed key for the param change proposal

type ParamSimulator Uses

type ParamSimulator func(r *rand.Rand)

ParamSimulator creates a parameter value from a source of random number

type Params Uses

type Params struct {
    PastEvidenceFraction      float64
    NumKeys                   int
    EvidenceFraction          float64
    InitialLivenessWeightings []int
    LivenessTransitionMatrix  TransitionMatrix
    BlockSizeTransitionMatrix TransitionMatrix
}

Params define the parameters necessary for running the simulations

func RandomParams Uses

func RandomParams(r *rand.Rand) Params

RandomParams returns random simulation parameters

func SimulateFromSeed Uses

func SimulateFromSeed(
    tb testing.TB, w io.Writer, app *baseapp.BaseApp,
    appStateFn AppStateFn, ops WeightedOperations,
    blackListedAccs map[string]bool, config Config,
) (stopEarly bool, exportedParams Params, err error)

SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided config.Seed. TODO: split this monster function up

type SimValFn Uses

type SimValFn func(r *rand.Rand) string

SimValFn function to generate the randomized parameter change value

type StandardLogWriter Uses

type StandardLogWriter struct {
    OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
}

log writter

func (*StandardLogWriter) AddEntry Uses

func (lw *StandardLogWriter) AddEntry(opEntry OperationEntry)

add an entry to the log writter

func (*StandardLogWriter) PrintLogs Uses

func (lw *StandardLogWriter) PrintLogs()

PrintLogs - print the logs to a simulation file

type TransitionMatrix Uses

type TransitionMatrix struct {
    // contains filtered or unexported fields
}

TransitionMatrix is _almost_ a left stochastic matrix. It is technically not one due to not normalizing the column values. In the future, if we want to find the steady state distribution, it will be quite easy to normalize these values to get a stochastic matrix. Floats aren't currently used as the default due to non-determinism across architectures

func CreateTransitionMatrix Uses

func CreateTransitionMatrix(weights [][]int) (TransitionMatrix, error)

CreateTransitionMatrix creates a transition matrix from the provided weights. TODO: Provide example usage

func (TransitionMatrix) NextState Uses

func (t TransitionMatrix) NextState(r *rand.Rand, i int) int

NextState returns the next state randomly chosen using r, and the weightings provided in the transition matrix.

type WeightedOperation Uses

type WeightedOperation struct {
    Weight int
    Op     Operation
}

WeightedOperation is an operation with associated weight. This is used to bias the selection operation within the simulator.

func NewWeightedOperation Uses

func NewWeightedOperation(weight int, op Operation) WeightedOperation

NewWeightedOperation creates a new WeightedOperation instance

type WeightedOperations Uses

type WeightedOperations []WeightedOperation

WeightedOperations is the group of all weighted operations to simulate.

type WeightedProposalContent Uses

type WeightedProposalContent struct {
    AppParamsKey       string             // key used to retrieve the value of the weight from the simulation application params
    DefaultWeight      int                // default weight
    ContentSimulatorFn ContentSimulatorFn // content simulator function
}

WeightedProposalContent defines a common struct for proposal contents defined by external modules (i.e outside gov)

Package simulation imports 23 packages (graph) and is imported by 35 packages. Updated 2019-12-12. Refresh now. Tools for package owners.