reconciler

package
v0.0.0-...-02d6beb Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 10, 2021 License: Apache-2.0 Imports: 13 Imported by: 0

README

Reconciler

GoDoc

The Reconciler package is used to ensure that balance changes derived from parsing Rosetta blocks are equivalent to the balance changes computed by the node. If you want to see an example of how to use this package, take a look at rosetta-cli.

Features

  • Customizable Helper and Handler to define your own logic for retrieving calculated balance changes and handling successful/unsuccessful comparisons
  • Perform balance lookup at either historical blocks or the current block (if historical balance query is not supported)
  • Provide a list of accounts to compare at each block (for quick and easy debugging)

Installation

go get github.com/keninqiu/rosetta-sdk-go/reconciler

Reconciliation Strategies

Active Addresses

The reconciler checks that the balance of an account computed by its operations is equal to the balance of the account according to the node. If this balance is not identical, the reconciler will error.

Inactive Addresses

The reconciler randomly checks the balances of accounts that aren't involved in any transactions. The balances of accounts could change on the blockchain node without being included in an operation returned by the Rosetta Data API. Recall that all balance-changing operations must be returned by the Rosetta Data API.

Documentation

Index

Constants

View Source 
const (
	// ActiveReconciliation is included in the reconciliation
	// error message if reconciliation failed during active
	// reconciliation.
	ActiveReconciliation = "ACTIVE"

	// InactiveReconciliation is included in the reconciliation
	// error message if reconciliation failed during inactive
	// reconciliation.
	InactiveReconciliation = "INACTIVE"
)
View Source 
const (
	// BlockGone is when the block where a reconciliation
	// is supposed to happen is orphaned.
	BlockGone = "BLOCK_GONE"

	// HeadBehind is when the synced tip (where balances
	// were last computed) is behind the *types.BlockIdentifier
	// returned by the call to /account/balance.
	HeadBehind = "HEAD_BEHIND"

	// BacklogFull is when the reconciliation backlog is full.
	BacklogFull = "BACKLOG_FULL"

	// TipFailure is returned when looking up the live
	// balance fails but we are at tip. This usually occurs
	// when the node processes an re-org that we have yet
	// to process (so the index we are querying at may be
	// ahead of the nodes tip).
	TipFailure = "TIP_FAILURE"

	// AccountMissing is returned when looking up computed
	// balance fails because the account does not exist in
	// balance storage.
	// This can happen when interesting accounts
	// are specified. We try to reconcile balances for
	// each of these accounts at each block height.
	// But, until we encounter a block with an interesting account
	// in it, there is no entry for it in balance storage.
	// So, we can not reconcile.
	AccountMissing = "ACCOUNT_MISSING"
)

Variables

View Source 
var (
	// ErrHeadBlockBehindLive is returned when the processed
	// head is behind the live head. Sometimes, it is
	// preferable to sleep and wait to catch up when
	// we are close to the live head (waitToCheckDiff).
	ErrHeadBlockBehindLive = errors.New("head block behind")

	// ErrBlockGone is returned when the processed block
	// head is greater than the live head but the block
	// does not exist in the store. This likely means
	// that the block was orphaned.
	ErrBlockGone = errors.New("block gone")

	ErrGetCurrentBlockFailed    = errors.New("unable to get current block for reconciliation")
	ErrBlockExistsFailed        = errors.New("unable to check if block exists")
	ErrGetComputedBalanceFailed = errors.New("unable to get computed balance")
	ErrLiveBalanceLookupFailed  = errors.New("unable to lookup live balance")
)

Named error types for Reconciler errors

Functions

func ContainsAccountCurrency 

func ContainsAccountCurrency(
	m map[string]struct{},
	change *types.AccountCurrency,
) bool

ContainsAccountCurrency returns a boolean indicating if a AccountCurrency set already contains an Account and Currency combination.

func Err 

func Err(err error) bool

Err takes an error as an argument and returns whether or not the error is one thrown by the reconciler package

Types

type Handler 

type Handler interface {
	ReconciliationFailed(
		ctx context.Context,
		reconciliationType string,
		account *types.AccountIdentifier,
		currency *types.Currency,
		computedBalance string,
		liveBalance string,
		block *types.BlockIdentifier,
	) error

	ReconciliationSucceeded(
		ctx context.Context,
		reconciliationType string,
		account *types.AccountIdentifier,
		currency *types.Currency,
		balance string,
		block *types.BlockIdentifier,
	) error

	ReconciliationExempt(
		ctx context.Context,
		reconciliationType string,
		account *types.AccountIdentifier,
		currency *types.Currency,
		computedBalance string,
		liveBalance string,
		block *types.BlockIdentifier,
		exemption *types.BalanceExemption,
	) error

	ReconciliationSkipped(
		ctx context.Context,
		reconciliationType string,
		account *types.AccountIdentifier,
		currency *types.Currency,
		cause string,
	) error
}

Handler is called by Reconciler after a reconciliation is performed. When a reconciliation failure is observed, it is up to the client to trigger a halt (by returning an error) or to continue (by returning nil).

type Helper 

type Helper interface {
	DatabaseTransaction(ctx context.Context) database.Transaction

	CurrentBlock(
		ctx context.Context,
		dbTx database.Transaction,
	) (*types.BlockIdentifier, error)

	IndexAtTip(
		ctx context.Context,
		index int64,
	) (bool, error)

	CanonicalBlock(
		ctx context.Context,
		dbTx database.Transaction,
		block *types.BlockIdentifier,
	) (bool, error)

	ComputedBalance(
		ctx context.Context,
		dbTx database.Transaction,
		account *types.AccountIdentifier,
		currency *types.Currency,
		index int64,
	) (*types.Amount, error)

	LiveBalance(
		ctx context.Context,
		account *types.AccountIdentifier,
		currency *types.Currency,
		index int64,
	) (*types.Amount, *types.BlockIdentifier, error)

	// PruneBalances is invoked by the reconciler
	// to indicate that all historical balance states
	// <= to index can be removed.
	PruneBalances(
		ctx context.Context,
		account *types.AccountIdentifier,
		currency *types.Currency,
		index int64,
	) error

	// ForceInactiveReconciliation is invoked by the
	// inactive reconciler when the next account to
	// reconcile has been checked within the configured
	// inactive reconciliation frequency. This allows
	// the helper to dynamically force inactive reconciliation
	// when desired (i.e. when at tip).
	ForceInactiveReconciliation(
		ctx context.Context,
		account *types.AccountIdentifier,
		currency *types.Currency,
		lastCheck *types.BlockIdentifier,
	) bool
}

Helper functions are used by Reconciler to compare computed balances from a block with the balance calculated by the node. Defining an interface allows the client to determine what sort of storage layer they want to use to provide the required information.

type InactiveEntry 

type InactiveEntry struct {
	Entry     *types.AccountCurrency
	LastCheck *types.BlockIdentifier
}

InactiveEntry is used to track the last time that an *types.AccountCurrency was reconciled.

type Option 

type Option func(r *Reconciler)

Option is used to overwrite default values in Reconciler construction. Any Option not provided falls back to the default value.

func WithActiveConcurrency 

func WithActiveConcurrency(concurrency int) Option

WithActiveConcurrency overrides the default active concurrency.

func WithBacklogSize 

func WithBacklogSize(size int) Option

WithBacklogSize overrides the defaultBacklogSize to some new size. This is often useful for blockchains that have high network activity.

func WithBalancePruning 

func WithBalancePruning() Option

WithBalancePruning determines if historical balance states should be pruned after they are used. This can prevent storage blowup if historical states are only ever used once.

func WithDebugLogging 

func WithDebugLogging() Option

WithDebugLogging determines if verbose logs should be printed.

func WithInactiveConcurrency 

func WithInactiveConcurrency(concurrency int) Option

WithInactiveConcurrency overrides the default inactive concurrency.

func WithInactiveFrequency 

func WithInactiveFrequency(blocks int64) Option

WithInactiveFrequency is how many blocks the reconciler should wait between inactive reconciliations on each account.

func WithInterestingAccounts 

func WithInterestingAccounts(interesting []*types.AccountCurrency) Option

WithInterestingAccounts adds interesting accounts to check at each block.

func WithLookupBalanceByBlock 

func WithLookupBalanceByBlock() Option

WithLookupBalanceByBlock sets lookupBlockByBalance to false.

func WithSeenAccounts 

func WithSeenAccounts(seen []*types.AccountCurrency) Option

WithSeenAccounts adds accounts to the seenAccounts slice and inactiveQueue for inactive reconciliation.

type Reconciler 

type Reconciler struct {

	// Reconciler concurrency is separated between
	// active and inactive concurrency to allow for
	// fine-grained tuning of reconciler behavior.
	// When there are many transactions in a block
	// on a resource-constrained machine (laptop),
	// it is useful to allocate more resources to
	// active reconciliation as it is synchronous
	// (when lookupBalanceByBlock is enabled).
	ActiveConcurrency   int
	InactiveConcurrency int
	// contains filtered or unexported fields
}

Reconciler contains all logic to reconcile balances of types.AccountIdentifiers returned in types.Operations by a Rosetta Server.

func New 

func New(
	helper Helper,
	handler Handler,
	p *parser.Parser,
	options ...Option,
) *Reconciler

New creates a new Reconciler.

func (*Reconciler) CompareBalance 

func (r *Reconciler) CompareBalance(
	ctx context.Context,
	account *types.AccountIdentifier,
	currency *types.Currency,
	liveBalance string,
	liveBlock *types.BlockIdentifier,
) (string, string, int64, error)

CompareBalance checks to see if the computed balance of an account is equal to the live balance of an account. This function ensures balance is checked correctly in the case of orphaned blocks.

We must transactionally fetch the last synced head, whether the liveBlock is canonical, and the computed balance of the *types.AccountIdentifier and *types.Currency.

func (*Reconciler) LastIndexReconciled 

func (r *Reconciler) LastIndexReconciled() int64

LastIndexReconciled is the last block index reconciled. This is used to ensure all the enqueued accounts for a particular block have been reconciled.

func (*Reconciler) QueueChanges 

func (r *Reconciler) QueueChanges(
	ctx context.Context,
	block *types.BlockIdentifier,
	balanceChanges []*parser.BalanceChange,
) error

QueueChanges enqueues a slice of *BalanceChanges for reconciliation.

func (*Reconciler) QueueSize 

func (r *Reconciler) QueueSize() int

QueueSize is a helper that returns the total number of items currently enqueued for active reconciliation.

func (*Reconciler) Reconcile 

func (r *Reconciler) Reconcile(ctx context.Context) error

Reconcile starts the active and inactive Reconciler goroutines. If any goroutine errors, the function will return an error.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.