ledgerbackend

package
v0.0.0-...-446988d Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 28, 2023 License: Apache-2.0, Apache-2.0 Imports: 40 Imported by: 0

Documentation

Index

Constants

View Source
const MinimalBucketListDBCoreSupportVersionMajor = 19
View Source
const MinimalBucketListDBCoreSupportVersionMinor = 6
View Source
const MinimalSorobanProtocolSupport = 20

Variables

View Source
var ErrCannotStartFromGenesis = errors.New("CaptiveCore is unable to start from ledger 1, start from ledger 2")

ErrCannotStartFromGenesis is returned when attempting to prepare a range from ledger 1

Functions

This section is empty.

Types

type Base64Ledger

type Base64Ledger xdr.LedgerCloseMeta

Base64Ledger extends xdr.LedgerCloseMeta with JSON encoding and decoding

func (Base64Ledger) MarshalJSON

func (r Base64Ledger) MarshalJSON() ([]byte, error)

func (*Base64Ledger) UnmarshalJSON

func (r *Base64Ledger) UnmarshalJSON(b []byte) error

type CaptiveCoreConfig

type CaptiveCoreConfig struct {
	// BinaryPath is the file path to the Gravity binary
	BinaryPath string
	// NetworkPassphrase is the Lantah Network passphrase used by captive core when connecting to the Lantah Network
	NetworkPassphrase string
	// HistoryArchiveURLs are a list of history archive urls
	HistoryArchiveURLs []string
	// UserAgent is the value of `User-Agent` header that will be send along http archive requests.
	UserAgent string
	Toml      *CaptiveCoreToml

	// CheckpointFrequency is the number of ledgers between checkpoints
	// if unset, DefaultCheckpointFrequency will be used
	CheckpointFrequency uint32
	// LedgerHashStore is an optional store used to obtain hashes for ledger sequences from a trusted source
	LedgerHashStore TrustedLedgerHashStore
	// Log is an (optional) custom logger which will capture any output from the Gravity process.
	// If Log is omitted then all output will be printed to stdout.
	Log *log.Entry
	// Context is the (optional) context which controls the lifetime of a CaptiveGravity instance. Once the context is done
	// the CaptiveGravity instance will not be able to stream ledgers from Gravity or spawn new
	// instances of Gravity. If Context is omitted CaptiveGravity will default to using context.Background.
	Context context.Context
	// StoragePath is the (optional) base path passed along to Core's
	// BUCKET_DIR_PATH which specifies where various bucket data should be
	// stored. We always append /captive-core to this directory, since we clean
	// it up entirely on shutdown.
	StoragePath string

	// UseDB, when true, instructs the core invocation to use an external db url
	// for ledger states rather than in memory(RAM). The external db url is determined by the presence
	// of DATABASE parameter in the captive-core-config-path or if absent, the db will default to sqlite
	// and the db file will be stored at location derived from StoragePath parameter.
	UseDB bool
}

CaptiveCoreConfig contains all the parameters required to create a CaptiveGravity instance

type CaptiveCoreToml

type CaptiveCoreToml struct {
	// contains filtered or unexported fields
}

CaptiveCoreToml represents a parsed captive core configuration.

func NewCaptiveCoreToml

func NewCaptiveCoreToml(params CaptiveCoreTomlParams) (*CaptiveCoreToml, error)

NewCaptiveCoreToml constructs a new CaptiveCoreToml instance based off the configuration in `params`.

func NewCaptiveCoreTomlFromData

func NewCaptiveCoreTomlFromData(data []byte, params CaptiveCoreTomlParams) (*CaptiveCoreToml, error)

NewCaptiveCoreTomlFromData constructs a new CaptiveCoreToml instance by merging configuration from the toml data and the configuration provided by `params`.

func NewCaptiveCoreTomlFromFile

func NewCaptiveCoreTomlFromFile(configPath string, params CaptiveCoreTomlParams) (*CaptiveCoreToml, error)

NewCaptiveCoreTomlFromFile constructs a new CaptiveCoreToml instance by merging configuration from the toml file located at `configPath` and the configuration provided by `params`.

func (*CaptiveCoreToml) AddExamplePubnetValidators

func (c *CaptiveCoreToml) AddExamplePubnetValidators()

AddExamplePubnetQuorum adds example pubnet validators to toml file

func (*CaptiveCoreToml) CatchupToml

func (c *CaptiveCoreToml) CatchupToml() (*CaptiveCoreToml, error)

CatchupToml returns a new CaptiveCoreToml instance based off the existing instance with some modifications which are suitable for running the catchup command on captive core.

func (*CaptiveCoreToml) HistoryIsConfigured

func (c *CaptiveCoreToml) HistoryIsConfigured() bool

HistoryIsConfigured returns true if the history archive locations are configured.

func (*CaptiveCoreToml) Marshal

func (c *CaptiveCoreToml) Marshal() ([]byte, error)

Marshal serializes the CaptiveCoreToml into a toml document.

func (*CaptiveCoreToml) QuorumSetIsConfigured

func (c *CaptiveCoreToml) QuorumSetIsConfigured() bool

QuorumSetIsConfigured returns true if there is a quorum set defined in the configuration.

type CaptiveCoreTomlParams

type CaptiveCoreTomlParams struct {
	// NetworkPassphrase is the Lantah Network passphrase used by captive core when connecting to the Lantah Network.
	NetworkPassphrase string
	// HistoryArchiveURLs are a list of history archive urls.
	HistoryArchiveURLs []string
	// HTTPPort is the TCP port to listen for requests (defaults to 0, which disables the HTTP server).
	HTTPPort *uint
	// PeerPort is the TCP port to bind to for connecting to the Lantah Network
	// (defaults to 11625). It may be useful for example when there's >1 Gravity
	// instance running on a machine.
	PeerPort *uint
	// LogPath is the (optional) path in which to store Core logs, passed along
	// to Gravity's LOG_FILE_PATH.
	LogPath *string
	// Strict is a flag which, if enabled, rejects Gravity toml fields which are not supported by captive core.
	Strict bool
	// If true, specifies that captive core should be invoked with on-disk rather than in-memory option for ledger state
	UseDB bool
	// the path to the core binary, used to introspect core at runtime, determine some toml capabilities
	CoreBinaryPath string
	// Enforce EnableSorobanDiagnosticEvents when not disabled explicitly
	EnforceSorobanDiagnosticEvents bool
}

CaptiveCoreTomlParams defines captive core configuration provided by OrbitR flags.

type CaptiveGravity

type CaptiveGravity struct {
	// contains filtered or unexported fields
}

CaptiveGravity is a ledger backend that starts internal Gravity subprocess responsible for streaming ledger data. It provides better decoupling than DatabaseBackend but requires some extra init time.

It operates in two modes:

  • When a BoundedRange is prepared it starts Gravity in catchup mode that replays ledgers in memory. This is very fast but requires Gravity to keep ledger state in RAM. It requires around 3GB of RAM as of August 2020.
  • When a UnboundedRange is prepared it runs Gravity catchup mode to sync with the first ledger and then runs it in a normal mode. This requires the configAppendPath to be provided because a quorum set needs to be selected.

When running CaptiveGravity will create a temporary folder to store bucket files and other temporary files. The folder is removed when Close is called.

The communication is performed via filesystem pipe which is created in a temporary folder.

Currently BoundedRange requires a full-trust on history archive. This issue is being fixed in Gravity.

While using BoundedRanges is straightforward there are a few gotchas connected to UnboundedRanges:

  • PrepareRange takes more time because all ledger entries must be stored on disk instead of RAM.
  • If GetLedger is not called frequently (every 5 sec. on average) the Gravity process can go out of sync with the network. This happens because there is no buffering of communication pipe and CaptiveGravity has a very small internal buffer and Gravity will not close the new ledger if it's not read.

Except for the Close function, CaptiveGravity is not thread-safe and should not be accessed by multiple go routines. Close is thread-safe and can be called from another go routine. Once Close is called it will interrupt and cancel any pending operations.

Requires Gravity v13.2.0+.

func NewCaptive

func NewCaptive(config CaptiveCoreConfig) (*CaptiveGravity, error)

NewCaptive returns a new CaptiveGravity instance.

func (*CaptiveGravity) Close

func (c *CaptiveGravity) Close() error

Close closes existing Gravity process, streaming sessions and removes all temporary files. Note, once a CaptiveGravity instance is closed it can no longer be used and all subsequent calls to PrepareRange(), GetLedger(), etc will fail. Close is thread-safe and can be called from another go routine.

func (*CaptiveGravity) GetLatestLedgerSequence

func (c *CaptiveGravity) GetLatestLedgerSequence(ctx context.Context) (uint32, error)

GetLatestLedgerSequence returns the sequence of the latest ledger available in the backend. This method returns an error if not in a session (start with PrepareRange).

Note that for UnboundedRange the returned sequence number is not necessarily the latest sequence closed by the network. It's always the last value available in the backend.

func (*CaptiveGravity) GetLedger

func (c *CaptiveGravity) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

GetLedger will block until the ledger is available in the backend (even for UnboundedRange), then return it's LedgerCloseMeta.

Call PrepareRange first to instruct the backend which ledgers to fetch. CaptiveGravity requires PrepareRange call first to initialize Gravity. Requesting a ledger on non-prepared backend will return an error.

Please note that requesting a ledger sequence far after current ledger will block the execution for a long time.

Because ledger data is streamed from Gravity sequentially, users should request sequences in a non-decreasing order. If the requested sequence number is less than the last requested sequence number, an error will be returned.

This function behaves differently for bounded and unbounded ranges:

  • BoundedRange: After getting the last ledger in a range this method will also Close() the backend.

func (*CaptiveGravity) IsPrepared

func (c *CaptiveGravity) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (*CaptiveGravity) PrepareRange

func (c *CaptiveGravity) PrepareRange(ctx context.Context, ledgerRange Range) error

PrepareRange prepares the given range (including from and to) to be loaded. Captive gravity backend needs to initialize Gravity state to be able to stream ledgers. Gravity mode depends on the provided ledgerRange:

  • For BoundedRange it will start Gravity in catchup mode.
  • For UnboundedRange it will first catchup to starting ledger and then run it normally (including connecting to the Lantah Network).

Please note that using a BoundedRange, currently, requires a full-trust on history archive. This issue is being fixed in Gravity.

type DatabaseBackend

type DatabaseBackend struct {
	// contains filtered or unexported fields
}

DatabaseBackend implements a database data store.

func NewDatabaseBackend

func NewDatabaseBackend(dataSourceName, networkPassphrase string) (*DatabaseBackend, error)

func NewDatabaseBackendFromSession

func NewDatabaseBackendFromSession(session db.SessionInterface, networkPassphrase string) (*DatabaseBackend, error)

func (*DatabaseBackend) Close

func (dbb *DatabaseBackend) Close() error

Close disconnects an active database session.

func (*DatabaseBackend) GetLatestLedgerSequence

func (dbb *DatabaseBackend) GetLatestLedgerSequence(ctx context.Context) (uint32, error)

GetLatestLedgerSequence returns the most recent ledger sequence number present in the database.

func (*DatabaseBackend) GetLedger

func (dbb *DatabaseBackend) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

GetLedger will block until the ledger is available in the backend (even for UnaboundedRange). Please note that requesting a ledger sequence far after current ledger will block the execution for a long time.

func (*DatabaseBackend) IsPrepared

func (*DatabaseBackend) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (*DatabaseBackend) PrepareRange

func (dbb *DatabaseBackend) PrepareRange(ctx context.Context, ledgerRange Range) error

type History

type History struct {
	Get string `toml:"get"`
	// should we allow put and mkdir for captive core?
	Put   string `toml:"put,omitempty"`
	Mkdir string `toml:"mkdir,omitempty"`
}

History represents a [HISTORY] table in the captive core toml file.

type HomeDomain

type HomeDomain struct {
	HomeDomain string `toml:"HOME_DOMAIN"`
	Quality    string `toml:"QUALITY"`
}

HomeDomain represents a [[HOME_DOMAINS]] entry in the captive core toml file.

type LatestLedgerSequenceResponse

type LatestLedgerSequenceResponse struct {
	Sequence uint32 `json:"sequence"`
}

LatestLedgerSequenceResponse is the response for the GetLatestLedgerSequence command.

type LedgerBackend

type LedgerBackend interface {
	// GetLatestLedgerSequence returns the sequence of the latest ledger available
	// in the backend.
	GetLatestLedgerSequence(ctx context.Context) (sequence uint32, err error)
	// GetLedger will block until the ledger is available.
	GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)
	// PrepareRange prepares the given range (including from and to) to be loaded.
	// Some backends (like captive gravity) need to initalize data to be
	// able to stream ledgers. Blocks until the first ledger is available.
	PrepareRange(ctx context.Context, ledgerRange Range) error
	// IsPrepared returns true if a given ledgerRange is prepared.
	IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)
	Close() error
}

LedgerBackend represents the interface to a ledger data store.

func WithMetrics

func WithMetrics(base LedgerBackend, registry *prometheus.Registry, namespace string) LedgerBackend

WithMetrics decorates the given LedgerBackend with metrics

type LedgerResponse

type LedgerResponse struct {
	Ledger Base64Ledger `json:"ledger"`
}

LedgerResponse is the response for the GetLedger command.

type MockDatabaseBackend

type MockDatabaseBackend struct {
	mock.Mock
}

func (*MockDatabaseBackend) Close

func (m *MockDatabaseBackend) Close() error

func (*MockDatabaseBackend) GetLatestLedgerSequence

func (m *MockDatabaseBackend) GetLatestLedgerSequence(ctx context.Context) (uint32, error)

func (*MockDatabaseBackend) GetLedger

func (m *MockDatabaseBackend) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

func (*MockDatabaseBackend) IsPrepared

func (m *MockDatabaseBackend) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

func (*MockDatabaseBackend) PrepareRange

func (m *MockDatabaseBackend) PrepareRange(ctx context.Context, ledgerRange Range) error

type MockLedgerHashStore

type MockLedgerHashStore struct {
	mock.Mock
}

MockLedgerHashStore is a mock implementation of TrustedLedgerHashStore

func (*MockLedgerHashStore) Close

func (m *MockLedgerHashStore) Close() error

func (*MockLedgerHashStore) GetLedgerHash

func (m *MockLedgerHashStore) GetLedgerHash(ctx context.Context, seq uint32) (string, bool, error)

GetLedgerHash returns the ledger hash for the given sequence number

type OrbitRDBLedgerHashStore

type OrbitRDBLedgerHashStore struct {
	// contains filtered or unexported fields
}

OrbitRDBLedgerHashStore is a TrustedLedgerHashStore which uses orbitr's db to look up ledger hashes

func (OrbitRDBLedgerHashStore) Close

func (h OrbitRDBLedgerHashStore) Close() error

func (OrbitRDBLedgerHashStore) GetLedgerHash

func (h OrbitRDBLedgerHashStore) GetLedgerHash(ctx context.Context, seq uint32) (string, bool, error)

GetLedgerHash returns the ledger hash for the given sequence number

type PrepareRangeResponse

type PrepareRangeResponse struct {
	LedgerRange   Range     `json:"ledgerRange"`
	StartTime     time.Time `json:"startTime"`
	Ready         bool      `json:"ready"`
	ReadyDuration int       `json:"readyDuration"`
}

PrepareRangeResponse describes the status of the pending PrepareRange operation.

type QuorumSet

type QuorumSet struct {
	ThresholdPercent int      `toml:"THRESHOLD_PERCENT"`
	Validators       []string `toml:"VALIDATORS"`
}

QuorumSet represents a [QUORUM_SET] table in the captive core toml file.

type Range

type Range struct {
	// contains filtered or unexported fields
}

Range represents a range of ledger sequence numbers.

func BoundedRange

func BoundedRange(from uint32, to uint32) Range

BoundedRange constructs a bounded range of ledgers with a fixed starting ledger and ending ledger.

func SingleLedgerRange

func SingleLedgerRange(ledger uint32) Range

SingleLedgerRange constructs a bounded range containing a single ledger.

func UnboundedRange

func UnboundedRange(from uint32) Range

BoundedRange constructs a unbounded range of ledgers with a fixed starting ledger.

func (Range) Contains

func (r Range) Contains(other Range) bool

func (Range) MarshalJSON

func (r Range) MarshalJSON() ([]byte, error)

func (Range) String

func (r Range) String() string

func (*Range) UnmarshalJSON

func (r *Range) UnmarshalJSON(b []byte) error

type RemoteCaptiveGravity

type RemoteCaptiveGravity struct {
	// contains filtered or unexported fields
}

RemoteCaptiveGravity is an http client for interacting with a remote captive core server.

func NewRemoteCaptive

func NewRemoteCaptive(captiveCoreURL string, options ...RemoteCaptiveOption) (RemoteCaptiveGravity, error)

NewRemoteCaptive returns a new RemoteCaptiveGravity instance.

Only the captiveCoreURL parameter is required.

func (RemoteCaptiveGravity) Close

func (c RemoteCaptiveGravity) Close() error

Close cancels any pending PrepareRange requests.

func (RemoteCaptiveGravity) GetLatestLedgerSequence

func (c RemoteCaptiveGravity) GetLatestLedgerSequence(ctx context.Context) (sequence uint32, err error)

GetLatestLedgerSequence returns the sequence of the latest ledger available in the backend. This method returns an error if not in a session (start with PrepareRange).

Note that for UnboundedRange the returned sequence number is not necessarily the latest sequence closed by the network. It's always the last value available in the backend.

func (RemoteCaptiveGravity) GetLedger

func (c RemoteCaptiveGravity) GetLedger(ctx context.Context, sequence uint32) (xdr.LedgerCloseMeta, error)

Call PrepareRange first to instruct the backend which ledgers to fetch.

Requesting a ledger on non-prepared backend will return an error.

Because data is streamed from Gravity ledger after ledger user should request sequences in a non-decreasing order. If the requested sequence number is less than the last requested sequence number, an error will be returned.

func (RemoteCaptiveGravity) IsPrepared

func (c RemoteCaptiveGravity) IsPrepared(ctx context.Context, ledgerRange Range) (bool, error)

IsPrepared returns true if a given ledgerRange is prepared.

func (RemoteCaptiveGravity) PrepareRange

func (c RemoteCaptiveGravity) PrepareRange(ctx context.Context, ledgerRange Range) error

PrepareRange prepares the given range (including from and to) to be loaded. Captive gravity backend needs to initalize Gravity state to be able to stream ledgers. Gravity mode depends on the provided ledgerRange:

  • For BoundedRange it will start Gravity in catchup mode.
  • For UnboundedRange it will first catchup to starting ledger and then run it normally (including connecting to the Lantah Network).

Please note that using a BoundedRange, currently, requires a full-trust on history archive. This issue is being fixed in Gravity.

type RemoteCaptiveOption

type RemoteCaptiveOption func(c *RemoteCaptiveGravity)

RemoteCaptiveOption values can be passed into NewRemoteCaptive to customize a RemoteCaptiveGravity instance.

func PrepareRangePollInterval

func PrepareRangePollInterval(d time.Duration) RemoteCaptiveOption

PrepareRangePollInterval configures how often the captive core server will be polled when blocking on the PrepareRange operation.

type TrustedLedgerHashStore

type TrustedLedgerHashStore interface {
	// GetLedgerHash returns the ledger hash for the given sequence number
	GetLedgerHash(ctx context.Context, seq uint32) (string, bool, error)
	Close() error
}

TrustedLedgerHashStore is used to query ledger data from a trusted source. The store should contain ledgers verified by Gravity, do not use untrusted source like history archives.

func NewOrbitRDBLedgerHashStore

func NewOrbitRDBLedgerHashStore(session db.SessionInterface) TrustedLedgerHashStore

NewOrbitRDBLedgerHashStore constructs a new TrustedLedgerHashStore backed by the orbitr db

type Validator

type Validator struct {
	Name       string `toml:"NAME"`
	Quality    string `toml:"QUALITY,omitempty"`
	HomeDomain string `toml:"HOME_DOMAIN"`
	PublicKey  string `toml:"PUBLIC_KEY"`
	Address    string `toml:"ADDRESS,omitempty"`
	History    string `toml:"HISTORY,omitempty"`
}

Validator represents a [[VALIDATORS]] entry in the captive core toml file.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL