Documentation
¶
Index ¶
- Variables
- func CollectSharesByNamespace(ctx context.Context, bg blockservice.BlockGetter, root *share.Root, ...) (shares share.NamespacedShares, err error)
- func ODSReader(carReader io.Reader) (io.Reader, error)
- func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (eds *rsmt2d.ExtendedDataSquare, err error)
- func RetrieveNamespaceFromStore(ctx context.Context, store *Store, dah *share.Root, namespace share.Namespace) (shares share.NamespacedShares, err error)
- func WriteEDS(ctx context.Context, eds *rsmt2d.ExtendedDataSquare, w io.Writer) (err error)
- type BlockGetter
- type BlockstoreCloser
- type Parameters
- type Retriever
- type Store
- func (s *Store) Blockstore() bstore.Blockstore
- func (s *Store) CARBlockstore(ctx context.Context, root share.DataHash) (*BlockstoreCloser, error)
- func (s *Store) Get(ctx context.Context, root share.DataHash) (*rsmt2d.ExtendedDataSquare, error)
- func (s *Store) GetCAR(ctx context.Context, root share.DataHash) (io.ReadCloser, error)
- func (s *Store) GetDAH(ctx context.Context, root share.DataHash) (*share.Root, error)
- func (s *Store) Has(ctx context.Context, root share.DataHash) (has bool, err error)
- func (s *Store) List() ([]share.DataHash, error)
- func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error
- func (s *Store) Remove(ctx context.Context, root share.DataHash) error
- func (s *Store) Start(ctx context.Context) error
- func (s *Store) Stop(context.Context) error
- func (s *Store) WithMetrics() error
Constants ¶
This section is empty.
Variables ¶
var ErrEmptySquare = errors.New("share: importing empty data")
var ErrNotFound = errors.New("eds not found in store")
var ErrNotFoundInIndex = errors.New("does not exist in index")
ErrNotFoundInIndex is returned instead of ErrNotFound if the multihash doesn't exist in the index
var RetrieveQuadrantTimeout = blockTime / numQuadrants * 2
RetrieveQuadrantTimeout defines how much time Retriever waits before starting to retrieve another quadrant.
NOTE: - The whole data square must be retrieved in less than block time. - We have 4 quadrants from two sources(rows, cols) which equals to 8 in total.
Functions ¶
func CollectSharesByNamespace ¶ added in v0.11.0
func CollectSharesByNamespace( ctx context.Context, bg blockservice.BlockGetter, root *share.Root, namespace share.Namespace, ) (shares share.NamespacedShares, err error)
CollectSharesByNamespace collects NamespaceShares within the given namespace from share.Root.
func ODSReader ¶ added in v0.5.0
ODSReader reads CARv1 encoded data from io.ReadCloser and limits the reader to the CAR header and first quadrant (ODS)
func ReadEDS ¶ added in v0.5.0
func ReadEDS(ctx context.Context, r io.Reader, root share.DataHash) (eds *rsmt2d.ExtendedDataSquare, err error)
ReadEDS reads the first EDS quadrant (1/4) from an io.Reader CAR file. Only the first quadrant will be read, which represents the original data. The returned EDS is guaranteed to be full and valid against the DataRoot, otherwise ReadEDS errors.
func RetrieveNamespaceFromStore ¶ added in v0.11.0
func RetrieveNamespaceFromStore( ctx context.Context, store *Store, dah *share.Root, namespace share.Namespace, ) (shares share.NamespacedShares, err error)
RetrieveNamespaceFromStore gets all EDS shares in the given namespace from the EDS store through the corresponding CAR-level blockstore. It is extracted from the store getter to make it available for reuse in the shrexnd server.
func WriteEDS ¶ added in v0.5.0
WriteEDS writes the entire EDS into the given io.Writer as CARv1 file. This includes all shares in quadrant order, followed by all inner nodes of the NMT tree. Order: [ Carv1Header | Q1 | Q2 | Q3 | Q4 | inner nodes ] For more information about the header: https://ipld.io/specs/transport/car/carv1/#header
Types ¶
type BlockGetter ¶ added in v0.6.2
type BlockGetter struct {
// contains filtered or unexported fields
}
BlockGetter is an adapter for dagstore.ReadBlockstore to implement blockservice.BlockGetter interface.
func NewBlockGetter ¶ added in v0.6.2
func NewBlockGetter(store dagstore.ReadBlockstore) *BlockGetter
NewBlockGetter creates new blockservice.BlockGetter adapter from dagstore.ReadBlockstore
func (*BlockGetter) GetBlocks ¶ added in v0.6.2
func (bg *BlockGetter) GetBlocks(ctx context.Context, cids []cid.Cid) <-chan blocks.Block
GetBlocks does a batch request for the given cids, returning blocks as they are found, in no particular order.
It implements blockservice.BlockGetter interface, that requires: It may not be able to find all requested blocks (or the context may be canceled). In that case, it will close the channel early. It is up to the consumer to detect this situation and keep track which blocks it has received and which it hasn't.
type BlockstoreCloser ¶ added in v0.11.0
type BlockstoreCloser struct {
dagstore.ReadBlockstore
io.Closer
}
BlockstoreCloser represents a blockstore that can also be closed. It combines the functionality of a dagstore.ReadBlockstore with that of an io.Closer.
type Parameters ¶ added in v0.11.0
type Parameters struct {
// GC performs DAG store garbage collection by reclaiming transient files of
// shards that are currently available but inactive, or errored.
// We don't use transient files right now, so GC is turned off by default.
GCInterval time.Duration
// RecentBlocksCacheSize is the size of the cache for recent blocks.
RecentBlocksCacheSize int
// BlockstoreCacheSize is the size of the cache for blockstore requested accessors.
BlockstoreCacheSize int
}
func DefaultParameters ¶ added in v0.11.0
func DefaultParameters() *Parameters
DefaultParameters returns the default configuration values for the EDS store parameters.
func (*Parameters) Validate ¶ added in v0.11.0
func (p *Parameters) Validate() error
type Retriever ¶
type Retriever struct {
// contains filtered or unexported fields
}
Retriever retrieves rsmt2d.ExtendedDataSquares from the IPLD network. Instead of requesting data 'share by share' it requests data by quadrants minimizing bandwidth usage in the happy cases.
---- ---- | 0 | 1 | ---- ---- | 2 | 3 | ---- ----
Retriever randomly picks one of the data square quadrants and tries to request them one by one until it is able to reconstruct the whole square.
func NewRetriever ¶
func NewRetriever(bServ blockservice.BlockService) *Retriever
NewRetriever creates a new instance of the Retriever over IPLD BlockService and rmst2d.Codec
func (*Retriever) Retrieve ¶
func (r *Retriever) Retrieve(ctx context.Context, dah *da.DataAvailabilityHeader) (*rsmt2d.ExtendedDataSquare, error)
Retrieve retrieves all the data committed to DataAvailabilityHeader.
If not available locally, it aims to request from the network only one quadrant (1/4) of the data square and reconstructs the other three quadrants (3/4). If the requested quadrant is not available within RetrieveQuadrantTimeout, it starts requesting another quadrant until either the data is reconstructed, context is canceled or ErrByzantine is generated.
type Store ¶ added in v0.5.0
type Store struct {
// contains filtered or unexported fields
}
Store maintains (via DAGStore) a top-level index enabling granular and efficient random access to every share and/or Merkle proof over every registered CARv1 file. The EDSStore provides a custom blockstore interface implementation to achieve access. The main use-case is randomized sampling over the whole chain of EDS block data and getting data by namespace.
func NewStore ¶ added in v0.5.0
func NewStore(params *Parameters, basePath string, ds datastore.Batching) (*Store, error)
NewStore creates a new EDS Store under the given basepath and datastore.
func (*Store) Blockstore ¶ added in v0.5.0
func (s *Store) Blockstore() bstore.Blockstore
Blockstore returns an IPFS blockstore providing access to individual shares/nodes of all EDS registered on the Store. NOTE: The blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.
func (*Store) CARBlockstore ¶ added in v0.5.0
func (s *Store) CARBlockstore( ctx context.Context, root share.DataHash, ) (*BlockstoreCloser, error)
CARBlockstore returns an IPFS Blockstore providing access to individual shares/nodes of a specific EDS identified by DataHash and registered on the Store. NOTE: The Blockstore does not store whole Celestia Blocks but IPFS blocks. We represent `shares` and NMT Merkle proofs as IPFS blocks and IPLD nodes so Bitswap can access those.
func (*Store) Get ¶ added in v0.5.0
Get reads EDS out of Store by given DataRoot.
It reads only one quadrant(1/4) of the EDS and verifies the integrity of the stored data by recomputing it.
func (*Store) GetCAR ¶ added in v0.5.0
GetCAR takes a DataRoot and returns a buffered reader to the respective EDS serialized as a CARv1 file. The Reader strictly reads the CAR header and first quadrant (1/4) of the EDS, omitting all the NMT Merkle proofs. Integrity of the store data is not verified.
The shard is cached in the Store, so subsequent calls to GetCAR with the same root will use the same reader. The cache is responsible for closing the underlying reader.
func (*Store) GetDAH ¶ added in v0.6.2
GetDAH returns the DataAvailabilityHeader for the EDS identified by DataHash.
func (*Store) Put ¶ added in v0.5.0
func (s *Store) Put(ctx context.Context, root share.DataHash, square *rsmt2d.ExtendedDataSquare) error
Put stores the given data square with DataRoot's hash as a key.
The square is verified on the Exchange level, and Put only stores the square, trusting it. The resulting file stores all the shares and NMT Merkle Proofs of the EDS. Additionally, the file gets indexed s.t. store.Blockstore can access them.
func (*Store) Remove ¶ added in v0.5.0
Remove removes EDS from Store by the given share.Root hash and cleans up all the indexing.
Source Files
Directories