Documentation
¶
Overview ¶
Package reedsolomon enables Erasure Coding in Go
For usage and examples, see https://github.com/klauspost/reedsolomon
Index ¶
- Variables
- type Option
- type ReedSolomon
- func (r *ReedSolomon) Encode(shards [][]byte) error
- func (r *ReedSolomon) Join(dst io.Writer, shards [][]byte, outSize int) error
- func (r *ReedSolomon) JoinMulti(dst io.Writer, shards [][]byte, subsize int, outSize int) error
- func (r *ReedSolomon) Reconstruct(shards [][]byte) error
- func (r *ReedSolomon) ReconstructData(shards [][]byte) error
- func (r *ReedSolomon) ReconstructDataMulti(shards [][]byte, subsize int) error
- func (r *ReedSolomon) ReconstructMulti(shards [][]byte, subsize int) error
- func (r *ReedSolomon) Split(data []byte) ([][]byte, error)
- func (r *ReedSolomon) Update(shards [][]byte, newDatashards [][]byte) error
- func (r *ReedSolomon) Verify(shards [][]byte) (bool, error)
Constants ¶
This section is empty.
Variables ¶
var ErrInvShardNum = errors.New("cannot create Encoder with zero or less data/parity shards")
ErrInvShardNum will be returned by New, if you attempt to create an Encoder where either data or parity shards is zero or less.
var ErrInvalidInput = errors.New("invalid input")
ErrInvalidInput is returned if invalid input parameter of Update.
var ErrMaxShardNum = errors.New("cannot create Encoder with more than 256 data+parity shards")
ErrMaxShardNum will be returned by New, if you attempt to create an Encoder where data and parity shards are bigger than the order of GF(2^8).
var ErrReconstructRequired = errors.New("reconstruction required as one or more required data shards are nil")
ErrReconstructRequired is returned if too few data shards are intact and a reconstruction is required before you can successfully join the shards.
var ErrShardNoData = errors.New("no shard data")
ErrShardNoData will be returned if there are no shards, or if the length of all shards is zero.
var ErrShardSize = errors.New("shard sizes do not match")
ErrShardSize is returned if shard length isn't the same for all shards.
var ErrShortData = errors.New("not enough data to fill the number of requested shards")
ErrShortData will be returned by Split(), if there isn't enough data to fill the number of shards.
var ErrTooFewShards = errors.New("too few shards given")
ErrTooFewShards is returned if too few shards where given to Encode/Verify/Reconstruct/Update. It will also be returned from Reconstruct if there were too few shards to reconstruct the missing data.
Functions ¶
This section is empty.
Types ¶
type Option ¶
type Option func(*options)
Option allows to override processing parameters.
func WithAutoGoroutines ¶
WithAutoGoroutines will adjust the number of goroutines for optimal speed with a specific shard size. Send in the shard size you expect to send. Other shard sizes will work, but may not run at the optimal speed. Overwrites WithMaxGoroutines. If shardSize <= 0, it is ignored.
func WithCauchyMatrix ¶
func WithCauchyMatrix() Option
WithCauchyMatrix will make the encoder build a Cauchy style matrix. The output of this is not compatible with the standard output. A Cauchy matrix is faster to generate. This does not affect data throughput, but will result in slightly faster start-up time.
func WithMaxGoroutines ¶
WithMaxGoroutines is the maximum number of goroutines number for encoding & decoding. Jobs will be split into this many parts, unless each goroutine would have to process less than minSplitSize bytes (set with WithMinSplitSize). For the best speed, keep this well above the GOMAXPROCS number for more fine grained scheduling. If n <= 0, it is ignored.
func WithMinSplitSize ¶
WithMinSplitSize is the minimum encoding size in bytes per goroutine. See WithMaxGoroutines on how jobs are split. If n <= 0, it is ignored.
func WithPAR1Matrix ¶
func WithPAR1Matrix() Option
WithPAR1Matrix causes the encoder to build the matrix how PARv1 does. Note that the method they use is buggy, and may lead to cases where recovery is impossible, even if there are enough parity shards.
type ReedSolomon ¶
type ReedSolomon struct {
DataShards int // Number of data shards, should not be modified.
ParityShards int // Number of parity shards, should not be modified.
Shards int // Total number of shards. Calculated, and should not be modified.
// contains filtered or unexported fields
}
ReedSolomon contains a matrix for a specific distribution of datashards and parity shards. Construct if using New()
func New ¶
func New(dataShards, parityShards int, opts ...Option) (*ReedSolomon, error)
New creates a new encoder and initializes it to the number of data shards and parity shards that you want to use. You can reuse this encoder. Note that the maximum number of total shards is 256. If no options are supplied, default options are used.
func (*ReedSolomon) Encode ¶
func (r *ReedSolomon) Encode(shards [][]byte) error
Encodes parity for a set of data shards. An array 'shards' containing data shards followed by parity shards. The number of shards must match the number given to New. Each shard is a byte array, and they must all be the same size. The parity shards will always be overwritten and the data shards will remain the same.
func (*ReedSolomon) Join ¶
Join the shards and write the data segment to dst.
Only the data shards are considered. You must supply the exact output size you want.
If there are to few shards given, ErrTooFewShards will be returned. If the total data size is less than outSize, ErrShortData will be returned. If one or more required data shards are nil, ErrReconstructRequired will be returned.
func (*ReedSolomon) JoinMulti ¶
JoinMulti joins the supplied multi-block shards, writing them to dst.
func (*ReedSolomon) Reconstruct ¶
func (r *ReedSolomon) Reconstruct(shards [][]byte) error
Reconstruct will recreate the missing shards, if possible.
Given a list of shards, some of which contain data, fills in the ones that don't have data.
The length of the array must be equal to Shards. You indicate that a shard is missing by setting it to nil or zero-length. If a shard is zero-length but has sufficient capacity, that memory will be used, otherwise a new []byte will be allocated.
If there are too few shards to reconstruct the missing ones, ErrTooFewShards will be returned.
The reconstructed shard set is complete, but integrity is not verified. Use the Verify function to check if data set is ok.
func (*ReedSolomon) ReconstructData ¶
func (r *ReedSolomon) ReconstructData(shards [][]byte) error
ReconstructData will recreate any missing data shards, if possible.
Given a list of shards, some of which contain data, fills in the data shards that don't have data.
The length of the array must be equal to Shards. You indicate that a shard is missing by setting it to nil or zero-length. If a shard is zero-length but has sufficient capacity, that memory will be used, otherwise a new []byte will be allocated.
If there are too few shards to reconstruct the missing ones, ErrTooFewShards will be returned.
As the reconstructed shard set may contain missing parity shards, calling the Verify function is likely to fail.
func (*ReedSolomon) ReconstructDataMulti ¶
func (r *ReedSolomon) ReconstructDataMulti(shards [][]byte, subsize int) error
ReconstructDataMulti reconstructs data shards in blocks of subsize bytes.
func (*ReedSolomon) ReconstructMulti ¶
func (r *ReedSolomon) ReconstructMulti(shards [][]byte, subsize int) error
ReconstructMulti reconstructs shards in blocks of subsize bytes.
func (*ReedSolomon) Split ¶
func (r *ReedSolomon) Split(data []byte) ([][]byte, error)
Split a data slice into the number of shards given to the encoder, and create empty parity shards if necessary.
The data will be split into equally sized shards. If the data size isn't divisible by the number of shards, the last shard will contain extra zeros.
There must be at least 1 byte otherwise ErrShortData will be returned.
The data will not be copied, except for the last shard, so you should not modify the data of the input slice afterwards.
Source Files