Documentation ¶
Index ¶
- Constants
- func Create(parPath string, filePaths []string, options CreateOptions) error
- func RepairErrorMeansRepairNecessaryButNotPossible(err error) bool
- type CreateDelegate
- type CreateOptions
- type Decoder
- type DecoderDelegate
- type DoNothingCreateDelegate
- type DoNothingDecoderDelegate
- func (DoNothingDecoderDelegate) OnCommentLoad(comment []byte)
- func (DoNothingDecoderDelegate) OnDataFileLoad(i, n int, path string, byteCount int, corrupt bool, err error)
- func (DoNothingDecoderDelegate) OnDataFileWrite(i, n int, path string, byteCount int, err error)
- func (DoNothingDecoderDelegate) OnFileEntryLoad(i, n int, filename, entryInfo string)
- func (DoNothingDecoderDelegate) OnHeaderLoad(headerInfo string)
- func (DoNothingDecoderDelegate) OnVolumeFileLoad(i uint64, path string, storedSetHash, computedSetHash [16]byte, ...)
- type DoNothingRepairDelegate
- type DoNothingVerifyDelegate
- type Encoder
- type EncoderDelegate
- type FileCounts
- type RepairDelegate
- type RepairOptions
- type RepairResult
- type VerifyDelegate
- type VerifyOptions
- type VerifyResult
Constants ¶
const NumParityFilesDefault = 3
NumParityFilesDefault is the default value used for CreateOptions.NumParityFiles if the latter is <= 0.
Variables ¶
This section is empty.
Functions ¶
func Create ¶
func Create(parPath string, filePaths []string, options CreateOptions) error
Create a par file for the given file paths at parPath with the given options.
func RepairErrorMeansRepairNecessaryButNotPossible ¶
RepairErrorMeansRepairNecessaryButNotPossible returns true if the error returned by Repair means that repair is necessary but not possible.
Types ¶
type CreateDelegate ¶
type CreateDelegate interface { EncoderDelegate OnFilesNotAllInSameDir() }
CreateDelegate extends EncoderDelegate with another delegate function.
type CreateOptions ¶
type CreateOptions struct { // The number of parity files to create. If <= 0, // NumParityFilesDefault is used. NumParityFiles int // The CreateDelegate to use. If nil, DoNothingCreateDelegate // is used. CreateDelegate CreateDelegate }
CreateOptions holds all the options for Create.
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
A Decoder keeps track of all information needed to check the integrity of a set of data files, and possibly repair any missing/corrupt data files from the parity files (.P00, .P01, etc.).
func NewDecoder ¶
func NewDecoder(delegate DecoderDelegate, indexFile string) (*Decoder, error)
NewDecoder reads the given index file, which usually has a .PAR extension.
func (*Decoder) FileCounts ¶
func (d *Decoder) FileCounts() FileCounts
FileCounts returns a FileCounts object for the current file set.
func (*Decoder) LoadFileData ¶
LoadFileData loads existing file data into memory.
func (*Decoder) LoadParityData ¶
LoadParityData searches for parity volumes and loads them into memory.
func (*Decoder) Repair ¶
Repair tries to repair any missing or corrupt data, using the parity volumes. Returns a list of paths to files that were successfully repaired (relative to the indexFile passed to NewDecoder) in no particular order, which is present even if an error is returned. If checkParity is true, extra checking is done of the reconstructed parity data.
func (*Decoder) VerifyAllData ¶
VerifyAllData checks and returns whether all data and parity files contain correct data. Note that this is worth calling only when there are no unusable data or parity files, since if there are, then false is guaranteed to be returned for ok.
type DecoderDelegate ¶
type DecoderDelegate interface { OnHeaderLoad(headerInfo string) OnFileEntryLoad(i, n int, filename, entryInfo string) OnCommentLoad(comment []byte) OnDataFileLoad(i, n int, path string, byteCount int, corrupt bool, err error) OnDataFileWrite(i, n int, path string, byteCount int, err error) OnVolumeFileLoad(i uint64, path string, storedSetHash, computedSetHash [16]byte, dataByteCount int, err error) }
DecoderDelegate holds methods that are called during the decode process.
type DoNothingCreateDelegate ¶
type DoNothingCreateDelegate struct{}
DoNothingCreateDelegate is an implementation of CreateDelegate that does nothing for all methods.
func (DoNothingCreateDelegate) OnDataFileLoad ¶
func (DoNothingCreateDelegate) OnDataFileLoad(i, n int, path string, byteCount int, err error)
OnDataFileLoad implements the CreateDelegate interface.
func (DoNothingCreateDelegate) OnFilesNotAllInSameDir ¶
func (DoNothingCreateDelegate) OnFilesNotAllInSameDir()
OnFilesNotAllInSameDir implements the CreateDelegate interface.
func (DoNothingCreateDelegate) OnVolumeFileWrite ¶
func (DoNothingCreateDelegate) OnVolumeFileWrite(i, n int, path string, dataByteCount, byteCount int, err error)
OnVolumeFileWrite implements the CreateDelegate interface.
type DoNothingDecoderDelegate ¶
type DoNothingDecoderDelegate struct{}
DoNothingDecoderDelegate is an implementation of DecoderDelegate that does nothing for all methods.
func (DoNothingDecoderDelegate) OnCommentLoad ¶
func (DoNothingDecoderDelegate) OnCommentLoad(comment []byte)
OnCommentLoad implements the DecoderDelegate interface.
func (DoNothingDecoderDelegate) OnDataFileLoad ¶
func (DoNothingDecoderDelegate) OnDataFileLoad(i, n int, path string, byteCount int, corrupt bool, err error)
OnDataFileLoad implements the DecoderDelegate interface.
func (DoNothingDecoderDelegate) OnDataFileWrite ¶
func (DoNothingDecoderDelegate) OnDataFileWrite(i, n int, path string, byteCount int, err error)
OnDataFileWrite implements the DecoderDelegate interface.
func (DoNothingDecoderDelegate) OnFileEntryLoad ¶
func (DoNothingDecoderDelegate) OnFileEntryLoad(i, n int, filename, entryInfo string)
OnFileEntryLoad implements the DecoderDelegate interface.
func (DoNothingDecoderDelegate) OnHeaderLoad ¶
func (DoNothingDecoderDelegate) OnHeaderLoad(headerInfo string)
OnHeaderLoad implements the DecoderDelegate interface.
func (DoNothingDecoderDelegate) OnVolumeFileLoad ¶
func (DoNothingDecoderDelegate) OnVolumeFileLoad(i uint64, path string, storedSetHash, computedSetHash [16]byte, dataByteCount int, err error)
OnVolumeFileLoad implements the DecoderDelegate interface.
type DoNothingRepairDelegate ¶
type DoNothingRepairDelegate struct {
DoNothingDecoderDelegate
}
DoNothingRepairDelegate is an implementation of RepairDelegate that does nothing for all methods.
type DoNothingVerifyDelegate ¶
type DoNothingVerifyDelegate struct {
DoNothingDecoderDelegate
}
DoNothingVerifyDelegate is an implementation of VerifyDelegate that does nothing for all methods.
type Encoder ¶
type Encoder struct {
// contains filtered or unexported fields
}
An Encoder keeps track of all information needed to create parity volumes for a set of data files, and write them out to parity files (.PAR, .P00, .P01, etc.).
func NewEncoder ¶
func NewEncoder(delegate EncoderDelegate, filePaths []string, volumeCount int) (*Encoder, error)
NewEncoder creates an encoder with the given list of file paths, and with the given number of intended parity volumes.
func (*Encoder) ComputeParityData ¶
ComputeParityData computes the parity data for the files.
func (*Encoder) LoadFileData ¶
LoadFileData loads the file data into memory.
type EncoderDelegate ¶
type EncoderDelegate interface { OnDataFileLoad(i, n int, path string, byteCount int, err error) OnVolumeFileWrite(i, n int, path string, dataByteCount, byteCount int, err error) }
EncoderDelegate holds methods that are called during the encode process.
type FileCounts ¶
type FileCounts struct { // UsableDataFileCount is the number of data files that are // usable, i.e. not missing and not corrupt. UsableDataFileCount int // UnusableDataFileCount is the number of data files that are // unusable, i.e. missing or corrupt. UnusableDataFileCount int // UsableParityFileCount is the number of parity files that // exist, i.e. not missing and not corrupt. UsableParityFileCount int // UnusableDataFileCount is the number of parity files that // are unusable, i.e. missing or corrupt. // // Note that we can only infer missing parity files from gaps // in the counts, e.g. if we only have foo.p01 and foo.p03, // then we can infer that foo.p02 is missing, but if we have // foo.p01 and foo.p02, then foo.p03 might be missing or it // might not have existed in the first place. UnusableParityFileCount int }
FileCounts contains file counts which can be used to deduce whether repair is necessary and/or possible.
func (FileCounts) AllFilesUsable ¶
func (fc FileCounts) AllFilesUsable() bool
AllFilesUsable returns whether or not all files are usable, i.e. whether UnusableDataFileCount == 0 and UnusableParityFileCount == 0.
func (FileCounts) RepairNeeded ¶
func (fc FileCounts) RepairNeeded() bool
RepairNeeded returns whether repair is needed, i.e. whether UnusableDataFileCount is non-zero.
func (FileCounts) RepairPossible ¶
func (fc FileCounts) RepairPossible() bool
RepairPossible returns whether repair is possible i.e. whether UsableParityFileCount >= UnusableDataFileCount.
type RepairDelegate ¶
type RepairDelegate interface { DecoderDelegate }
RepairDelegate is just DecoderDelegate for now.
type RepairOptions ¶
type RepairOptions struct { // If DoubleCheck is true, then extra checking is done after // the repair to verify that the repaired shards are correct. DoubleCheck bool // The RepairDelegate to use. If nil, DoNothingRepairDelegate // is used. RepairDelegate RepairDelegate }
RepairOptions holds all the options for Repair.
type RepairResult ¶
type RepairResult struct { // RepairedPaths contains the paths of the files that were // repaired. RepairedPaths []string }
RepairResult holds the result of a Repair call.
func Repair ¶
func Repair(parPath string, options RepairOptions) (RepairResult, error)
Repair a par file at parPath with the given options. The returned RepairResult may be partially or not filled in if an error is returned.
type VerifyDelegate ¶
type VerifyDelegate interface { DecoderDelegate }
VerifyDelegate is just DecoderDelegate for now.
type VerifyOptions ¶
type VerifyOptions struct { // If VerifyAllData is true, then check whether all data and // parity files contain correct data even if no missing or // corrupt files are detected. VerifyAllData bool // The VerifyDelegate to use. If nil, DoNothingVerifyDelegate // is used. VerifyDelegate VerifyDelegate }
VerifyOptions holds all the options for Verify.
type VerifyResult ¶
type VerifyResult struct { // FileCounts contains file counts which can be used to deduce // whether repair is necessary and/or possible. FileCounts FileCounts // AllDataOk contains the result of calling VerifyAllData(), // when VerifyAllData is set to true in VerifyOptions and // FileCounts.AllFilesUsable() returns true, and contains // false otherwise. AllDataOk bool }
VerifyResult holds the result of a Verify call.
func Verify ¶
func Verify(parPath string, options VerifyOptions) (VerifyResult, error)
Verify a par file at parPath with the given options. The returned VerifyResult is not filled in if an error is returned.