archive

func CreateFromAFiles ¶

func CreateFromAFiles(feed *config.Feed, aFiles []storage.AFile, sourceAStore storage.ReadableAStore,
	targetAStore storage.WritableAStore, tempDStore storage.DStore) (storage.AFile, []storage.AFile, error)

CreateFromDFiles creates an AFile from a collection of AFiles located in a source AStore. The AFile is written to a target AStore. This method is used, for example, when merging multiple AFiles into a single AFile.

The function uses a temporary DStore to unpack the contents of the provided AFiles. This can be in-memory, but that can create memory issues.

The function returns the key of the AFile that was written and a slice containing all AFiles whose contents were successfully written to the archive. It is safe to delete these AFiles afterward because their contents are guaranteed to be present in the new AFile.

Errors encountered when creating the archive are handled in one of two ways. If the error concerns a single AFile (for example, it doesn't exist in the AStore) then the error is essentially ignored and that DFile will not be returned in the slice. Otherwise, errors are propagated through the returned error type. This two-prong approach means the function can at least succeed if some AFiles can be written.

func CreateFromDFiles ¶

func CreateFromDFiles(feed *config.Feed, dFiles []storage.DFile,
	sourceDStore storage.ReadableDStore, targetAStore storage.WritableAStore) (storage.AFile, []storage.DFile, error)

CreateFromDFiles creates an AFile from a collection of DFiles located in a source DStore. The AFile is written to a target AStore. This method is used, for example, when packing recently downloaded files into a single archive.

The function returns the key of the AFile that was written and a slice containing all DFiles that were successfully written to the archive. It is safe to delete these DFiles afterward because they are guaranteed to be present in the AFile.

Errors encountered when creating the archive are handled in one of two ways. If the error concerns a single DFile (for example, it doesn't exist in the DStore) then the error is essentially ignored and that DFile will not be returned in the slice. Otherwise, errors are propagated through the returned error type. This two-prong approach means the function can at least succeed if some DFiles can be written.

func Recompress ¶

func Recompress(feed *config.Feed, aFile storage.AFile,
	sourceAStore storage.ReadableAStore, targetAStore storage.WritableAStore) (newAFile storage.AFile, err error)

Recompress reads the provided AFile from the source AStore and recompresses the archive so that its compression settings match those of the feed configuration. If the compression settings already match, this is a no-op.

func Unpack ¶

func Unpack(aFile storage.AFile, aStore storage.ReadableAStore, dStore storage.WritableDStore) error

Unpack reads the contents of an AFile into the provided DStore.