pachyderm: github.com/pachyderm/pachyderm/src/server/pkg/storage/fileset Index | Files | Directories

package fileset

import "github.com/pachyderm/pachyderm/src/server/pkg/storage/fileset"

Index

Package Files

merge.go reader.go storage.go util.go writer.go

func CopyN Uses

func CopyN(w *Writer, r *Reader, n int64) error

CopyN copies n bytes from a file set reader to a file set writer. The bytes copied correspond to the current file being read / written respectively.

type Reader Uses

type Reader struct {
    // contains filtered or unexported fields
}

Reader reads the serialized format of a fileset.

func (*Reader) Next Uses

func (r *Reader) Next() (*index.Header, error)

Next returns the next header, and prepares the file's content for reading.

func (*Reader) Read Uses

func (r *Reader) Read(data []byte) (int, error)

Read reads from the current file in the tar stream.

type Storage Uses

type Storage struct {
    // contains filtered or unexported fields
}

Storage is the abstraction that manages fileset storage.

func NewStorage Uses

func NewStorage(objC obj.Client, chunks *chunk.Storage) *Storage

NewStorage creates a new Storage.

func (*Storage) NewIndexReader Uses

func (s *Storage) NewIndexReader(ctx context.Context, fileSet, idxPrefix string) *index.Reader

NewIndexReader creates a new index.Reader.

func (*Storage) NewIndexWriter Uses

func (s *Storage) NewIndexWriter(ctx context.Context, fileSet string) *index.Writer

NewIndexWriter creates a new index.Writer.

func (*Storage) NewReader Uses

func (s *Storage) NewReader(ctx context.Context, fileSet, idxPrefix string) *Reader

NewReader creates a new Reader.

func (*Storage) NewWriter Uses

func (s *Storage) NewWriter(ctx context.Context, fileSet string) *Writer

NewWriter creates a new Writer.

type Writer Uses

type Writer struct {
    // contains filtered or unexported fields
}

Writer writes the serialized format of a fileset. The serialized format of a fileset consists of indexes and content which are both realized as compressed tar stream chunks.

func (*Writer) Close Uses

func (w *Writer) Close() error

Close closes the writer. (bryce) is not closing the tar writer the right choice here? We cannot include it in the last file's range because it will effect the hash. Our indexing is what will exit the reading of the tar stream, not the end of tar entry (two empty 512 bytes).

func (*Writer) StartTag Uses

func (w *Writer) StartTag(id string)

StartTag starts a tag for the next set of bytes (used for the reverse index, mapping file output to datums).

func (*Writer) Write Uses

func (w *Writer) Write(data []byte) (int, error)

Write writes to the current file in the tar stream.

func (*Writer) WriteHeader Uses

func (w *Writer) WriteHeader(hdr *index.Header) error

WriteHeader writes a tar header and prepares to accept the file's contents.

Directories

PathSynopsis
index
tarPackage tar implements access to tar archives.

Package fileset imports 9 packages (graph). Updated 2019-08-13. Refresh now. Tools for package owners.