shardpacker

package

v0.0.0-...-4aac04f Latest Latest Go to latest Published: Apr 23, 2024 License: BSD-2-Clause Imports: 13 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/gravwell/cloudarchive

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
type Ftype
- func FilenameToType(name string) (ft Ftype, err error)
- func (ft Ftype) Filename(id string) string
- func (ft Ftype) Filepath(id string) string
type Packer
- func NewPacker(id string) (p *Packer)
type UnpackHandler
type Unpacker
- func NewUnpacker(id string, rdr io.Reader) (up *Unpacker, err error)
- func (up *Unpacker) Cancel()
- func (up *Unpacker) Unpack(uph UnpackHandler) (err error)

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	ErrClosed          = errors.New("closed")
	ErrInvalidFilePath = errors.New("invalid filepath")
	ErrInvalidFileType = errors.New("invalid file type")
)

View Source

var (
	ErrFailedWrite           = errors.New("Failed writing out complete file")
	ErrInvalidUnpackerParams = errors.New("Invalid unpacker parameters")
)

Functions ¶

This section is empty.

Types ¶

type Ftype ¶

type Ftype int

const (
	Store              Ftype = 1
	Index              Ftype = 2
	Verify             Ftype = 3
	AccelFile          Ftype = 4 //for bloom filter accelerators
	IndexAccelKeyFile  Ftype = 5 //when sending fully indexed accelerators
	IndexAccelDataFile Ftype = 6
	TagsUpdate         Ftype = 7
	WellTags           Ftype = 8
)

func FilenameToType ¶

func FilenameToType(name string) (ft Ftype, err error)

func (Ftype) Filename ¶

func (ft Ftype) Filename(id string) string

func (Ftype) Filepath ¶

func (ft Ftype) Filepath(id string) string

type Packer ¶

type Packer struct {
	io.ReadCloser
	sync.Mutex
	// contains filtered or unexported fields
}

func NewPacker ¶

func NewPacker(id string) (p *Packer)

func (*Packer) AddFile ¶

func (p *Packer) AddFile(tp Ftype, sz int64, rdr io.Reader) (err error)

func (*Packer) AddTags ¶

func (p *Packer) AddTags(tps []tags.TagPair) (err error)

AddTags pushes a complete list of tag pairs, this is the complete mapping of tagname to tag id. The webserver merges the tag set, potentially adding tags

func (*Packer) AddWellTags ¶

func (p *Packer) AddWellTags(tgs []string) (err error)

AddWellTags adds the list of tags that are assigned to the well every time we push a shard we push an updated set of tags that are assigned the default well will have an empty list

func (*Packer) Cancel ¶

func (p *Packer) Cancel()

func (*Packer) Close ¶

func (p *Packer) Close() (err error)

Close is used for the Nominal Close where no errors are passed to the reader

func (*Packer) CloseWithError ¶

func (p *Packer) CloseWithError(err error) error

Close with error is used to close but also send a read error to the pipe

func (*Packer) CloseWithTimeout ¶

func (p *Packer) CloseWithTimeout(to time.Duration) error

func (*Packer) Flush ¶

func (p *Packer) Flush() (err error)

func (*Packer) Read ¶

func (p *Packer) Read(b []byte) (n int, err error)

type UnpackHandler ¶

type UnpackHandler interface {
	HandleFile(string, io.Reader) error
	HandleTagUpdate([]tags.TagPair) error
}

type Unpacker ¶

type Unpacker struct {
	io.WriteCloser
	sync.Mutex
	// contains filtered or unexported fields
}

func NewUnpacker ¶

func NewUnpacker(id string, rdr io.Reader) (up *Unpacker, err error)

func (*Unpacker) Cancel ¶

func (up *Unpacker) Cancel()

func (*Unpacker) Unpack ¶

func (up *Unpacker) Unpack(uph UnpackHandler) (err error)

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL