stor

package

v0.0.0-...-62c91b2 Latest Latest Go to latest Published: Apr 30, 2024 License: MIT Imports: 15 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/apmckinlay/gsuneido

Links

Open Source Insights

Documentation ¶

Rendered for

Overview ¶

Package stor is used to access physical storage, normally by memory mapped file access.

Storage is chunked. Allocations may not straddle chunks.

Index ¶

Constants
func AppendSmallOffset(buf []byte, offset uint64) []byte
func LenStr(s string) int
func LenStrs(ss []string) int
func ReadSmallOffset(buf []byte) uint64
func WriteSmallOffset(buf []byte, offset uint64)
type Mode
type Offset
type Reader
- func NewReader(buf []byte) *Reader
type SmallOffset
type Stor
type Writer
- func NewWriter(buf []byte) *Writer

Constants ¶

View Source

const MaxSmallOffset = 1<<40 - 1

View Source

const SmallOffsetLen = 5

Variables ¶

This section is empty.

Functions ¶

func AppendSmallOffset ¶

func AppendSmallOffset(buf []byte, offset uint64) []byte

func LenStr ¶

func LenStr(s string) int

LenStr returns the space requirecd by PutStr

func LenStrs ¶

func LenStrs(ss []string) int

LenStrs returns the space requirecd by PutStrs

func ReadSmallOffset ¶

func ReadSmallOffset(buf []byte) uint64

func WriteSmallOffset ¶

func WriteSmallOffset(buf []byte, offset uint64)

Types ¶

type Mode ¶

type Mode int

const (
	Read Mode = iota
	Create
	Update
)

type Reader ¶

type Reader struct {
	// contains filtered or unexported fields
}

func NewReader ¶

func NewReader(buf []byte) *Reader

func (*Reader) Get1 ¶

func (r *Reader) Get1() int

Get1 reads an unsigned byte value

func (*Reader) Get2 ¶

func (r *Reader) Get2() int

Get2 reads an unsigned two byte value

func (*Reader) Get3 ¶

func (r *Reader) Get3() int

Get3 reads an unsigned three byte value

func (*Reader) Get4 ¶

func (r *Reader) Get4() int

Get4 reads an unsigned four byte value

func (*Reader) Get5 ¶

func (r *Reader) Get5() uint64

Get5 reads an unsigned five byte value

func (*Reader) GetStr ¶

func (r *Reader) GetStr() string

GetStr reads a string

func (*Reader) GetStrs ¶

func (r *Reader) GetStrs() []string

GetStrs reads a slice of strings

func (*Reader) Remaining ¶

func (r *Reader) Remaining() int

Remaining returns the number of unread bytes left

type SmallOffset ¶

type SmallOffset [SmallOffsetLen]byte

SmallOffset is used to store database offsets in the database file to save space rather than using int64. 5 bytes = 40 bits = 1tb In memory we use int64. Beware of padding or you won't actually save space.

type Stor ¶

type Stor struct {
	// contains filtered or unexported fields
}

Stor is the externally visible storage

func HeapStor ¶

func HeapStor(chunksize int) *Stor

HeapStor returns an empty in-memory stor for testing.

func MmapStor ¶

func MmapStor(filename string, mode Mode) (*Stor, error)

MmapStor returns a memory mapped file stor.

func NewStor ¶

func NewStor(impl storage, chunksize uint64, size uint64, chunks [][]byte) *Stor

func (*Stor) Alloc ¶

func (s *Stor) Alloc(n int) (Offset, []byte)

Alloc allocates n bytes of storage and returns its Offset and byte slice Returning data here allows slicing to the correct length and capacity to prevent erroneously writing too far. If insufficient room in the current chunk, advance to next (allocations may not straddle chunks). Usual case is just an atomic increment, and an atomic load to check if hit the end of the chunk. Locking is only used to extend the storage.

func (*Stor) Close ¶

func (s *Stor) Close(unmap bool, callback ...func(uint64))

func (*Stor) Data ¶

func (s *Stor) Data(offset Offset) []byte

Data returns a byte slice starting at the given offset and extending to the end of the chunk since we don't know the size of the original alloc.

func (*Stor) FirstOffset ¶

func (s *Stor) FirstOffset(off uint64, str string) uint64

FirstOffset searches forewards from a given offset for a given byte slice and returns the offset, or 0 if not found

func (*Stor) Flush ¶

func (s *Stor) Flush()

func (*Stor) LastOffset ¶

func (s *Stor) LastOffset(off uint64, str string) uint64

LastOffset searches backwards from a given offset for a given byte slice and returns the offset, or 0 if not found. It is used by repair and by asof/history.

func (*Stor) Reader ¶

func (s *Stor) Reader(off uint64) *Reader

func (*Stor) Size ¶

func (s *Stor) Size() uint64

Size returns the current (allocated) size of the data. The actual file size will be rounded up to the next chunk size.

func (*Stor) Write ¶

func (s *Stor) Write(off uint64, data []byte)

type Writer ¶

type Writer struct {
	// contains filtered or unexported fields
}

func NewWriter ¶

func NewWriter(buf []byte) *Writer

NewWriter returns a new Writer on a byte slice

func (*Writer) Len ¶

func (w *Writer) Len() int

Len returns the current position within this writer

func (*Writer) Put1 ¶

func (w *Writer) Put1(n int) *Writer

Put1 writes an unsigned byte value

func (*Writer) Put2 ¶

func (w *Writer) Put2(n int) *Writer

Put2 writes an unsigned two byte value

func (*Writer) Put3 ¶

func (w *Writer) Put3(n int) *Writer

Put3 writes an unsigned three byte value

func (*Writer) Put4 ¶

func (w *Writer) Put4(n int) *Writer

Put4 writes an unsigned four byte value

func (*Writer) Put5 ¶

func (w *Writer) Put5(n uint64) *Writer

Put5 writes an unsigned five byte value

func (*Writer) PutStr ¶

func (w *Writer) PutStr(s string) *Writer

PutStr writes a string with a maximum length of 64k

func (*Writer) PutStrs ¶

func (w *Writer) PutStrs(ss []string) *Writer

PutStrs writes a slice of up to 64k strings

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL