tools

package
v0.0.0-...-bff2b11 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 27, 2022 License: MIT Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var NewFile = errors.New("Create a new file")

Functions

func Madvise

func Madvise(b []byte, readahead bool) error

Madvise uses the madvise system call to give advise about the use of memory when using a slice that is memory-mapped to a file. Set the readahead flag to false if page references are expected in random order.

func Mmap

func Mmap(fd *os.File, writable bool, size int64) ([]byte, error)

Mmap uses the mmap system call to memory-map a file. If writable is true, memory protection of the pages is set so that they may be written to as well.

func Msync

func Msync(b []byte) error

Msync would call sync on the mmapped data.

func Munmap

func Munmap(b []byte) error

Munmap unmaps a previously mapped slice.

func SyncDir

func SyncDir(dir string) error

Types

type Buffer

type Buffer struct {
	// contains filtered or unexported fields
}

Buffer is equivalent of bytes.Buffer without the ability to read. It is NOT thread-safe.

In UseMmap mode, Buffer uses file mmap to allocate memory. This allows us to store big data structures without using physical memory.

MaxSize can be set to limit the memory usage.

type MmapFile

type MmapFile struct {
	Data []byte
	Fd   *os.File
}

MmapFile represents an mmapd file and includes both the buffer to the data and the file descriptor.

func OpenMmapFile

func OpenMmapFile(filename string, flag int, maxSz int) (*MmapFile, error)

OpenMmapFile opens an existing file or creates a new file. If the file is created, it would truncate the file to maxSz. In both cases, it would mmap the file to maxSz and returned it. In case the file is created, z.NewFile is returned.

func OpenMmapFileUsing

func OpenMmapFileUsing(fd *os.File, sz int, writable bool) (*MmapFile, error)

func (*MmapFile) AllocateSlice

func (m *MmapFile) AllocateSlice(sz, offset int) ([]byte, int, error)

AllocateSlice allocates a slice of the given size at the given offset.

func (*MmapFile) Bytes

func (m *MmapFile) Bytes(off, sz int) ([]byte, error)

Bytes returns data starting from offset off of size sz. If there's not enough data, it would return nil slice and io.EOF.

func (*MmapFile) Close

func (m *MmapFile) Close(maxSz int64) error

Close would close the file. It would also truncate the file if maxSz >= 0.

func (*MmapFile) Delete

func (m *MmapFile) Delete() error

func (*MmapFile) NewReader

func (m *MmapFile) NewReader(offset int) io.Reader

func (*MmapFile) Slice

func (m *MmapFile) Slice(offset int) []byte

Slice returns the slice at the given offset.

func (*MmapFile) Sync

func (m *MmapFile) Sync() error

func (*MmapFile) Truncate

func (m *MmapFile) Truncate(maxSz int64) error

Truncate would truncate the mmapped file to the given size. On Linux, we truncate the underlying file and then call mremap, but on other systems, we unmap first, then truncate, then re-map.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL