filesys

package
v0.12.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 16, 2024 License: AGPL-3.0 Imports: 18 Imported by: 1

Documentation

Overview

Package filesys provides functions to operate general files and filesystems.

The functions in this package are for abstract files and filesystems defined in package io/fs. To get functions for local files and filesystems, see its subpackage local.

For better performance, all functions in this package and its subpackages are unsafe for concurrency unless otherwise specified.

Index

Constants

This section is empty.

Variables

ErrFileReaderClosed is an error indicating that the file reader is already closed.

The client should use errors.Is to test whether an error is ErrFileReaderClosed.

ErrFileWriterClosed is an error indicating that the file writer is already closed.

The client should use errors.Is to test whether an error is ErrFileWriterClosed.

View Source
var ErrIsDir = errors.AutoNewCustom(
	"file is a directory",
	errors.PrependFullPkgName,
	0,
)

ErrIsDir is an error indicating that the file is a directory.

The client should use errors.Is to test whether an error is ErrIsDir.

View Source
var ErrNotTar = errors.AutoNewCustom(
	"file is not archived by tar or is opened in raw mode",
	errors.PrependFullPkgName,
	0,
)

ErrNotTar is an error indicating that the file is not archived by tar, or is opened in raw mode.

The client should use errors.Is to test whether an error is ErrNotTar.

View Source
var ErrNotZip = errors.AutoNewCustom(
	"file is not archived by ZIP or is opened in raw mode",
	errors.PrependFullPkgName,
	0,
)

ErrNotZip is an error indicating that the file is not archived by ZIP, or is opened in raw mode.

The client should use errors.Is to test whether an error is ErrNotZip.

View Source
var ErrReadZip = errors.AutoNewCustom(
	"read a ZIP archive",
	errors.PrependFullPkgName,
	0,
)

ErrReadZip is an error indicating that a read method of Reader is called on a ZIP archive.

The client should use errors.Is to test whether an error is ErrReadZip.

View Source
var ErrZipWriteBeforeCreate = errors.AutoNewCustom(
	"call a write method before creating a new ZIP file",
	errors.PrependFullPkgName,
	0,
)

ErrZipWriteBeforeCreate is an error indicating that for a ZIP archive, a write method of Writer is called before creating a new ZIP file.

The client should use errors.Is to test whether an error is ErrZipWriteBeforeCreate.

Functions

func Checksum

func Checksum(
	file fs.File,
	closeFile bool,
	upper bool,
	newHashes ...func() hash.Hash,
) (checksums []string, err error)

Checksum calculates hash checksums of the specified file, and returns the result in hexadecimal representation and any read error encountered.

If the file is a directory, Checksum reports ErrIsDir and returns nil checksums. (To test whether err is ErrIsDir, use function errors.Is.)

To ensure that this function can work as expected, the input file must be ready to be read from the beginning and must not be operated by anyone else during the call to this function.

closeFile indicates whether this function should close the file. If closeFile is false, the client is responsible for closing file after use. If closeFile is true, file is closed by this function.

upper indicates whether to use uppercase in hexadecimal representation.

newHashes are functions that create new hash functions (e.g., crypto/sha256.New, crypto.SHA256.New).

The length of the returned checksums is the same as that of newHashes. The hash result of newHashes[i] is checksums[i], encoded in hexadecimal. In particular, if newHashes[i] is nil or returns nil, checksums[i] is an empty string. If len(newHashes) is 0, checksums is nil.

This function panics if file is nil.

func ChecksumFromFS added in v0.6.0

func ChecksumFromFS(
	fsys fs.FS,
	name string,
	upper bool,
	newHashes ...func() hash.Hash,
) (checksums []string, err error)

ChecksumFromFS calculates hash checksums of the file opened from fsys by specified name, and returns the result in hexadecimal representation and any error encountered during opening and reading the file.

If the file is a directory, ChecksumFromFS reports ErrIsDir and returns nil checksums. (To test whether err is ErrIsDir, use function errors.Is.)

upper indicates whether to use uppercase in hexadecimal representation.

newHashes are functions that create new hash functions (e.g., crypto/sha256.New, crypto.SHA256.New).

The length of the returned checksums is the same as that of newHashes. The hash result of newHashes[i] is checksums[i], encoded in hexadecimal. In particular, if newHashes[i] is nil or returns nil, checksums[i] is an empty string. If len(newHashes) is 0, checksums is nil.

This function panics if fsys is nil.

func VerifyChecksum

func VerifyChecksum(file fs.File, closeFile bool, hvs ...HashVerifier) bool

VerifyChecksum verifies a file by hash checksum.

To ensure that this function can work as expected, the input file must be ready to be read from the beginning and must not be operated by anyone else during the call to this function.

closeFile indicates whether this function should close the file. If closeFile is false, the client is responsible for closing file after use. If closeFile is true and file is not nil, file is closed by this function.

It returns true if the file is not nil and can be read, and matches all HashVerifier in hvs (nil and duplicate HashVerifier are ignored). In particular, it returns true if there is no non-nil HashVerifier in hvs. In this case, the file is not read.

Note that VerifyChecksum does not reset the hash state of anyone in hvs. The client should use new HashVerifier returned by NewHashVerifier or call the Reset method of HashVerifier before calling this function if needed.

func VerifyChecksumFromFS added in v0.5.0

func VerifyChecksumFromFS(fsys fs.FS, name string, hvs ...HashVerifier) bool

VerifyChecksumFromFS verifies a file by hash checksum, where the file is opened from fsys by specified name.

It returns true if fsys is not nil, and the file can be read and matches all HashVerifier in hvs (nil and duplicate HashVerifier are ignored). In particular, it returns true if there is no non-nil HashVerifier in hvs and the file can be opened for reading. In this case, the file is not read.

Note that VerifyChecksumFromFS does not reset the hash state of anyone in hvs. The client should use new HashVerifier returned by NewHashVerifier or call the Reset method of HashVerifier before calling this function if needed.

Types

type HashVerifier added in v0.6.0

type HashVerifier interface {
	hash.Hash

	// Match reports whether the current hash (as returned by Sum(nil))
	// matches the pre-specified prefix.
	Match() bool
}

HashVerifier extends hash.Hash by adding a Match method to report whether the hash result matches a pre-specified prefix.

func NewHashVerifier added in v0.6.0

func NewHashVerifier(newHash func() hash.Hash, prefixHex string) HashVerifier

NewHashVerifier creates a new HashVerifier with specified arguments.

newHash is a function that creates a new hash function (e.g., crypto/sha256.New, crypto.SHA256.New).

prefixHex is the prefix of the expected hash result, in hexadecimal representation.

It panics if newHash is nil or returns nil, or prefixHex is not hexadecimal.

type ReadOptions

type ReadOptions struct {
	// Size of the buffer for reading the file at least.
	// Non-positive values for using default value.
	BufSize int

	// Offset of the file to read, in bytes,
	// relative to the origin of the file for positive values,
	// and relative to the end of the file for negative values.
	Offset int64

	// Limit of the file to read, in bytes.
	// Non-positive values for no limit.
	Limit int64

	// True if not to decompress when the file is compressed by gzip or bzip2,
	// and not to restore when the file is archived by tar (i.e., tape archive).
	Raw bool

	// A method-decompressor map for reading the ZIP archive.
	// These decompressors are registered to the archive/zip.Reader.
	// (Nil decompressors are ignored.)
	//
	// Package archive/zip has two built-in decompressors for the common methods
	// archive/zip.Store (0) and archive/zip.Deflate (8).
	//
	// For more details, see the documentation of the method
	// RegisterDecompressor of archive/zip.Reader and
	// the function archive/zip.RegisterDecompressor.
	ZipDcomp map[uint16]zip.Decompressor

	// A function that wraps the reader to io.ReaderAt
	// to create an archive/zip.Reader.
	//
	// It also returns the size that can be read, and any error encountered.
	//
	// It will only be called when the reader does not implement io.ReaderAt.
	ZipReaderAtFunc func(r io.Reader) (ra io.ReaderAt, size int64, err error)
}

ReadOptions are options for Read functions.

type Reader

type Reader interface {
	inout.Closer
	inout.BufferedReader

	// TarEnabled returns true if the file is archived by tar
	// (i.e., tape archive) and is not opened in raw mode.
	TarEnabled() bool

	// TarNext advances to the next entry in the tar archive.
	//
	// The tar.Header.Size determines how many bytes can be read
	// for the next file.
	// Any remaining data in current file is automatically discarded.
	//
	// io.EOF is returned at the end of the input.
	//
	// If the file is not archived by tar or is opened in raw mode,
	// it does nothing and reports ErrNotTar.
	// (To test whether err is ErrNotTar, use function errors.Is.)
	TarNext() (hdr *tar.Header, err error)

	// ZipEnabled returns true if the file is archived by ZIP
	// and is not opened in raw mode.
	ZipEnabled() bool

	// ZipOpen opens the file with specified name in the ZIP archive.
	//
	// The name must be a relative path: it must not start with a drive letter
	// (e.g., "C:") or leading slash. Only forward slashes are allowed;
	// "../" elements and trailing slashes (e.g., "a/") are not allowed.
	//
	// If the reader's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipOpen(name string) (file fs.File, err error)

	// ZipFiles returns the files in the ZIP archive, sorted by filename.
	//
	// If the reader's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipFiles() (files []*zip.File, err error)

	// ZipComment returns the end-of-central-directory comment field
	// of the ZIP archive.
	//
	// If the reader's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipComment() (comment string, err error)

	// Options returns a copy of options used by this reader.
	Options() *ReadOptions

	// FileStat returns the io/fs.FileInfo structure describing file.
	FileStat() (info fs.FileInfo, err error)
}

Reader is a device to read data from a file.

Its method Close closes all closable objects opened by this reader (may include the file). After successfully closing this reader, its method Close does nothing and returns nil, and its read methods report ErrFileReaderClosed. (To test whether the error is ErrFileReaderClosed, use function errors.Is.)

func Read

func Read(file fs.File, opts *ReadOptions, closeFile bool) (
	r Reader, err error)

Read creates a reader on the specified file with options opts.

If the file is a directory, Read reports ErrIsDir and returns a nil Reader. (To test whether err is ErrIsDir, use function errors.Is.)

If opts are nil, a zero-value ReadOptions is used.

To ensure that this function and the returned reader can work as expected, the input file must not be operated by anyone else before closing the returned reader. If the option Offset is non-zero and the file is not an io.Seeker, the file must be ready to be read from the beginning.

closeFile indicates whether the reader should close the file when calling its method Close. If closeFile is false, the client is responsible for closing file after closing the reader. If closeFile is true, the client should not close the file, even if this function reports an error. In this case, the file is closed during the method Close of the reader, and it is also closed by this function when encountering an error.

This function panics if file is nil.

func ReadFromFS added in v0.5.0

func ReadFromFS(fsys fs.FS, name string, opts *ReadOptions) (
	r Reader, err error)

ReadFromFS opens a file from fsys with specified name and options opts for reading.

If the file is a directory, ReadFromFS reports ErrIsDir and returns a nil Reader. (To test whether err is ErrIsDir, use function errors.Is.)

If opts are nil, a zero-value ReadOptions is used.

The file is closed when the returned reader is closed.

This function panics if fsys is nil.

type WritableFile added in v0.6.0

type WritableFile interface {
	io.WriteCloser

	// Stat returns the io/fs.FileInfo structure describing file.
	Stat() (info fs.FileInfo, err error)
}

WritableFile represents a single file that can be written to.

It is similar to io/fs.File but has the method Write instead of Read.

type WriteOptions added in v0.6.0

type WriteOptions struct {
	// Size of the buffer for writing the file at least.
	// Non-positive values for using default value.
	BufSize int

	// True if not to compress the file with gzip and not to archive the file
	// with tar (i.e., tape archive) according to the file extension.
	Raw bool

	// The compression level of DEFLATE.
	// It should be in the range [-2, 9].
	// The zero value (0) stands for no compression
	// other than the default value.
	// To use the default value, set it to compress/flate.DefaultCompression.
	// For more details, see the documentation of compress/flate.NewWriter.
	//
	// This option only takes effect when DEFLATE compression is applied.
	// That is, when Raw is false, and the file extension is ".gz" or ".tgz",
	// or ".zip" and the ZIP archive uses DEFLATE compression.
	DeflateLv int

	// The offset of the beginning of the ZIP data within the underlying writer.
	// It should be used when the ZIP data is appended to an existing file,
	// such as a binary executable.
	//
	// Non-positive values are ignored.
	ZipOffset int64

	// The end-of-central-directory comment field of the ZIP archive.
	// It should be 65535 bytes at most.
	// If the comment is too long, an error is reported.
	ZipComment string

	// A method-compressor map for writing the ZIP archive.
	// These compressors are registered to the archive/zip.Writer.
	// (Nil compressors are ignored.)
	//
	// Package archive/zip has two built-in compressors for the common methods
	// archive/zip.Store (0) and archive/zip.Deflate (8).
	//
	// For more details, see the documentation of the method RegisterCompressor
	// of archive/zip.Writer and the function archive/zip.RegisterCompressor.
	ZipComp map[uint16]zip.Compressor
}

WriteOptions are options for Write functions.

type Writer added in v0.6.0

type Writer interface {
	inout.Closer
	inout.BufferedWriter

	// TarEnabled returns true if the file is archived by tar
	// (i.e., tape archive) and is not opened in raw mode.
	TarEnabled() bool

	// TarWriteHeader writes hdr and prepares to accept the content of
	// the next file.
	//
	// The tar.Header.Size determines how many bytes can be written for
	// the next file.
	// If the current file is not fully written, it returns an error.
	// It implicitly flushes any padding necessary before writing the header.
	//
	// If the file is not archived by tar or is opened in raw mode,
	// it does nothing and reports ErrNotTar.
	// (To test whether the error is ErrNotTar, use function errors.Is.)
	TarWriteHeader(hdr *tar.Header) error

	// TarAddFS adds the files from the specified filesystem
	// to the tape archive.
	// It walks the directory tree starting at the root of the filesystem
	// adding each file to the tape archive
	// while maintaining the directory structure.
	//
	// If the file is not archived by tar or is opened in raw mode,
	// it does nothing and reports ErrNotTar.
	// (To test whether the error is ErrNotTar, use function errors.Is.)
	TarAddFS(fsys fs.FS) error

	// ZipEnabled returns true if the file is archived by ZIP
	// and is not opened in raw mode.
	ZipEnabled() bool

	// ZipCreate adds a file with specified name to the ZIP archive and
	// switches the writer to that file.
	//
	// The file contents are compressed with DEFLATE.
	//
	// The name must be a relative path: it must not start with a drive letter
	// (e.g., "C:") or leading slash. Only forward slashes are allowed;
	// "../" elements are not allowed.
	// To create a directory instead of a file, add a trailing slash
	// to the name (e.g., "dir/").
	//
	// The file's contents must be written to the writer before the next call
	// to ZipCreate, ZipCreateHeader, ZipCreateRaw, ZipCopy, or Close.
	//
	// If the writer's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipCreate(name string) error

	// ZipCreateHeader adds a file to the ZIP archive using the specified
	// file header for the file metadata and switches the writer to that file.
	//
	// The writer takes ownership of fh and may mutate its fields.
	// The client must not modify fh after calling ZipCreateHeader.
	//
	// The file's contents must be written to the writer before the next call
	// to ZipCreate, ZipCreateHeader, ZipCreateRaw, ZipCopy, or Close.
	//
	// If the writer's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipCreateHeader(fh *zip.FileHeader) error

	// ZipCreateRaw is like ZipCreateHeader,
	// but the bytes passed to the writer are not compressed.
	ZipCreateRaw(fh *zip.FileHeader) error

	// ZipCopy copies the file f (obtained from an archive/zip.Reader)
	// into the writer.
	//
	// It copies the raw form directly bypassing
	// decompression, compression, and validation.
	//
	// If the writer's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipCopy(f *zip.File) error

	// ZipAddFS adds the files from the specified filesystem
	// to the ZIP archive.
	// It walks the directory tree starting at the root of the filesystem
	// adding each file to the ZIP using deflate
	// while maintaining the directory structure.
	//
	// If the writer's file is not archived by ZIP or is opened in raw mode,
	// it does nothing and reports ErrNotZip.
	// (To test whether the error is ErrNotZip, use function errors.Is.)
	ZipAddFS(fsys fs.FS) error

	// Options returns a copy of options used by this writer.
	Options() *WriteOptions

	// FileStat returns the io/fs.FileInfo structure describing file.
	FileStat() (info fs.FileInfo, err error)
}

Writer is a device to write data to a local file.

Its method Close closes all closable objects opened by this writer (may include the file). After successfully closing this writer, its method Close does nothing and returns nil, and its write methods report ErrFileWriterClosed. (To test whether the error is ErrFileWriterClosed, use function errors.Is.)

func Write added in v0.6.0

func Write(file WritableFile, opts *WriteOptions, closeFile bool) (
	w Writer, err error)

Write creates a writer on the specified file with options opts.

If the file is a directory, Write reports ErrIsDir and returns a nil Writer. (To test whether err is ErrIsDir, use function errors.Is.)

If opts are nil, the default options are used. The default options are as follows:

  • BufSize: 0
  • Raw: false
  • DeflateLv: compress/flate.BestCompression
  • ZipOffset: 0
  • ZipComment: ""
  • ZipComp: nil

To ensure that this function and the returned writer can work as expected, the specified file must not be operated by anyone else before closing the returned writer.

closeFile indicates whether the writer should close the file when calling its method Close. If closeFile is false, the client is responsible for closing file after closing the writer. If closeFile is true, the client should not close the file, even if this function reports an error. In this case, the file is closed during the method Close of the writer, and it is also closed by this function when encountering an error.

This function panics if file is nil.

Directories

Path Synopsis
Package local provides functions to operate local files and local filesystems.
Package local provides functions to operate local files and local filesystems.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL