base: github.com/grailbio/base/fileio Index | Files

package fileio

import "github.com/grailbio/base/fileio"

Index

Package Files

close.go named_rw.go names.go

func CloseAndReport Uses

func CloseAndReport(f io.Closer, err *error)

CloseAndReport returns a defer-able helper that calls f.Close and reports errors, if any, to *err. Pass your function's named return error. Example usage:

func processFile(filename string) (_ int, err error) {
  f, err := os.Open(filename)
  if err != nil { ... }
  defer fileio.CloseAndReport(f, &err)
  ...
}

If your function returns with an error, any f.Close error will be chained appropriately.

func FileSuffix Uses

func FileSuffix(typ FileType) string

FileSuffix returns the filename suffix associated with the specified FileType.

func IsGrailRecordio Uses

func IsGrailRecordio(ft FileType) bool

IsGrailRecordio returns true if the filetype is one of the Grail recordio types.

func MustClose Uses

func MustClose(f io.Closer)

MustClose is a defer-able function that calls f.Close and panics on error.

Example:

f, err := os.Open(filename)
if err != nil { panic(err) }
defer fileio.MustClose(f)
...

type Closer Uses

type Closer interface {
    io.Closer
    Name() string
}

Closer is io.Closer with an additional Name method that returns the name of the original source of the closer.

type FileType Uses

type FileType int

FileType represents the type of a file based on its filename

const (
    // Other represents a filetype other than the ones supported here.
    Other FileType = iota
    // Gzip file.
    Gzip
    // Bzip2 file.
    Bzip2
    // GrailRIO recordio.
    GrailRIO
    // GrailRIOPacked packed recordio.
    GrailRIOPacked
    // GrailRIOPackedEncrypted encrypted packed recordio.
    GrailRIOPackedEncrypted
    // GrailRIOPackedCompressed compressed packed recordio.
    GrailRIOPackedCompressed
    // GrailRIOPackedCompressedAndEncrypted compressed and encrypted packed recordio.
    GrailRIOPackedCompressedAndEncrypted
    // JSON text file
    JSON
    // Zstd format.
    // https://facebook.github.io/zstd/
    // https://tools.ietf.org/html/rfc8478
    Zstd
)

func DetermineType Uses

func DetermineType(filename string) FileType

DetermineType determines the type of the file given its filename.

type ReadCloser Uses

type ReadCloser interface {
    io.ReadCloser
    Name() string
}

ReadCloser is io.ReadCloser with an additional Name method that returns the name of the original source of the reader.

type ReadWriteCloser Uses

type ReadWriteCloser interface {
    io.ReadWriteCloser
    Name() string
}

ReadWriteCloser is an interface that implements io.ReadWriteCloser with an additional Name method that returns the name of the original source of the writer.

type Reader Uses

type Reader interface {
    io.Reader
    Name() string
}

Reader is io.Reader with an additional Name method that returns the name of the original source of the reader.

type StorageAPI Uses

type StorageAPI int

StorageAPI represents the Storage API required to access a file.

const (
    // LocalAPI represents a local fileystem accessible via a unix/posix API
    // and hence the io/os packages.
    LocalAPI StorageAPI = iota
    // S3API represents an Amazon S3 API.
    S3API
)

func DetermineAPI Uses

func DetermineAPI(pathname string) StorageAPI

DetermineAPI determines the Storage API that stores the file referred to by pathname.

func SpellCorrectS3 Uses

func SpellCorrectS3(s3path string) (StorageAPI, bool, string)

SpellCorrectS3 returns true if the S3 path looks like an S3 path and returns the spell corrected path. That is, it returns true for common mispellings such as those show below along with the corrected s3://<path> s3:///<path> s3:<path> s3:/<path> s://<path> s:/<path> s3//<path>

type WriteCloser Uses

type WriteCloser interface {
    io.WriteCloser
    Name() string
}

WriteCloser is io.WriteCloser with an additional Name method that returns the name of the original source of the writer.

type Writer Uses

type Writer interface {
    io.Writer
    Name() string
}

Writer is io.Writer with an additional Name method that returns the name of the original source of the writer.

Package fileio imports 5 packages (graph) and is imported by 4 packages. Updated 2019-10-14. Refresh now. Tools for package owners.