filesystem

package
v2.32.0+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 10, 2021 License: Apache-2.0, BSD-3-Clause, MIT Imports: 6 Imported by: 0

Documentation

Overview

Package filesystem contains an extensible file system abstraction. It allows various kinds of storage systems to be used uniformly, notably through textio.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Read

func Read(ctx context.Context, fs Interface, filename string) ([]byte, error)

Read fully reads the given file from the file system.

func Register

func Register(scheme string, fs func(context.Context) Interface)

Register registers a file system backend under the given scheme. For example, "hdfs" would be registered a HFDS file system and HDFS paths used transparently.

func ValidateScheme

func ValidateScheme(path string)

ValidateScheme panics if the given path's scheme does not have a corresponding file system registered.

func Write

func Write(ctx context.Context, fs Interface, filename string, data []byte) error

Write writes the given content to the file system.

Types

type Interface

type Interface interface {
	io.Closer

	// List expands a patten to a list of filenames.
	List(ctx context.Context, glob string) ([]string, error)

	// OpenRead opens a file for reading.
	OpenRead(ctx context.Context, filename string) (io.ReadCloser, error)
	// OpenRead opens a file for writing. If the file already exist, it will be
	// overwritten.
	OpenWrite(ctx context.Context, filename string) (io.WriteCloser, error)
	// Size returns the size of a file in bytes.
	Size(ctx context.Context, filename string) (int64, error)
}

Interface is a filesystem abstraction that allows beam io sources and sinks to use various underlying storage systems transparently.

func New

func New(ctx context.Context, path string) (Interface, error)

New returns a new Interface for the given file path's scheme.

Directories

Path Synopsis
Package gcs contains a Google Cloud Storage (GCS) implementation of the Beam file system.
Package gcs contains a Google Cloud Storage (GCS) implementation of the Beam file system.
Package local contains a local file implementation of the Beam file system.
Package local contains a local file implementation of the Beam file system.
Package memfs contains a in-memory Beam filesystem.
Package memfs contains a in-memory Beam filesystem.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL