package filesystem

import "github.com/apache/beam/sdks/go/pkg/beam/io/filesystem"

Package filesystem contains an extensible file system abstraction. It allows various kinds of storage systems to be used uniformly, notably through textio.

Index ¶

• func Read(ctx context.Context, fs Interface, filename string) ([]byte, error)
• func Register(scheme string, fs func(context.Context) Interface)
• func ValidateScheme(path string)
• func Write(ctx context.Context, fs Interface, filename string, data []byte) error
• type Interface
• func New(ctx context.Context, path string) (Interface, error)

Package Files ¶

filesystem.go util.go

func Read ¶ Uses

func Read(ctx context.Context, fs Interface, filename string) ([]byte, error)

Read fully reads the given file from the file system.

func Register ¶ Uses

func Register(scheme string, fs func(context.Context) Interface)

Register registers a file system backend under the given scheme. For example, "hdfs" would be registered a HFDS file system and HDFS paths used transparently.

func ValidateScheme ¶ Uses

func ValidateScheme(path string)

ValidateScheme panics if the given path's scheme does not have a corresponding file system registered.

func Write ¶ Uses

func Write(ctx context.Context, fs Interface, filename string, data []byte) error

Write writes the given content to the file system.

type Interface ¶ Uses

type Interface interface {
    io.Closer

    // List expands a patten to a list of filenames.
    List(ctx context.Context, glob string) ([]string, error)

    // OpenRead opens a file for reading.
    OpenRead(ctx context.Context, filename string) (io.ReadCloser, error)
    // OpenRead opens a file for writing. If the file already exist, it will be
    // overwritten.
    OpenWrite(ctx context.Context, filename string) (io.WriteCloser, error)
    // Size returns the size of a file in bytes.
    Size(ctx context.Context, filename string) (int64, error)
}

Interface is a filesystem abstraction that allows beam io sources and sinks to use various underlying storage systems transparently.

func New ¶ Uses

func New(ctx context.Context, path string) (Interface, error)

New returns a new Interface for the given file path's scheme.

Directories ¶