beam: github.com/apache/beam/sdks/go/pkg/beam/io/textio Index | Files

package textio

import "github.com/apache/beam/sdks/go/pkg/beam/io/textio"

Package textio contains transforms for reading and writing text files.

Index

Package Files

sdf.go textio.go

func Immediate Uses

❖
func Immediate(s beam.Scope, filename string) (beam.PCollection, error)

Immediate reads a local file at pipeline construction-time and embeds the data into a I/O-free pipeline source. Should be used for small files only.

func Read Uses

❖
func Read(s beam.Scope, glob string) beam.PCollection

Read reads a set of file and returns the lines as a PCollection<string>. The newlines are not part of the lines.

func ReadAll Uses

❖
func ReadAll(s beam.Scope, col beam.PCollection) beam.PCollection

ReadAll expands and reads the filename given as globs by the incoming PCollection<string>. It returns the lines of all files as a single PCollection<string>. The newlines are not part of the lines.

func ReadAllSdf Uses

❖
func ReadAllSdf(s beam.Scope, col beam.PCollection) beam.PCollection

ReadAllSdf is a variation of ReadAll implemented via SplittableDoFn. This should result in increased performance with runners that support splitting.

func ReadSdf Uses

❖
func ReadSdf(s beam.Scope, glob string) beam.PCollection

ReadSdf is a variation of Read implemented via SplittableDoFn. This should result in increased performance with runners that support splitting.

func Write Uses

❖
func Write(s beam.Scope, filename string, col beam.PCollection)

Write writes a PCollection<string> to a file as separate lines. The writer add a newline after each element.

Package textio imports 12 packages (graph) and is imported by 8 packages. Updated 2020-11-23. Refresh now. Tools for package owners.