explode

package

v0.0.0-...-63fa78c Latest Latest Go to latest Published: Nov 7, 2017 License: MIT Imports: 16 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/recogni/sorten

Links

Open Source Insights

Documentation ¶

Overview ¶

Package explode wraps all needed logic to run a parallel job that will take a directory of tf records and explode them into their respective classes. Each record found is split into its constituent record pieces and are streamed into the output directory based on the classes requested in the `--filter`.

Index ¶

Variables
func RunJob(nworkers int, args []string, wl *logger.WorkerLogger) error
type AtomicClassNamer
- func (a *AtomicClassNamer) GetNextIndexForClass(id int) int

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	CLI = struct {
		// Command line args common to all commands
		inputDir  string
		outputDir string
		filter    string

		// Private stuff
		numWorkers int
		classes    []int
	}{}
)

Functions ¶

func RunJob ¶

func RunJob(nworkers int, args []string, wl *logger.WorkerLogger) error

Types ¶

type AtomicClassNamer ¶

type AtomicClassNamer struct {
	*sync.RWMutex
	// contains filtered or unexported fields
}

func (*AtomicClassNamer) GetNextIndexForClass ¶

func (a *AtomicClassNamer) GetNextIndexForClass(id int) int

GetNextIndexForClass returns the next valid integer value for the next tf record to be named in a given directory of said `id`.

Source Files ¶

View all Source files

explode.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL