mapreduce

package

v0.4.1 Latest Latest Go to latest Published: Feb 20, 2015 License: Apache-2.0 Imports: 16 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/jwayong2/pfs

Links

Open Source Insights

Documentation ¶

Index ¶

Constants
func Map(job Job, jobPath string, m materializeInfo, host string, shard, modulos uint64)
func Materialize(in_repo, branch, commit, out_repo, jobDir string, shard, modulos uint64) error
func PrepJob(job Job, jobPath string, m materializeInfo) error
func Reduce(job Job, jobPath string, m materializeInfo, host string, shard, modulos uint64)
func WaitJob(in_repo, commit, job string)
type Job

Constants ¶

View Source

const (
	ProtoPfs = iota
	ProtoS3  = iota
)

Variables ¶

This section is empty.

Functions ¶

func Map ¶

func Map(job Job, jobPath string, m materializeInfo, host string, shard, modulos uint64)

func Materialize ¶

func Materialize(in_repo, branch, commit, out_repo, jobDir string, shard, modulos uint64) error

Materialize parses the jobs found in `in_repo`/`commit`/`jobDir` runs them with `in_repo/commit` as input, outputs the results to `out_repo`/`branch` and commits them as `out_repo`/`commit`

func PrepJob ¶

func PrepJob(job Job, jobPath string, m materializeInfo) error

func Reduce ¶

func Reduce(job Job, jobPath string, m materializeInfo, host string, shard, modulos uint64)

func WaitJob ¶

func WaitJob(in_repo, commit, job string)

Types ¶

type Job ¶

type Job struct {
	Type     string   `json:"type"`
	Input    string   `json:"input"`
	Image    string   `json:"image"`
	Command  []string `json:"command"`
	Limit    int      `json:"limit"`
	Parallel int      `json:"parallel"`
}

Source Files ¶

View all Source files

mapreduce.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL