mr

package
v0.0.0-...-ebc10ec Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 29, 2021 License: MIT Imports: 15 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Worker

func Worker(mapf func(string, string) []KeyValue,
	reducef func(string, []string) string)

main/mrworker.go calls this function.

Types

type ByKey

type ByKey []KeyValue

for sorting by key.

func (ByKey) Len

func (a ByKey) Len() int

for sorting by key.

func (ByKey) Less

func (a ByKey) Less(i, j int) bool

func (ByKey) Swap

func (a ByKey) Swap(i, j int)

type Empty

type Empty struct{}

Empty empty structure

type JobDoneRequest

type JobDoneRequest struct {
	Id int
}

JobDoneRequest if job is done return job id

type KeyValue

type KeyValue struct {
	Key   string
	Value string
}

Map functions return a slice of KeyValue.

type Master

type Master struct {
	Tasks   []Task
	NReduce int
	// contains filtered or unexported fields
}

Master master node internal state, logically represent a single MapReduce job

func MakeMaster

func MakeMaster(files []string, nReduce int) *Master

create a Master. main/mrmaster.go calls this function. nReduce is the number of reduce tasks to use.

func (*Master) Done

func (m *Master) Done() bool

main/mrmaster.go calls Done() periodically to find out if the entire job has finished.

func (*Master) TaskDone

func (m *Master) TaskDone(args *JobDoneRequest, reply *Empty) error

TaskDone worker calls this when map/reduce task is finished

func (*Master) WantTask

func (m *Master) WantTask(args *Empty, reply *NewJobReply) error

WantTask worker requests for a job

type NewJobReply

type NewJobReply struct {
	Filename string
	Type     string // "map" or "reduce"
	NReduce  int
	Id       int
}

NewJobReply get filename to process

type Task

type Task struct {
	Filename string
	State    string // "idle", "in-progress" or "completed"; according to MapReduce paper
	Type     string // "map" or "reduce"
	Id       int
	// contains filtered or unexported fields
}

Task defines map or reduce operation

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL