mr

package
v0.0.0-...-6aae7b8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 22, 2020 License: MIT Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Worker

func Worker(mapf func(string, string) []KeyValue,
	reducef func(string, []string) string)

main/mrworker.go calls this function.

Types

type ByKey

type ByKey []KeyValue

for sorting by key.

func (ByKey) Len

func (a ByKey) Len() int

for sorting by key.

func (ByKey) Less

func (a ByKey) Less(i, j int) bool

func (ByKey) Swap

func (a ByKey) Swap(i, j int)

type CompletedJob

type CompletedJob struct {
}

CompletedJob Empty struct, indicates that all tasks for job are completed

type ExampleArgs

type ExampleArgs struct {
	X int
}

type ExampleReply

type ExampleReply struct {
	Y int
}

type IntermediateDataRequest

type IntermediateDataRequest struct {
	TaskID int
}

IntermediateDataRequest Specifies the reduce task for which intermediate files need to be accessed

type IntermediateFiles

type IntermediateFiles struct {
	TaskID      int
	Files       []string
	Instruction int
}

IntermediateFiles Collection of intermediate files for a specific request task

type KeyValue

type KeyValue struct {
	Key   string
	Value string
}

Map functions return a slice of KeyValue.

type MapCompletion

type MapCompletion struct {
	TaskID            int
	IntermediateFiles []string
}

MapCompletion Completion details for map task. Meant to be sent to master

type MapTask

type MapTask struct {
	Filepath    string
	NReduce     int
	MapTaskID   int
	Instruction int
}

MapTask Specifies task information for map operations

type MapTaskData

type MapTaskData struct {
	State                 int
	IntermediateFilePaths []string
	FilePath              string
	TaskID                int
}

MapTaskData Stores map task state and other relevant information

type MapTaskList

type MapTaskList struct {
	// contains filtered or unexported fields
}

MapTaskList Maintains state of the map tasks

type Master

type Master struct {
	// contains filtered or unexported fields
}

Master Master server code. Holds data structures and logic for assigning and managing map, reduce tasks Its exported methods are exposed by RPC which are used by workers

func MakeMaster

func MakeMaster(files []string, nReduce int) *Master

MakeMaster create a Master. main/mrmaster.go calls this function. nReduce is the number of reduce tasks to use.

func (*Master) Done

func (m *Master) Done() bool

Done main/mrmaster.go calls Done() periodically to find out if the entire job has finished.

func (*Master) GetIntermediateFiles

func (m *Master) GetIntermediateFiles(args *IntermediateDataRequest, reply *IntermediateFiles) error

GetIntermediateFiles RPC handler, return files for reduce task

func (*Master) GetMapTask

func (m *Master) GetMapTask(args *Request, reply *MapTask) error

GetMapTask RPC handler, returns map tasks if any

func (*Master) GetReduceTask

func (m *Master) GetReduceTask(args *Request, reply *ReduceTask) error

GetReduceTask RPC handler, returns reduce task if any

func (*Master) NotifyMapTaskCompletion

func (m *Master) NotifyMapTaskCompletion(args *MapCompletion, reply *Reply) error

NotifyMapTaskCompletion RPC handler, updates a currently running task to be completed, and stores intermediate file path

func (*Master) NotifyReduceTaskCompletion

func (m *Master) NotifyReduceTaskCompletion(args *ReduceCompletion, reply *Reply) error

NotifyReduceTaskCompletion RPC handler, updates a currently running task to completed

type NoData

type NoData struct {
}

type ReduceCompletion

type ReduceCompletion struct {
	TaskID int
}

ReduceCompletion Completion details for reduce task. Meant to be sent to master

type ReduceTask

type ReduceTask struct {
	TaskID      int
	Instruction int
}

ReduceTask Specifies task information for reduce operations

type ReduceTaskData

type ReduceTaskData struct {
	State    int
	MapTasks map[int](struct{})
	TaskID   int
}

ReduceTaskData Stores reduce task state and other relevant information

type ReduceTaskList

type ReduceTaskList struct {
	// contains filtered or unexported fields
}

ReduceTaskList Maintains state of reduce tasks

type Reply

type Reply struct {
}

Reply Dummy reply, carries no information

type Request

type Request struct {
}

Request Dummy request, carries no information

type TimeStamp

type TimeStamp struct {
	Deadline time.Time
	TaskID   int
}

TimeStamp Holds time at specific point

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL