mr

package
v0.0.0-...-a83dedc Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 17, 2022 License: MIT Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CallExample

func CallExample()

example function to show how to make an RPC call to the master.

the RPC argument and reply types are defined in rpc.go.

func DoMapTask

func DoMapTask(mapf func(string, string) []KeyValue, response *Task)

执行Map任务

func DoReduceTask

func DoReduceTask(reducef func(string, []string) string, response *Task)

执行reduce任务

func Worker

func Worker(mapf func(string, string) []KeyValue,
	reducef func(string, []string) string)

main/mrworker.go calls this function.

Types

type ExampleArgs

type ExampleArgs struct {
	X int
}

type ExampleReply

type ExampleReply struct {
	Y int
}

type KeyValue

type KeyValue struct {
	Key   string
	Value string
}

Map functions return a slice of KeyValue.

type Master

type Master struct {
	// Your definitions here.
	ReducerNum        int         //reducer数目
	TaskID            int         //task对应id
	SysPhase          Phase       //框架当前所处阶段
	MapTaskChannel    chan *Task  //Map任务队列
	ReduceTaskChannel chan *Task  //Reduce任务队列
	TaskMetaMap       TaskMetaMap //全部任务元数据Map
	FileSlice         []string    //输入文件对应切片
}

func MakeMaster

func MakeMaster(files []string, nReduce int) *Master

create a Master. main/mrmaster.go calls this function. nReduce is the number of reduce tasks to use.

func (*Master) CrashDetector

func (m *Master) CrashDetector()

检测发生crash的task并进行对应处理

func (*Master) Done

func (m *Master) Done() bool

main/mrmaster.go calls Done() periodically to find out if the entire job has finished.

func (*Master) Example

func (m *Master) Example(args *ExampleArgs, reply *ExampleReply) error

an example RPC handler.

the RPC argument and reply types are defined in rpc.go.

func (*Master) MarkFinished

func (m *Master) MarkFinished(args *Task, reply *Task) error

func (*Master) PollTask

func (m *Master) PollTask(args *TaskArgs, reply *Task) error

分发任务

type Phase

type Phase int

当前框架所处阶段type

const (
	MapPhase    Phase = iota // 框架当前处于Map阶段
	ReducePhase              // 框架当前处于Reduce阶段
	AllDone                  // 任务已全部完成
)

枚举当前框架所处阶段

type SortedKV

type SortedKV []KeyValue

func (SortedKV) Len

func (KV SortedKV) Len() int

func (SortedKV) Less

func (KV SortedKV) Less(i, j int) bool

func (SortedKV) Swap

func (KV SortedKV) Swap(i, j int)

type State

type State int

任务执行状态type

const (
	Working State = iota //任务正在工作
	Waiting              //任务正在等待
	Done                 //任务已完成
)

枚举任务状态

type Task

type Task struct {
	TaskType   Type     //任务类型
	TaskID     int      //任务ID
	ReducerNum int      //reducer数目
	FileSlice  []string //输入文件切片,map任务该项只有一个值,reduce任务则有多个中间文件名作为输入
}

func GetTask

func GetTask() Task

通过rpc获取任务

type TaskArgs

type TaskArgs struct {
}

任务参数用不到 为空

type TaskMetaInfo

type TaskMetaInfo struct {
	TaskState State     //任务状态
	StartTime time.Time //任务开始时间
	TaskAddr  *Task     //任务指针
}

任务元数据

type TaskMetaMap

type TaskMetaMap struct {
	MetaMap map[int]*TaskMetaInfo
}

Map结构保存全部任务元数据

type Type

type Type int

任务类型type

const (
	MapTask      Type = iota // 任务为Map类型
	ReduceTask               // 任务为Reduce类型
	WaittingTask             // 任务已经分发完,在等待执行完成(伪任务)
	ExitTask                 // worker可以结束(伪任务)
)

枚举任务类型

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL