mapreduce

package

v0.0.0-...-34b08cd Latest Latest Go to latest Published: Jun 23, 2022 License: MIT Imports: 10 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/hakusai22/6.824

Links

Open Source Insights

Documentation ¶

Index ¶

func RunWorker(MasterAddress string, me string, MapFunc func(string, string) []KeyValue, ...)
type KeyValue
type Master
- func Distributed(jobName string, files []string, nreduce int, master string) (mr *Master)
- func Sequential(jobName string, files []string, nreduce int, ...) (mr *Master)
type Parallelism
type Worker
- func (wk *Worker) DoTask(arg *DoTaskArgs, _ *struct{}) error
- func (wk *Worker) Shutdown(_ *struct{}, res *ShutdownReply) error

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func RunWorker ¶

func RunWorker(MasterAddress string, me string,
	MapFunc func(string, string) []KeyValue,
	ReduceFunc func(string, []string) string,
	nRPC int, parallelism *Parallelism,
)

RunWorker 与 master 建立连接，注册其地址，并等待任务被调度。

Types ¶

type KeyValue ¶

type KeyValue struct {
	Key   string
	Value string
}

KeyValue 是一种用于保存传递给 map 和 reduce 函数的键值对的类型。

type Master ¶

type Master struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Master 拥有 Master 需要跟踪的所有状态。

func Distributed ¶

func Distributed(jobName string, files []string, nreduce int, master string) (mr *Master)

Distributed 分布式调度在通过 RPC 向主服务器注册的工作人员上映射和减少任务。

func Sequential ¶

func Sequential(jobName string, files []string, nreduce int,
	mapF func(string, string) []KeyValue,
	reduceF func(string, []string) string,
) (mr *Master)

Sequential 依次运行 map 和 reduce 任务，等待每个任务完成后再运行下一个任务。

func (*Master) Register ¶

func (mr *Master) Register(args *RegisterArgs, _ *struct{}) error

Register 是一个 RPC 方法，worker 启动后调用它来报告他们已准备好接收任务。

func (*Master) Shutdown ¶

func (mr *Master) Shutdown(_, _ *struct{}) error

Shutdown is an RPC method that shuts down the Master's RPC server.

func (*Master) Wait ¶

func (mr *Master) Wait()

Wait blocks until the currently scheduled work has completed. This happens when all tasks have scheduled and completed, the final output have been computed, and all workers have been shut down.

type Parallelism ¶

type Parallelism struct {
	// contains filtered or unexported fields
}

track whether workers executed in parallel.

type Worker ¶

type Worker struct {
	sync.Mutex

	Map    func(string, string) []KeyValue
	Reduce func(string, []string) string
	// contains filtered or unexported fields
}

Worker holds the state for a server waiting for DoTask or Shutdown RPCs

func (*Worker) DoTask ¶

func (wk *Worker) DoTask(arg *DoTaskArgs, _ *struct{}) error

DoTask is called by the master when a new task is being scheduled on this worker.

func (*Worker) Shutdown ¶

func (wk *Worker) Shutdown(_ *struct{}, res *ShutdownReply) error

Shutdown 当所有工作完成时，master关闭。我们应该用我们已经处理的任务数量来回应。

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL