luci: go.chromium.org/luci/appengine/mapper/internal/tasks Index | Files

package tasks

import "go.chromium.org/luci/appengine/mapper/internal/tasks"

Package tasks contains definition of task queue tasks used by the mapper.

Index

Package Files

gen.go tasks.pb.go

Variables

var File_go_chromium_org_luci_appengine_mapper_internal_tasks_tasks_proto protoreflect.FileDescriptor

type FanOutShards Uses

type FanOutShards struct {
    JobId int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
    // contains filtered or unexported fields
}

FanOutShards enqueues a bunch of ProcessShard named tasks (one per shard).

Enqueued transactionally by SplitAndLaunch after it has constructed shards.

func (*FanOutShards) Descriptor Uses

func (*FanOutShards) Descriptor() ([]byte, []int)

Deprecated: Use FanOutShards.ProtoReflect.Descriptor instead.

func (*FanOutShards) GetJobId Uses

func (x *FanOutShards) GetJobId() int64

func (*FanOutShards) ProtoMessage Uses

func (*FanOutShards) ProtoMessage()

func (*FanOutShards) ProtoReflect Uses

func (x *FanOutShards) ProtoReflect() protoreflect.Message

func (*FanOutShards) Reset Uses

func (x *FanOutShards) Reset()

func (*FanOutShards) String Uses

func (x *FanOutShards) String() string

type ProcessShard Uses

type ProcessShard struct {
    JobId   int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
    ShardId int64 `protobuf:"varint,2,opt,name=shard_id,json=shardId,proto3" json:"shard_id,omitempty"`
    TaskNum int64 `protobuf:"varint,3,opt,name=task_num,json=taskNum,proto3" json:"task_num,omitempty"`
    // contains filtered or unexported fields
}

ProcessShard sequentially reads the entities belonging to a key range assigned to a shard and applies the mapper to them.

Upon reaching 1 min mark, relaunches itself, increasing task_num. Thus ProcessShard is actually a chain of tasks that runs as long as needed to completely process the shard.

func (*ProcessShard) Descriptor Uses

func (*ProcessShard) Descriptor() ([]byte, []int)

Deprecated: Use ProcessShard.ProtoReflect.Descriptor instead.

func (*ProcessShard) GetJobId Uses

func (x *ProcessShard) GetJobId() int64

func (*ProcessShard) GetShardId Uses

func (x *ProcessShard) GetShardId() int64

func (*ProcessShard) GetTaskNum Uses

func (x *ProcessShard) GetTaskNum() int64

func (*ProcessShard) ProtoMessage Uses

func (*ProcessShard) ProtoMessage()

func (*ProcessShard) ProtoReflect Uses

func (x *ProcessShard) ProtoReflect() protoreflect.Message

func (*ProcessShard) Reset Uses

func (x *ProcessShard) Reset()

func (*ProcessShard) String Uses

func (x *ProcessShard) String() string

type RequestJobStateUpdate Uses

type RequestJobStateUpdate struct {
    JobId   int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
    ShardId int64 `protobuf:"varint,2,opt,name=shard_id,json=shardId,proto3" json:"shard_id,omitempty"` // mostly FYI
    // contains filtered or unexported fields
}

RequestJobStateUpdate is transactionally emitted by ProcessShard when shard's state changes.

It eventually (with some throttling) causes UpdateJobState to be emitted, which updates the job state based on states of the shards.

func (*RequestJobStateUpdate) Descriptor Uses

func (*RequestJobStateUpdate) Descriptor() ([]byte, []int)

Deprecated: Use RequestJobStateUpdate.ProtoReflect.Descriptor instead.

func (*RequestJobStateUpdate) GetJobId Uses

func (x *RequestJobStateUpdate) GetJobId() int64

func (*RequestJobStateUpdate) GetShardId Uses

func (x *RequestJobStateUpdate) GetShardId() int64

func (*RequestJobStateUpdate) ProtoMessage Uses

func (*RequestJobStateUpdate) ProtoMessage()

func (*RequestJobStateUpdate) ProtoReflect Uses

func (x *RequestJobStateUpdate) ProtoReflect() protoreflect.Message

func (*RequestJobStateUpdate) Reset Uses

func (x *RequestJobStateUpdate) Reset()

func (*RequestJobStateUpdate) String Uses

func (x *RequestJobStateUpdate) String() string

type SplitAndLaunch Uses

type SplitAndLaunch struct {
    JobId int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
    // contains filtered or unexported fields
}

SplitAndLaunch task splits the key range into shards and kicks off processing of each individual shard.

Enqueued transactionally when creating a new mapping job.

func (*SplitAndLaunch) Descriptor Uses

func (*SplitAndLaunch) Descriptor() ([]byte, []int)

Deprecated: Use SplitAndLaunch.ProtoReflect.Descriptor instead.

func (*SplitAndLaunch) GetJobId Uses

func (x *SplitAndLaunch) GetJobId() int64

func (*SplitAndLaunch) ProtoMessage Uses

func (*SplitAndLaunch) ProtoMessage()

func (*SplitAndLaunch) ProtoReflect Uses

func (x *SplitAndLaunch) ProtoReflect() protoreflect.Message

func (*SplitAndLaunch) Reset Uses

func (x *SplitAndLaunch) Reset()

func (*SplitAndLaunch) String Uses

func (x *SplitAndLaunch) String() string

type UpdateJobState Uses

type UpdateJobState struct {
    JobId int64 `protobuf:"varint,1,opt,name=job_id,json=jobId,proto3" json:"job_id,omitempty"`
    // contains filtered or unexported fields
}

UpdateJobState is emitted after one or more shards have changed their state.

It recalculates the job's state based on state of all its shards. Throttled to 0.5 QPS.

func (*UpdateJobState) Descriptor Uses

func (*UpdateJobState) Descriptor() ([]byte, []int)

Deprecated: Use UpdateJobState.ProtoReflect.Descriptor instead.

func (*UpdateJobState) GetJobId Uses

func (x *UpdateJobState) GetJobId() int64

func (*UpdateJobState) ProtoMessage Uses

func (*UpdateJobState) ProtoMessage()

func (*UpdateJobState) ProtoReflect Uses

func (x *UpdateJobState) ProtoReflect() protoreflect.Message

func (*UpdateJobState) Reset Uses

func (x *UpdateJobState) Reset()

func (*UpdateJobState) String Uses

func (x *UpdateJobState) String() string

Package tasks imports 5 packages (graph) and is imported by 2 packages. Updated 2020-09-22. Refresh now. Tools for package owners.