Documentation
¶
Index ¶
- Constants
- type Chunk
- type FlowHandle
- type ImportScheduler
- func (s *ImportScheduler) CleanupSubtaskExecEnv(ctx context.Context) (err error)
- func (s *ImportScheduler) InitSubtaskExecEnv(ctx context.Context) error
- func (s *ImportScheduler) OnSubtaskFinished(context.Context, []byte) error
- func (s *ImportScheduler) Rollback(context.Context) error
- func (s *ImportScheduler) SplitSubtask(ctx context.Context, bs []byte) ([]proto.MinimalTask, error)
- type ImportSubtaskExecutor
- type MinimalTaskMeta
- type SubtaskMeta
- type TaskMeta
Constants ¶
const (
Import int64 = 1
)
TaskStep of LoadData.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Chunk ¶
type Chunk struct {
Path string
Offset int64
EndOffset int64
PrevRowIDMax int64
RowIDMax int64
Type mydump.SourceType
Compression mydump.Compression
Timestamp int64
}
Chunk records the chunk information.
type FlowHandle ¶
type FlowHandle struct{}
FlowHandle is the dispatcher for load data.
func (*FlowHandle) ProcessErrFlow ¶
func (*FlowHandle) ProcessErrFlow(_ context.Context, _ dispatcher.TaskHandle, gTask *proto.Task, receiveErr [][]byte) ([]byte, error)
ProcessErrFlow implements dispatcher.ProcessErrFlow interface.
func (*FlowHandle) ProcessNormalFlow ¶
func (*FlowHandle) ProcessNormalFlow(ctx context.Context, _ dispatcher.TaskHandle, gTask *proto.Task) ([][]byte, error)
ProcessNormalFlow implements dispatcher.TaskFlowHandle interface.
type ImportScheduler ¶
type ImportScheduler struct {
// contains filtered or unexported fields
}
ImportScheduler is a scheduler for load data.
func (*ImportScheduler) CleanupSubtaskExecEnv ¶
func (s *ImportScheduler) CleanupSubtaskExecEnv(ctx context.Context) (err error)
CleanupSubtaskExecEnv implements the Scheduler.CleanupSubtaskExecEnv interface.
func (*ImportScheduler) InitSubtaskExecEnv ¶
func (s *ImportScheduler) InitSubtaskExecEnv(ctx context.Context) error
InitSubtaskExecEnv implements the Scheduler.InitSubtaskExecEnv interface.
func (*ImportScheduler) OnSubtaskFinished ¶
func (s *ImportScheduler) OnSubtaskFinished(context.Context, []byte) error
OnSubtaskFinished implements the Scheduler.OnSubtaskFinished interface.
func (*ImportScheduler) Rollback ¶
func (s *ImportScheduler) Rollback(context.Context) error
Rollback implements the Scheduler.Rollback interface. TODO: add rollback
func (*ImportScheduler) SplitSubtask ¶
func (s *ImportScheduler) SplitSubtask(ctx context.Context, bs []byte) ([]proto.MinimalTask, error)
SplitSubtask implements the Scheduler.SplitSubtask interface.
type ImportSubtaskExecutor ¶
type ImportSubtaskExecutor struct {
// contains filtered or unexported fields
}
ImportSubtaskExecutor is a subtask executor for load data.
type MinimalTaskMeta ¶
type MinimalTaskMeta struct {
Plan importer.Plan
Chunk Chunk
TableImporter *importer.TableImporter
DataEngine *backend.OpenedEngine
IndexEngine *backend.OpenedEngine
}
MinimalTaskMeta is the minimal task of LoadData. Scheduler will split the subtask into minimal tasks(Chunks -> Chunk)
func (MinimalTaskMeta) IsMinimalTask ¶
func (MinimalTaskMeta) IsMinimalTask()
IsMinimalTask implements the MinimalTask interface.
type SubtaskMeta ¶
SubtaskMeta is the subtask of LoadData. Dispatcher will split the task into subtasks(FileInfos -> Chunks) All the field should be serializable.
Source Files