Documentation
¶
Overview ¶
Package pipeline provides functions to build synchronisation pipeline.
Index ¶
- Variables
- func IsContextCancelErr(err error) bool
- type Group
- func (group *Group) AddPipeStep(step Step)
- func (group *Group) ErrChan() <-chan error
- func (group *Group) GetStepInfo(stepNum int) StepInfo
- func (group *Group) GetStepsInfo() []StepInfo
- func (group *Group) Run()
- func (group *Group) SetSource(st storage.Storage)
- func (group *Group) SetTarget(st storage.Storage)
- type ObjectError
- type PipelineError
- type Step
- type StepConfigurationError
- type StepFn
- type StepInfo
- type StepStats
Constants ¶
This section is empty.
Variables ¶
var Log = logrus.New()
Log implement Logrus logger for debug logging.
Functions ¶
func IsContextCancelErr ¶
IsContextCancelErr check that input error is caused by a context cancellation.
Types ¶
type Group ¶
type Group struct {
Source storage.Storage
Target storage.Storage
StartTime time.Time
// contains filtered or unexported fields
}
Group store a Source and Target storage's and pipeline configuration.
func NewGroup ¶
func NewGroup() Group
NewGroup return a new prepared Group. You should always create new Group{} with this constructor.
func (*Group) AddPipeStep ¶
AddPipeStep add pipeline step to group. Steps will executed sequentially, in order of addition.
func (*Group) ErrChan ¶
ErrChan return a Group error chan. All pipeline errors will be sent errors to this channel.
"nil" message mean that all pipeline functions completed and pipeline was terminated. To prevent leakage of resources in the event of a context cancellation, you should read all messages from this channel. ErrChan will be closed after receiving a "nil" message.
func (*Group) GetStepInfo ¶
GetStepInfo return info about step with given sequential number.
func (*Group) GetStepsInfo ¶
GetStepsInfo return info about all pipeline steps.
func (*Group) Run ¶
func (group *Group) Run()
Run start the pipeline execution.
For result and error handling see ErrChan() function.
type ObjectError ¶
ObjectError contain a pointer to an Object that failed with error
func (*ObjectError) Error ¶
func (e *ObjectError) Error() string
func (*ObjectError) Unwrap ¶
func (e *ObjectError) Unwrap() error
Unwrap ObjectError error and return the underlying error.
type PipelineError ¶
PipelineError implement wrapper for pipeline errors.
func (*PipelineError) Error ¶
func (e *PipelineError) Error() string
func (*PipelineError) Unwrap ¶
func (e *PipelineError) Unwrap() error
Unwrap PipelineError error and return the underlying error.
type Step ¶
type Step struct {
Name string
Fn StepFn
AddWorkers uint
Config interface{}
ChanSize uint
// contains filtered or unexported fields
}
Step contain configuration of pipeline step and it's internal structure. Be careful with Config interface! Check of its type should implemented in StepFn. If typing fails, you get a StepConfigurationError in runtime.
type StepConfigurationError ¶
StepConfigurationError raises when step have interface typing error.
func (*StepConfigurationError) Error ¶
func (e *StepConfigurationError) Error() string
func (*StepConfigurationError) Unwrap ¶
func (e *StepConfigurationError) Unwrap() error
Unwrap StepConfigurationError error and return the underlying error.
type StepFn ¶
type StepFn func(group *Group, stepNum int, input <-chan *storage.Object, output chan<- *storage.Object, errChan chan<- error)
StepFn implement the type of pipeline Step function.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package collection contains different StepFn functions to do different pipeline actions.
|
Package collection contains different StepFn functions to do different pipeline actions. |