transform

package
v2.9.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 8, 2024 License: Apache-2.0 Imports: 45 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var File_server_worker_pipeline_transform_transform_proto protoreflect.FileDescriptor

Functions

func NewRegistry added in v2.9.0 

func NewRegistry(driver driver.Driver, logger logs.TaggedLogger) (*registry, error)

TODO: Prometheus stats? (previously in the driver, which included testing we should reuse if possible) capture logs (reuse driver tests and reintroduce tagged logger).

func PreprocessingWorker added in v2.9.0 

func PreprocessingWorker(pachClient *client.APIClient, taskService task.Service, pipelineInfo *pps.PipelineInfo) error

func ProcessingWorker added in v2.9.0 

func ProcessingWorker(ctx context.Context, driver driver.Driver, logger logs.TaggedLogger, status *Status) error

func Run 

func Run(driver driver.Driver, logger logs.TaggedLogger) error

Run will run a transform pipeline until the driver is canceled.

Types

type CreateDatumSetsTask 

type CreateDatumSetsTask struct {
	FileSetId string         `protobuf:"bytes,1,opt,name=file_set_id,json=fileSetId,proto3" json:"file_set_id,omitempty"`
	PathRange *pfs.PathRange `protobuf:"bytes,2,opt,name=path_range,json=pathRange,proto3" json:"path_range,omitempty"`
	SetSpec   *datum.SetSpec `protobuf:"bytes,3,opt,name=set_spec,json=setSpec,proto3" json:"set_spec,omitempty"`
	AuthToken string         `protobuf:"bytes,4,opt,name=auth_token,json=authToken,proto3" json:"auth_token,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateDatumSetsTask) Descriptor deprecated 

func (*CreateDatumSetsTask) Descriptor() ([]byte, []int)

Deprecated: Use CreateDatumSetsTask.ProtoReflect.Descriptor instead.

func (*CreateDatumSetsTask) GetAuthToken added in v2.9.0 

func (x *CreateDatumSetsTask) GetAuthToken() string

func (*CreateDatumSetsTask) GetFileSetId 

func (x *CreateDatumSetsTask) GetFileSetId() string

func (*CreateDatumSetsTask) GetPathRange 

func (x *CreateDatumSetsTask) GetPathRange() *pfs.PathRange

func (*CreateDatumSetsTask) GetSetSpec 

func (x *CreateDatumSetsTask) GetSetSpec() *datum.SetSpec

func (*CreateDatumSetsTask) MarshalLogObject 

func (x *CreateDatumSetsTask) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*CreateDatumSetsTask) ProtoMessage 

func (*CreateDatumSetsTask) ProtoMessage()

func (*CreateDatumSetsTask) ProtoReflect added in v2.7.0 

func (x *CreateDatumSetsTask) ProtoReflect() protoreflect.Message

func (*CreateDatumSetsTask) Reset 

func (x *CreateDatumSetsTask) Reset()

func (*CreateDatumSetsTask) String 

func (x *CreateDatumSetsTask) String() string

func (*CreateDatumSetsTask) Validate added in v2.8.0 

func (m *CreateDatumSetsTask) Validate() error

Validate checks the field values on CreateDatumSetsTask with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*CreateDatumSetsTask) ValidateAll added in v2.8.0 

func (m *CreateDatumSetsTask) ValidateAll() error

ValidateAll checks the field values on CreateDatumSetsTask with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in CreateDatumSetsTaskMultiError, or nil if none found.

type CreateDatumSetsTaskMultiError added in v2.8.0 

type CreateDatumSetsTaskMultiError []error

CreateDatumSetsTaskMultiError is an error wrapping multiple validation errors returned by CreateDatumSetsTask.ValidateAll() if the designated constraints aren't met.

func (CreateDatumSetsTaskMultiError) AllErrors added in v2.8.0 

func (m CreateDatumSetsTaskMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (CreateDatumSetsTaskMultiError) Error added in v2.8.0 

func (m CreateDatumSetsTaskMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type CreateDatumSetsTaskResult 

type CreateDatumSetsTaskResult struct {
	DatumSets []*pfs.PathRange `protobuf:"bytes,1,rep,name=datum_sets,json=datumSets,proto3" json:"datum_sets,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateDatumSetsTaskResult) Descriptor deprecated 

func (*CreateDatumSetsTaskResult) Descriptor() ([]byte, []int)

Deprecated: Use CreateDatumSetsTaskResult.ProtoReflect.Descriptor instead.

func (*CreateDatumSetsTaskResult) GetDatumSets 

func (x *CreateDatumSetsTaskResult) GetDatumSets() []*pfs.PathRange

func (*CreateDatumSetsTaskResult) MarshalLogObject 

func (x *CreateDatumSetsTaskResult) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*CreateDatumSetsTaskResult) ProtoMessage 

func (*CreateDatumSetsTaskResult) ProtoMessage()

func (*CreateDatumSetsTaskResult) ProtoReflect added in v2.7.0 

func (x *CreateDatumSetsTaskResult) ProtoReflect() protoreflect.Message

func (*CreateDatumSetsTaskResult) Reset 

func (x *CreateDatumSetsTaskResult) Reset()

func (*CreateDatumSetsTaskResult) String 

func (x *CreateDatumSetsTaskResult) String() string

func (*CreateDatumSetsTaskResult) Validate added in v2.8.0 

func (m *CreateDatumSetsTaskResult) Validate() error

Validate checks the field values on CreateDatumSetsTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*CreateDatumSetsTaskResult) ValidateAll added in v2.8.0 

func (m *CreateDatumSetsTaskResult) ValidateAll() error

ValidateAll checks the field values on CreateDatumSetsTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in CreateDatumSetsTaskResultMultiError, or nil if none found.

type CreateDatumSetsTaskResultMultiError added in v2.8.0 

type CreateDatumSetsTaskResultMultiError []error

CreateDatumSetsTaskResultMultiError is an error wrapping multiple validation errors returned by CreateDatumSetsTaskResult.ValidateAll() if the designated constraints aren't met.

func (CreateDatumSetsTaskResultMultiError) AllErrors added in v2.8.0 

func (m CreateDatumSetsTaskResultMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (CreateDatumSetsTaskResultMultiError) Error added in v2.8.0 

func (m CreateDatumSetsTaskResultMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type CreateDatumSetsTaskResultValidationError added in v2.8.0 

type CreateDatumSetsTaskResultValidationError struct {
	// contains filtered or unexported fields
}

CreateDatumSetsTaskResultValidationError is the validation error returned by CreateDatumSetsTaskResult.Validate if the designated constraints aren't met.

func (CreateDatumSetsTaskResultValidationError) Cause added in v2.8.0 

func (e CreateDatumSetsTaskResultValidationError) Cause() error

Cause function returns cause value.

func (CreateDatumSetsTaskResultValidationError) Error added in v2.8.0 

func (e CreateDatumSetsTaskResultValidationError) Error() string

Error satisfies the builtin error interface

func (CreateDatumSetsTaskResultValidationError) ErrorName added in v2.8.0 

func (e CreateDatumSetsTaskResultValidationError) ErrorName() string

ErrorName returns error name.

func (CreateDatumSetsTaskResultValidationError) Field added in v2.8.0 

func (e CreateDatumSetsTaskResultValidationError) Field() string

Field function returns field value.

func (CreateDatumSetsTaskResultValidationError) Key added in v2.8.0 

func (e CreateDatumSetsTaskResultValidationError) Key() bool

Key function returns key value.

func (CreateDatumSetsTaskResultValidationError) Reason added in v2.8.0 

func (e CreateDatumSetsTaskResultValidationError) Reason() string

Reason function returns reason value.

type CreateDatumSetsTaskValidationError added in v2.8.0 

type CreateDatumSetsTaskValidationError struct {
	// contains filtered or unexported fields
}

CreateDatumSetsTaskValidationError is the validation error returned by CreateDatumSetsTask.Validate if the designated constraints aren't met.

func (CreateDatumSetsTaskValidationError) Cause added in v2.8.0 

func (e CreateDatumSetsTaskValidationError) Cause() error

Cause function returns cause value.

func (CreateDatumSetsTaskValidationError) Error added in v2.8.0 

func (e CreateDatumSetsTaskValidationError) Error() string

Error satisfies the builtin error interface

func (CreateDatumSetsTaskValidationError) ErrorName added in v2.8.0 

func (e CreateDatumSetsTaskValidationError) ErrorName() string

ErrorName returns error name.

func (CreateDatumSetsTaskValidationError) Field added in v2.8.0 

func (e CreateDatumSetsTaskValidationError) Field() string

Field function returns field value.

func (CreateDatumSetsTaskValidationError) Key added in v2.8.0 

func (e CreateDatumSetsTaskValidationError) Key() bool

Key function returns key value.

func (CreateDatumSetsTaskValidationError) Reason added in v2.8.0 

func (e CreateDatumSetsTaskValidationError) Reason() string

Reason function returns reason value.

type CreateParallelDatumsTask 

type CreateParallelDatumsTask struct {
	Job           *pps.Job       `protobuf:"bytes,1,opt,name=job,proto3" json:"job,omitempty"`
	Salt          string         `protobuf:"bytes,2,opt,name=salt,proto3" json:"salt,omitempty"`
	FileSetId     string         `protobuf:"bytes,3,opt,name=file_set_id,json=fileSetId,proto3" json:"file_set_id,omitempty"`
	BaseFileSetId string         `protobuf:"bytes,4,opt,name=base_file_set_id,json=baseFileSetId,proto3" json:"base_file_set_id,omitempty"`
	PathRange     *pfs.PathRange `protobuf:"bytes,5,opt,name=path_range,json=pathRange,proto3" json:"path_range,omitempty"`
	AuthToken     string         `protobuf:"bytes,6,opt,name=auth_token,json=authToken,proto3" json:"auth_token,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateParallelDatumsTask) Descriptor deprecated 

func (*CreateParallelDatumsTask) Descriptor() ([]byte, []int)

Deprecated: Use CreateParallelDatumsTask.ProtoReflect.Descriptor instead.

func (*CreateParallelDatumsTask) GetAuthToken added in v2.9.0 

func (x *CreateParallelDatumsTask) GetAuthToken() string

func (*CreateParallelDatumsTask) GetBaseFileSetId 

func (x *CreateParallelDatumsTask) GetBaseFileSetId() string

func (*CreateParallelDatumsTask) GetFileSetId 

func (x *CreateParallelDatumsTask) GetFileSetId() string

func (*CreateParallelDatumsTask) GetJob 

func (x *CreateParallelDatumsTask) GetJob() *pps.Job

func (*CreateParallelDatumsTask) GetPathRange 

func (x *CreateParallelDatumsTask) GetPathRange() *pfs.PathRange

func (*CreateParallelDatumsTask) GetSalt 

func (x *CreateParallelDatumsTask) GetSalt() string

func (*CreateParallelDatumsTask) MarshalLogObject 

func (x *CreateParallelDatumsTask) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*CreateParallelDatumsTask) ProtoMessage 

func (*CreateParallelDatumsTask) ProtoMessage()

func (*CreateParallelDatumsTask) ProtoReflect added in v2.7.0 

func (x *CreateParallelDatumsTask) ProtoReflect() protoreflect.Message

func (*CreateParallelDatumsTask) Reset 

func (x *CreateParallelDatumsTask) Reset()

func (*CreateParallelDatumsTask) String 

func (x *CreateParallelDatumsTask) String() string

func (*CreateParallelDatumsTask) Validate added in v2.8.0 

func (m *CreateParallelDatumsTask) Validate() error

Validate checks the field values on CreateParallelDatumsTask with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*CreateParallelDatumsTask) ValidateAll added in v2.8.0 

func (m *CreateParallelDatumsTask) ValidateAll() error

ValidateAll checks the field values on CreateParallelDatumsTask with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in CreateParallelDatumsTaskMultiError, or nil if none found.

type CreateParallelDatumsTaskMultiError added in v2.8.0 

type CreateParallelDatumsTaskMultiError []error

CreateParallelDatumsTaskMultiError is an error wrapping multiple validation errors returned by CreateParallelDatumsTask.ValidateAll() if the designated constraints aren't met.

func (CreateParallelDatumsTaskMultiError) AllErrors added in v2.8.0 

func (m CreateParallelDatumsTaskMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (CreateParallelDatumsTaskMultiError) Error added in v2.8.0 

func (m CreateParallelDatumsTaskMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type CreateParallelDatumsTaskResult 

type CreateParallelDatumsTaskResult struct {
	FileSetId string       `protobuf:"bytes,1,opt,name=file_set_id,json=fileSetId,proto3" json:"file_set_id,omitempty"`
	Stats     *datum.Stats `protobuf:"bytes,2,opt,name=stats,proto3" json:"stats,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateParallelDatumsTaskResult) Descriptor deprecated 

func (*CreateParallelDatumsTaskResult) Descriptor() ([]byte, []int)

Deprecated: Use CreateParallelDatumsTaskResult.ProtoReflect.Descriptor instead.

func (*CreateParallelDatumsTaskResult) GetFileSetId 

func (x *CreateParallelDatumsTaskResult) GetFileSetId() string

func (*CreateParallelDatumsTaskResult) GetStats 

func (x *CreateParallelDatumsTaskResult) GetStats() *datum.Stats

func (*CreateParallelDatumsTaskResult) MarshalLogObject 

func (x *CreateParallelDatumsTaskResult) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*CreateParallelDatumsTaskResult) ProtoMessage 

func (*CreateParallelDatumsTaskResult) ProtoMessage()

func (*CreateParallelDatumsTaskResult) ProtoReflect added in v2.7.0 

func (x *CreateParallelDatumsTaskResult) ProtoReflect() protoreflect.Message

func (*CreateParallelDatumsTaskResult) Reset 

func (x *CreateParallelDatumsTaskResult) Reset()

func (*CreateParallelDatumsTaskResult) String 

func (x *CreateParallelDatumsTaskResult) String() string

func (*CreateParallelDatumsTaskResult) Validate added in v2.8.0 

func (m *CreateParallelDatumsTaskResult) Validate() error

Validate checks the field values on CreateParallelDatumsTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*CreateParallelDatumsTaskResult) ValidateAll added in v2.8.0 

func (m *CreateParallelDatumsTaskResult) ValidateAll() error

ValidateAll checks the field values on CreateParallelDatumsTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in CreateParallelDatumsTaskResultMultiError, or nil if none found.

type CreateParallelDatumsTaskResultMultiError added in v2.8.0 

type CreateParallelDatumsTaskResultMultiError []error

CreateParallelDatumsTaskResultMultiError is an error wrapping multiple validation errors returned by CreateParallelDatumsTaskResult.ValidateAll() if the designated constraints aren't met.

func (CreateParallelDatumsTaskResultMultiError) AllErrors added in v2.8.0 

func (m CreateParallelDatumsTaskResultMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (CreateParallelDatumsTaskResultMultiError) Error added in v2.8.0 

func (m CreateParallelDatumsTaskResultMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type CreateParallelDatumsTaskResultValidationError added in v2.8.0 

type CreateParallelDatumsTaskResultValidationError struct {
	// contains filtered or unexported fields
}

CreateParallelDatumsTaskResultValidationError is the validation error returned by CreateParallelDatumsTaskResult.Validate if the designated constraints aren't met.

func (CreateParallelDatumsTaskResultValidationError) Cause added in v2.8.0 

func (e CreateParallelDatumsTaskResultValidationError) Cause() error

Cause function returns cause value.

func (CreateParallelDatumsTaskResultValidationError) Error added in v2.8.0 

func (e CreateParallelDatumsTaskResultValidationError) Error() string

Error satisfies the builtin error interface

func (CreateParallelDatumsTaskResultValidationError) ErrorName added in v2.8.0 

func (e CreateParallelDatumsTaskResultValidationError) ErrorName() string

ErrorName returns error name.

func (CreateParallelDatumsTaskResultValidationError) Field added in v2.8.0 

func (e CreateParallelDatumsTaskResultValidationError) Field() string

Field function returns field value.

func (CreateParallelDatumsTaskResultValidationError) Key added in v2.8.0 

func (e CreateParallelDatumsTaskResultValidationError) Key() bool

Key function returns key value.

func (CreateParallelDatumsTaskResultValidationError) Reason added in v2.8.0 

func (e CreateParallelDatumsTaskResultValidationError) Reason() string

Reason function returns reason value.

type CreateParallelDatumsTaskValidationError added in v2.8.0 

type CreateParallelDatumsTaskValidationError struct {
	// contains filtered or unexported fields
}

CreateParallelDatumsTaskValidationError is the validation error returned by CreateParallelDatumsTask.Validate if the designated constraints aren't met.

func (CreateParallelDatumsTaskValidationError) Cause added in v2.8.0 

func (e CreateParallelDatumsTaskValidationError) Cause() error

Cause function returns cause value.

func (CreateParallelDatumsTaskValidationError) Error added in v2.8.0 

func (e CreateParallelDatumsTaskValidationError) Error() string

Error satisfies the builtin error interface

func (CreateParallelDatumsTaskValidationError) ErrorName added in v2.8.0 

func (e CreateParallelDatumsTaskValidationError) ErrorName() string

ErrorName returns error name.

func (CreateParallelDatumsTaskValidationError) Field added in v2.8.0 

func (e CreateParallelDatumsTaskValidationError) Field() string

Field function returns field value.

func (CreateParallelDatumsTaskValidationError) Key added in v2.8.0 

func (e CreateParallelDatumsTaskValidationError) Key() bool

Key function returns key value.

func (CreateParallelDatumsTaskValidationError) Reason added in v2.8.0 

func (e CreateParallelDatumsTaskValidationError) Reason() string

Reason function returns reason value.

type CreateSerialDatumsTask 

type CreateSerialDatumsTask struct {
	Job            *pps.Job       `protobuf:"bytes,1,opt,name=job,proto3" json:"job,omitempty"`
	Salt           string         `protobuf:"bytes,2,opt,name=salt,proto3" json:"salt,omitempty"`
	FileSetId      string         `protobuf:"bytes,3,opt,name=file_set_id,json=fileSetId,proto3" json:"file_set_id,omitempty"`
	BaseMetaCommit *pfs.Commit    `protobuf:"bytes,4,opt,name=base_meta_commit,json=baseMetaCommit,proto3" json:"base_meta_commit,omitempty"`
	NoSkip         bool           `protobuf:"varint,5,opt,name=no_skip,json=noSkip,proto3" json:"no_skip,omitempty"`
	PathRange      *pfs.PathRange `protobuf:"bytes,6,opt,name=path_range,json=pathRange,proto3" json:"path_range,omitempty"`
	AuthToken      string         `protobuf:"bytes,7,opt,name=auth_token,json=authToken,proto3" json:"auth_token,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateSerialDatumsTask) Descriptor deprecated 

func (*CreateSerialDatumsTask) Descriptor() ([]byte, []int)

Deprecated: Use CreateSerialDatumsTask.ProtoReflect.Descriptor instead.

func (*CreateSerialDatumsTask) GetAuthToken added in v2.9.0 

func (x *CreateSerialDatumsTask) GetAuthToken() string

func (*CreateSerialDatumsTask) GetBaseMetaCommit 

func (x *CreateSerialDatumsTask) GetBaseMetaCommit() *pfs.Commit

func (*CreateSerialDatumsTask) GetFileSetId 

func (x *CreateSerialDatumsTask) GetFileSetId() string

func (*CreateSerialDatumsTask) GetJob 

func (x *CreateSerialDatumsTask) GetJob() *pps.Job

func (*CreateSerialDatumsTask) GetNoSkip 

func (x *CreateSerialDatumsTask) GetNoSkip() bool

func (*CreateSerialDatumsTask) GetPathRange 

func (x *CreateSerialDatumsTask) GetPathRange() *pfs.PathRange

func (*CreateSerialDatumsTask) GetSalt 

func (x *CreateSerialDatumsTask) GetSalt() string

func (*CreateSerialDatumsTask) MarshalLogObject 

func (x *CreateSerialDatumsTask) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*CreateSerialDatumsTask) ProtoMessage 

func (*CreateSerialDatumsTask) ProtoMessage()

func (*CreateSerialDatumsTask) ProtoReflect added in v2.7.0 

func (x *CreateSerialDatumsTask) ProtoReflect() protoreflect.Message

func (*CreateSerialDatumsTask) Reset 

func (x *CreateSerialDatumsTask) Reset()

func (*CreateSerialDatumsTask) String 

func (x *CreateSerialDatumsTask) String() string

func (*CreateSerialDatumsTask) Validate added in v2.8.0 

func (m *CreateSerialDatumsTask) Validate() error

Validate checks the field values on CreateSerialDatumsTask with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*CreateSerialDatumsTask) ValidateAll added in v2.8.0 

func (m *CreateSerialDatumsTask) ValidateAll() error

ValidateAll checks the field values on CreateSerialDatumsTask with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in CreateSerialDatumsTaskMultiError, or nil if none found.

type CreateSerialDatumsTaskMultiError added in v2.8.0 

type CreateSerialDatumsTaskMultiError []error

CreateSerialDatumsTaskMultiError is an error wrapping multiple validation errors returned by CreateSerialDatumsTask.ValidateAll() if the designated constraints aren't met.

func (CreateSerialDatumsTaskMultiError) AllErrors added in v2.8.0 

func (m CreateSerialDatumsTaskMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (CreateSerialDatumsTaskMultiError) Error added in v2.8.0 

func (m CreateSerialDatumsTaskMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type CreateSerialDatumsTaskResult 

type CreateSerialDatumsTaskResult struct {
	FileSetId             string       `protobuf:"bytes,1,opt,name=file_set_id,json=fileSetId,proto3" json:"file_set_id,omitempty"`
	OutputDeleteFileSetId string       `` /* 130-byte string literal not displayed */
	MetaDeleteFileSetId   string       `protobuf:"bytes,3,opt,name=meta_delete_file_set_id,json=metaDeleteFileSetId,proto3" json:"meta_delete_file_set_id,omitempty"`
	Stats                 *datum.Stats `protobuf:"bytes,4,opt,name=stats,proto3" json:"stats,omitempty"`
	// contains filtered or unexported fields
}

func (*CreateSerialDatumsTaskResult) Descriptor deprecated 

func (*CreateSerialDatumsTaskResult) Descriptor() ([]byte, []int)

Deprecated: Use CreateSerialDatumsTaskResult.ProtoReflect.Descriptor instead.

func (*CreateSerialDatumsTaskResult) GetFileSetId 

func (x *CreateSerialDatumsTaskResult) GetFileSetId() string

func (*CreateSerialDatumsTaskResult) GetMetaDeleteFileSetId 

func (x *CreateSerialDatumsTaskResult) GetMetaDeleteFileSetId() string

func (*CreateSerialDatumsTaskResult) GetOutputDeleteFileSetId 

func (x *CreateSerialDatumsTaskResult) GetOutputDeleteFileSetId() string

func (*CreateSerialDatumsTaskResult) GetStats 

func (x *CreateSerialDatumsTaskResult) GetStats() *datum.Stats

func (*CreateSerialDatumsTaskResult) MarshalLogObject 

func (x *CreateSerialDatumsTaskResult) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*CreateSerialDatumsTaskResult) ProtoMessage 

func (*CreateSerialDatumsTaskResult) ProtoMessage()

func (*CreateSerialDatumsTaskResult) ProtoReflect added in v2.7.0 

func (x *CreateSerialDatumsTaskResult) ProtoReflect() protoreflect.Message

func (*CreateSerialDatumsTaskResult) Reset 

func (x *CreateSerialDatumsTaskResult) Reset()

func (*CreateSerialDatumsTaskResult) String 

func (x *CreateSerialDatumsTaskResult) String() string

func (*CreateSerialDatumsTaskResult) Validate added in v2.8.0 

func (m *CreateSerialDatumsTaskResult) Validate() error

Validate checks the field values on CreateSerialDatumsTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*CreateSerialDatumsTaskResult) ValidateAll added in v2.8.0 

func (m *CreateSerialDatumsTaskResult) ValidateAll() error

ValidateAll checks the field values on CreateSerialDatumsTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in CreateSerialDatumsTaskResultMultiError, or nil if none found.

type CreateSerialDatumsTaskResultMultiError added in v2.8.0 

type CreateSerialDatumsTaskResultMultiError []error

CreateSerialDatumsTaskResultMultiError is an error wrapping multiple validation errors returned by CreateSerialDatumsTaskResult.ValidateAll() if the designated constraints aren't met.

func (CreateSerialDatumsTaskResultMultiError) AllErrors added in v2.8.0 

func (m CreateSerialDatumsTaskResultMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (CreateSerialDatumsTaskResultMultiError) Error added in v2.8.0 

func (m CreateSerialDatumsTaskResultMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type CreateSerialDatumsTaskResultValidationError added in v2.8.0 

type CreateSerialDatumsTaskResultValidationError struct {
	// contains filtered or unexported fields
}

CreateSerialDatumsTaskResultValidationError is the validation error returned by CreateSerialDatumsTaskResult.Validate if the designated constraints aren't met.

func (CreateSerialDatumsTaskResultValidationError) Cause added in v2.8.0 

func (e CreateSerialDatumsTaskResultValidationError) Cause() error

Cause function returns cause value.

func (CreateSerialDatumsTaskResultValidationError) Error added in v2.8.0 

func (e CreateSerialDatumsTaskResultValidationError) Error() string

Error satisfies the builtin error interface

func (CreateSerialDatumsTaskResultValidationError) ErrorName added in v2.8.0 

func (e CreateSerialDatumsTaskResultValidationError) ErrorName() string

ErrorName returns error name.

func (CreateSerialDatumsTaskResultValidationError) Field added in v2.8.0 

func (e CreateSerialDatumsTaskResultValidationError) Field() string

Field function returns field value.

func (CreateSerialDatumsTaskResultValidationError) Key added in v2.8.0 

func (e CreateSerialDatumsTaskResultValidationError) Key() bool

Key function returns key value.

func (CreateSerialDatumsTaskResultValidationError) Reason added in v2.8.0 

func (e CreateSerialDatumsTaskResultValidationError) Reason() string

Reason function returns reason value.

type CreateSerialDatumsTaskValidationError added in v2.8.0 

type CreateSerialDatumsTaskValidationError struct {
	// contains filtered or unexported fields
}

CreateSerialDatumsTaskValidationError is the validation error returned by CreateSerialDatumsTask.Validate if the designated constraints aren't met.

func (CreateSerialDatumsTaskValidationError) Cause added in v2.8.0 

func (e CreateSerialDatumsTaskValidationError) Cause() error

Cause function returns cause value.

func (CreateSerialDatumsTaskValidationError) Error added in v2.8.0 

func (e CreateSerialDatumsTaskValidationError) Error() string

Error satisfies the builtin error interface

func (CreateSerialDatumsTaskValidationError) ErrorName added in v2.8.0 

func (e CreateSerialDatumsTaskValidationError) ErrorName() string

ErrorName returns error name.

func (CreateSerialDatumsTaskValidationError) Field added in v2.8.0 

func (e CreateSerialDatumsTaskValidationError) Field() string

Field function returns field value.

func (CreateSerialDatumsTaskValidationError) Key added in v2.8.0 

func (e CreateSerialDatumsTaskValidationError) Key() bool

Key function returns key value.

func (CreateSerialDatumsTaskValidationError) Reason added in v2.8.0 

func (e CreateSerialDatumsTaskValidationError) Reason() string

Reason function returns reason value.

type DatumSetTask 

type DatumSetTask struct {
	Job          *pps.Job       `protobuf:"bytes,1,opt,name=job,proto3" json:"job,omitempty"`
	FileSetId    string         `protobuf:"bytes,2,opt,name=file_set_id,json=fileSetId,proto3" json:"file_set_id,omitempty"`
	PathRange    *pfs.PathRange `protobuf:"bytes,3,opt,name=path_range,json=pathRange,proto3" json:"path_range,omitempty"`
	OutputCommit *pfs.Commit    `protobuf:"bytes,4,opt,name=output_commit,json=outputCommit,proto3" json:"output_commit,omitempty"`
	// contains filtered or unexported fields
}

func (*DatumSetTask) Descriptor deprecated 

func (*DatumSetTask) Descriptor() ([]byte, []int)

Deprecated: Use DatumSetTask.ProtoReflect.Descriptor instead.

func (*DatumSetTask) GetFileSetId 

func (x *DatumSetTask) GetFileSetId() string

func (*DatumSetTask) GetJob 

func (x *DatumSetTask) GetJob() *pps.Job

func (*DatumSetTask) GetOutputCommit 

func (x *DatumSetTask) GetOutputCommit() *pfs.Commit

func (*DatumSetTask) GetPathRange 

func (x *DatumSetTask) GetPathRange() *pfs.PathRange

func (*DatumSetTask) MarshalLogObject 

func (x *DatumSetTask) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*DatumSetTask) ProtoMessage 

func (*DatumSetTask) ProtoMessage()

func (*DatumSetTask) ProtoReflect added in v2.7.0 

func (x *DatumSetTask) ProtoReflect() protoreflect.Message

func (*DatumSetTask) Reset 

func (x *DatumSetTask) Reset()

func (*DatumSetTask) String 

func (x *DatumSetTask) String() string

func (*DatumSetTask) Validate added in v2.8.0 

func (m *DatumSetTask) Validate() error

Validate checks the field values on DatumSetTask with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*DatumSetTask) ValidateAll added in v2.8.0 

func (m *DatumSetTask) ValidateAll() error

ValidateAll checks the field values on DatumSetTask with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in DatumSetTaskMultiError, or nil if none found.

type DatumSetTaskMultiError added in v2.8.0 

type DatumSetTaskMultiError []error

DatumSetTaskMultiError is an error wrapping multiple validation errors returned by DatumSetTask.ValidateAll() if the designated constraints aren't met.

func (DatumSetTaskMultiError) AllErrors added in v2.8.0 

func (m DatumSetTaskMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (DatumSetTaskMultiError) Error added in v2.8.0 

func (m DatumSetTaskMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type DatumSetTaskResult 

type DatumSetTaskResult struct {
	OutputFileSetId string       `protobuf:"bytes,1,opt,name=output_file_set_id,json=outputFileSetId,proto3" json:"output_file_set_id,omitempty"`
	MetaFileSetId   string       `protobuf:"bytes,2,opt,name=meta_file_set_id,json=metaFileSetId,proto3" json:"meta_file_set_id,omitempty"`
	Stats           *datum.Stats `protobuf:"bytes,3,opt,name=stats,proto3" json:"stats,omitempty"`
	// contains filtered or unexported fields
}

func (*DatumSetTaskResult) Descriptor deprecated 

func (*DatumSetTaskResult) Descriptor() ([]byte, []int)

Deprecated: Use DatumSetTaskResult.ProtoReflect.Descriptor instead.

func (*DatumSetTaskResult) GetMetaFileSetId 

func (x *DatumSetTaskResult) GetMetaFileSetId() string

func (*DatumSetTaskResult) GetOutputFileSetId 

func (x *DatumSetTaskResult) GetOutputFileSetId() string

func (*DatumSetTaskResult) GetStats 

func (x *DatumSetTaskResult) GetStats() *datum.Stats

func (*DatumSetTaskResult) MarshalLogObject 

func (x *DatumSetTaskResult) MarshalLogObject(enc zapcore.ObjectEncoder) error

func (*DatumSetTaskResult) ProtoMessage 

func (*DatumSetTaskResult) ProtoMessage()

func (*DatumSetTaskResult) ProtoReflect added in v2.7.0 

func (x *DatumSetTaskResult) ProtoReflect() protoreflect.Message

func (*DatumSetTaskResult) Reset 

func (x *DatumSetTaskResult) Reset()

func (*DatumSetTaskResult) String 

func (x *DatumSetTaskResult) String() string

func (*DatumSetTaskResult) Validate added in v2.8.0 

func (m *DatumSetTaskResult) Validate() error

Validate checks the field values on DatumSetTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the first error encountered is returned, or nil if there are no violations.

func (*DatumSetTaskResult) ValidateAll added in v2.8.0 

func (m *DatumSetTaskResult) ValidateAll() error

ValidateAll checks the field values on DatumSetTaskResult with the rules defined in the proto definition for this message. If any rules are violated, the result is a list of violation errors wrapped in DatumSetTaskResultMultiError, or nil if none found.

type DatumSetTaskResultMultiError added in v2.8.0 

type DatumSetTaskResultMultiError []error

DatumSetTaskResultMultiError is an error wrapping multiple validation errors returned by DatumSetTaskResult.ValidateAll() if the designated constraints aren't met.

func (DatumSetTaskResultMultiError) AllErrors added in v2.8.0 

func (m DatumSetTaskResultMultiError) AllErrors() []error

AllErrors returns a list of validation violation errors.

func (DatumSetTaskResultMultiError) Error added in v2.8.0 

func (m DatumSetTaskResultMultiError) Error() string

Error returns a concatenation of all the error messages it wraps.

type DatumSetTaskResultValidationError added in v2.8.0 

type DatumSetTaskResultValidationError struct {
	// contains filtered or unexported fields
}

DatumSetTaskResultValidationError is the validation error returned by DatumSetTaskResult.Validate if the designated constraints aren't met.

func (DatumSetTaskResultValidationError) Cause added in v2.8.0 

func (e DatumSetTaskResultValidationError) Cause() error

Cause function returns cause value.

func (DatumSetTaskResultValidationError) Error added in v2.8.0 

func (e DatumSetTaskResultValidationError) Error() string

Error satisfies the builtin error interface

func (DatumSetTaskResultValidationError) ErrorName added in v2.8.0 

func (e DatumSetTaskResultValidationError) ErrorName() string

ErrorName returns error name.

func (DatumSetTaskResultValidationError) Field added in v2.8.0 

func (e DatumSetTaskResultValidationError) Field() string

Field function returns field value.

func (DatumSetTaskResultValidationError) Key added in v2.8.0 

func (e DatumSetTaskResultValidationError) Key() bool

Key function returns key value.

func (DatumSetTaskResultValidationError) Reason added in v2.8.0 

func (e DatumSetTaskResultValidationError) Reason() string

Reason function returns reason value.

type DatumSetTaskValidationError added in v2.8.0 

type DatumSetTaskValidationError struct {
	// contains filtered or unexported fields
}

DatumSetTaskValidationError is the validation error returned by DatumSetTask.Validate if the designated constraints aren't met.

func (DatumSetTaskValidationError) Cause added in v2.8.0 

func (e DatumSetTaskValidationError) Cause() error

Cause function returns cause value.

func (DatumSetTaskValidationError) Error added in v2.8.0 

func (e DatumSetTaskValidationError) Error() string

Error satisfies the builtin error interface

func (DatumSetTaskValidationError) ErrorName added in v2.8.0 

func (e DatumSetTaskValidationError) ErrorName() string

ErrorName returns error name.

func (DatumSetTaskValidationError) Field added in v2.8.0 

func (e DatumSetTaskValidationError) Field() string

Field function returns field value.

func (DatumSetTaskValidationError) Key added in v2.8.0 

func (e DatumSetTaskValidationError) Key() bool

Key function returns key value.

func (DatumSetTaskValidationError) Reason added in v2.8.0 

func (e DatumSetTaskValidationError) Reason() string

Reason function returns reason value.

type Status 

type Status struct {
	// contains filtered or unexported fields
}

Status is a struct representing the current status of the transform worker, its public interface only allows getting the status of a task and canceling the currently-processing datum.

func (*Status) Cancel 

func (s *Status) Cancel(jobID string, datumFilter []string) bool

Cancel cancels the currently running datum if it matches the specified job and inputs

func (*Status) GetStatus 

func (s *Status) GetStatus() (*pps.WorkerStatus, error)

GetStatus returns the current WorkerStatus for the transform worker

func (*Status) NextDatum 

func (s *Status) NextDatum(ctx context.Context, err error) ([]string, error)

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.