cvmfs

package
v0.0.0-...-dfb7885 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 9, 2019 License: BSD-3-Clause Imports: 30 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// ClientProfile is used by the submit and check commands
	ClientProfile = iota
	// ServerProfile is used by the server command
	ServerProfile
	// WorkerProfile is used by the worker command
	WorkerProfile
)
View Source 
const (
	// SchemaVersion is the latest schema version of the job database
	SchemaVersion = 1
)

Variables

View Source 
var Log zerolog.Logger

Log is the application-wide logger

Functions

func ConfigLogging 

func ConfigLogging(cfg *Config)

ConfigLogging updates the logging settings with values from a Config object (Meant to be called after ReadConfig)

func InitLogging 

func InitLogging(sink io.Writer)

InitLogging initializes the logger

func StartServer 

func StartServer(cfg *Config) error

StartServer starts the conveyor server component. This function will block until the server finishes.

Types

type BackendConfig 

type BackendConfig struct {
	Type     string
	Database string
	Username string
	Password string
	Host     string
	Port     int
}

BackendConfig - database backend configuration for the conveyor job server DB backend

type BasicReply 

type BasicReply struct {
	Status string // "ok" || "error"
	Reason string `json:",omitempty"`
}

BasicReply is a status message and optional error cause

type Config 

type Config struct {
	SharedKey      string `mapstructure:"shared_key"`
	JobWaitTimeout int    `mapstructure:"job_wait_timeout"`
	Debug          bool
	LogTimestamps  bool `mapstructure:"log_timestamps"`
	Server         ServerConfig
	Queue          QueueConfig
	Backend        BackendConfig
	Worker         WorkerConfig
}

Config - main configuration object

func ReadConfig 

func ReadConfig(cmd *cobra.Command, profile int) (*Config, error)

ReadConfig - populate the config object using the global viper object and the config file, based on profile (client, worker, or server). The different sections may not needed in all profiles

func (*Config) HTTPEndpoints 

func (c *Config) HTTPEndpoints() HTTPEndpoints

HTTPEndpoints constructs an HTTPEndpoints object

type GetJobStatusReply 

type GetJobStatusReply struct {
	BasicReply
	IDs  []JobStatus    `json:",omitempty"`
	Jobs []ProcessedJob `json:",omitempty"`
}

GetJobStatusReply is the return type of the GetJob query

type HTTPEndpoints 

type HTTPEndpoints struct {
	// contains filtered or unexported fields
}

HTTPEndpoints holds the different HTTP end points of the conveyor job server

func (HTTPEndpoints) CompletedJobs 

func (o HTTPEndpoints) CompletedJobs(withBase bool) string

CompletedJobs returns the endpoint for completed jobs. If "withBase" is true, the base URL is prepended

func (HTTPEndpoints) NewJobs 

func (o HTTPEndpoints) NewJobs(withBase bool) string

NewJobs returns the endpoint for new jobs. If "withBase" is true, the base URL is prepended

type JobClient 

type JobClient struct {
	// contains filtered or unexported fields
}

JobClient offers functionality for interacting with the job server

func NewJobClient 

func NewJobClient(cfg *Config) (*JobClient, error)

NewJobClient constructs a new JobClient object using a configuration object and a set of keys

func (*JobClient) Close 

func (c *JobClient) Close()

Close all the internal connections of the object

func (*JobClient) GetJobStatus 

func (c *JobClient) GetJobStatus(
	ids []string, full bool, quit <-chan struct{}) (*GetJobStatusReply, error)

GetJobStatus queries the status of a set of jobs from the server

func (*JobClient) PostJobStatus 

func (c *JobClient) PostJobStatus(job *ProcessedJob) (*PostJobStatusReply, error)

PostJobStatus posts the status of a completed job to the server

func (*JobClient) PostNewJob 

func (c *JobClient) PostNewJob(job *JobSpecification) (*PostNewJobReply, error)

PostNewJob posts a new unprocessed job to the server

func (*JobClient) SubscribeNewJobs 

func (c *JobClient) SubscribeNewJobs(keyID string) (<-chan amqp.Delivery, error)

SubscribeNewJobs returns a channel with new job messages coming from the conveyor server

func (*JobClient) WaitForJobs 

func (c *JobClient) WaitForJobs(
	ids []string, timeout int) ([]JobStatus, error)

WaitForJobs waits for the completion of a set of jobs referenced through theirs unique ids. The job status is obtained from the completed job notification channel of the job queue and from the job server

type JobSpecification 

type JobSpecification struct {
	JobName      string
	Repository   string
	Payload      string
	LeasePath    string
	Dependencies []string
}

JobSpecification contains all the parameters of a new job which is to be submitted

func (*JobSpecification) Prepare 

func (spec *JobSpecification) Prepare()

Prepare a job specification for submission: normalizes the lease path and embeds the transaction script in the job description, if the script is a local file

type JobStatus 

type JobStatus struct {
	ID         uuid.UUID
	Successful bool
}

JobStatus holds a job ID and its completion status

type PostJobStatusReply 

type PostJobStatusReply struct {
	BasicReply
}

PostJobStatusReply is the return value of the PutJobStatus action

type PostNewJobReply 

type PostNewJobReply struct {
	BasicReply
	ID uuid.UUID
}

PostNewJobReply is the return type of the PostNewJob action

type ProcessedJob 

type ProcessedJob struct {
	UnprocessedJob
	WorkerName   string
	StartTime    time.Time
	FinishTime   time.Time
	Successful   bool
	ErrorMessage string
}

ProcessedJob describes a completed job. Additional fields with respect to an unprocessed job are related to the execution time of the job and its completion status

type QueueClient 

type QueueClient struct {
	Conn              *amqp.Connection
	Chan              *amqp.Channel
	NewJobQueue       *amqp.Queue
	CompletedJobQueue *amqp.Queue
}

QueueClient containts all the objects associated with a connection to a RabbitMQ instance

func NewQueueClient 

func NewQueueClient(cfg *QueueConfig, connType int) (*QueueClient, error)

NewQueueClient creates a new connection to the queue. connType can either be consumerConnection or publisherConnection

func (*QueueClient) Close 

func (c *QueueClient) Close() error

Close the connection to the queue

type QueueConfig 

type QueueConfig struct {
	Username             string
	Password             string
	Host                 string
	VHost                string
	Port                 int
	NewJobExchange       string `mapstructure:"new_job_exchange"`
	NewJobQueue          string `mapstructure:"new_job_queue"`
	CompletedJobExchange string `mapstructure:"completed_job_exchange"`
}

QueueConfig - configuration of message queue (RabbitMQ)

type RequestCancelled 

type RequestCancelled struct{}

RequestCancelled is an error value that signals a cancelled HTTP request

func (RequestCancelled) Error 

func (c RequestCancelled) Error() string

type ServerConfig 

type ServerConfig struct {
	Host string
	Port int
}

ServerConfig - configuration of the Conveyor jov server

type UnprocessedJob 

type UnprocessedJob struct {
	ID uuid.UUID
	JobSpecification
}

UnprocessedJob describes a job which has been submitted, having been assigned a unique ID

type Waiter 

type Waiter struct {
	// contains filtered or unexported fields
}

Waiter implements an exponential backoff retry scheme: each successive call to Wait() will block for double the time of the previous call, up to a specified maximum time

func DefaultWaiter 

func DefaultWaiter() Waiter

DefaultWaiter constructs a Waiter object with the default wait values

func NewWaiter 

func NewWaiter(initWait, maxWait int) Waiter

NewWaiter constructs a Waiter object with the specified initial and maximum wait values

func (*Waiter) Reset 

func (w *Waiter) Reset()

Reset the state of the Waiter object. Next call to Wait will block for the initial wait duration

func (*Waiter) Wait 

func (w *Waiter) Wait()

Wait blocks for an amount of time as per the exponential backoff scheme

type Worker 

type Worker struct {
	// contains filtered or unexported fields
}

Worker encapsulates the loop where job descriptions received from the conveyor server are downloaded and processed

func NewWorker 

func NewWorker(cfg *Config) (*Worker, error)

NewWorker creates a new Worker object using a config object

func (*Worker) Close 

func (w *Worker) Close()

Close all the internal connections of the Worker object

func (*Worker) Loop 

func (w *Worker) Loop() error

Loop subscribes to the new job messages from the conveyor server and processes them one by one

type WorkerConfig 

type WorkerConfig struct {
	Name       string
	JobRetries int    `mapstructure:"job_retries"`
	TempDir    string `mapstructure:"temp_dir"`
}

WorkerConfig - configuration of the Conveyor worker daemon

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.