batchs

package
v1.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 27, 2021 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrTask means a task had a non-zero exit code
	ErrTask = errors.New("error running a task")
)

Functions

func NotImplementedHandler added in v1.1.6

func NotImplementedHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

NotImplementedHandler will return a 501 not implemented error.

Types

type Context

type Context struct {
	// contains filtered or unexported fields
}

Context tracks the path of where we store the queues. It is the only structure which knows about how the queues are persisted to disk. It should be promoted to an interface, probably.

func NewContext

func NewContext(q JobSource, taskpath, version string) *Context

NewContext creates a new context structure. taskpath is used to resolve task names into commands.

func (*Context) Run

func (ctx *Context) Run() error

Run the main event loop forever. Errors are logged. This function never returns.

type FileQueue added in v1.2.3

type FileQueue struct {
	// contains filtered or unexported fields
}

A FileQueue is a JobSource that uses a filesystem as the backing store.

func NewFileQueue

func NewFileQueue(basepath string) *FileQueue

NewFileQueue creates a new FileQueue having its state directories at basepath.

func (*FileQueue) FinishJob added in v1.2.3

func (fq *FileQueue) FinishJob(jb *Job) error

FinishJob moves a job to either the success or error directories depending on the job's state.

func (*FileQueue) Init added in v1.2.3

func (fq *FileQueue) Init() error

Init ensures all the required subdirectories exist, and returns any jobs in the processing directory to the queue.

func (*FileQueue) NextJob added in v1.2.3

func (fq *FileQueue) NextJob() (*Job, error)

NextJob returns the next job available on the file system. If there is no job available, it will block until a job appears.

type Job

type Job struct {
	Todo     []string // FIFO list of task names to execute
	Finished []Task   // list of finished tasks, from earliest to latest
	// contains filtered or unexported fields
}

Job collects all the logical consists of a single processing job. It is, essentially, a list of tasks.

type JobInfo added in v1.1.7

type JobInfo struct {
	Name, Status string
}

A JobInfo contains the information on jobs that we send to clients.

type JobSource

type JobSource interface {
	// Perform any initializations
	Init() error
	// Return the next available job. Block until one is available.
	// May return a job AND an error, in which case the job should be finalized.
	NextJob() (*Job, error)
	// Save and finalize the given job
	FinishJob(job *Job) error
}

A JobSource abstracts the way the main processor gets jobs.

type RESTServer added in v1.1.6

type RESTServer struct {
	// Port number to run bendo on. defaults to 15000
	PortNumber string
	QueuePath  *FileQueue
	Version    string
}

RESTServer holds the configuration for a Batch REST API server.

Set all the public fields and then call Run. Run will listen on the given port and handle requests. At the moment there is no maximum simultaneous request limit. Do not change any fields after calling Run.

func (*RESTServer) DeleteJobIDFileHandler added in v1.2.3

func (server *RESTServer) DeleteJobIDFileHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

DeleteJobIDFileHandler implements DELETE /jobs/:id/files/*path Returns 404 if data directory for given job id does not exist

func (*RESTServer) DeleteJobIDHandler added in v1.2.3

func (server *RESTServer) DeleteJobIDHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

DeleteJobIDHandler handles requests to DELETE /jobs/:id Returns 200 if id directory was deleted in one or more of [success, error, data], or doesn't exist

500 if something went wrong

func (*RESTServer) GetJobIDFileHandler added in v1.2.3

func (server *RESTServer) GetJobIDFileHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

GetJobIDFileHandler implements GET /jobs/:id/files/*path Returns 404 if job directory, or desired file for given job, does not exist

func (*RESTServer) GetJobIDHandler added in v1.2.3

func (server *RESTServer) GetJobIDHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

GetJobIDHandler handles requests to GET /jobs/:id Returns 404 if job id not found

200 + Json { "Name": id, "Status": [ success, error, queue, processing, ready ] }

func (*RESTServer) GetJobsHandler added in v1.1.6

func (server *RESTServer) GetJobsHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

GetJobsHandler handles requests to GET /jobs

func (*RESTServer) PutJobIDFileHandler added in v1.2.3

func (server *RESTServer) PutJobIDFileHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

PutJobIDFileHandler implements PUT /jobs/:id/files/*path Returns 404 if data directory for given job id does not exist

func (*RESTServer) PutJobIDHandler added in v1.2.3

func (server *RESTServer) PutJobIDHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

PutJobIDHandler handles requests to PUT /jobs/:id Returns 200 if id directory can be created under data

403 if it cannot (already exists)

func (*RESTServer) Run added in v1.1.6

func (server *RESTServer) Run()

Run initializes and starts all the goroutines used by the server. It then blocks listening for and handling http requests. This function never returns.

func (*RESTServer) SubmitJobIDHandler added in v1.2.3

func (server *RESTServer) SubmitJobIDHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params)

SubmitJobIDHandler handles requests to POST /jobs/:id Returns 200 if id directory was queued from one of [data, error, success], in that order

500 if something went wrong
404 if the job id was not found  in any of the searched directories

func (*RESTServer) WelcomeHandler added in v1.1.6

func (server *RESTServer) WelcomeHandler(writer http.ResponseWriter, request *http.Request, params httprouter.Params)

WelcomeHandler writes the version number as a response.

type State

type State int

State encodes the processing status of a Job.

const (
	// StateUnknown is the "zero state"
	StateUnknown State = iota
	// StateQueue means job is awaiting processing
	StateQueue
	// StateProcessing means job is being processed
	StateProcessing
	// StateSuccess means job is finished and was successful
	StateSuccess
	// StateError means job is done being processed and there was an error
	StateError
)

func (State) String

func (s State) String() string

type Task

type Task struct {
	Name   string
	Start  time.Time
	Finish time.Time
	Status string
}

Task is a singular work item inside a Job.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL