qri: github.com/qri-io/qri/update/cron Index | Files | Directories

package cron

import "github.com/qri-io/qri/update/cron"

Package cron schedules dataset and shell script updates

Index

Package Files

cron.go file_job_store.go http.go job.go job_store.go jobs.go

Variables

var (

    // DefaultCheckInterval is the frequency cron will check all stored jobs
    // for scheduled updates without any additional configuration. Qri recommends
    // not running updates more than once an hour for performance and storage
    // consumption reasons, making a check every minute a reasonable default
    DefaultCheckInterval = time.Minute
)
var ErrUnreachable = fmt.Errorf("cannot establish a connection to the server")

ErrUnreachable defines errors where the server cannot be reached TODO (b5): consider moving this to qfs

type Cron Uses

type Cron struct {
    // contains filtered or unexported fields
}

Cron coordinates the scheduling of running jobs at specified periodicities (intervals) with a provided job runner function

func NewCron Uses

func NewCron(schedule, log JobStore, factory RunJobFactory) *Cron

NewCron creates a Cron with the default check interval

func NewCronInterval Uses

func NewCronInterval(schedule, log JobStore, factory RunJobFactory, checkInterval time.Duration) *Cron

NewCronInterval creates a Cron with a custom check interval

func (*Cron) Job Uses

func (c *Cron) Job(ctx context.Context, name string) (*Job, error)

Job proxies to the schedule store for reading a job by name

func (*Cron) ListJobs Uses

func (c *Cron) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs proxies to the schedule store for reading jobs

func (*Cron) ListLogs Uses

func (c *Cron) ListLogs(ctx context.Context, offset, limit int) ([]*Job, error)

ListLogs returns a list of jobs that have been executed

func (*Cron) Log Uses

func (c *Cron) Log(ctx context.Context, logName string) (*Job, error)

Log gives a specific Job by logged job name

func (*Cron) LogFile Uses

func (c *Cron) LogFile(ctx context.Context, logName string) (io.ReadCloser, error)

LogFile returns a reader for a file at the given name

func (*Cron) Schedule Uses

func (c *Cron) Schedule(ctx context.Context, job *Job) error

Schedule adds a job to the cron scheduler

func (*Cron) ServeHTTP Uses

func (c *Cron) ServeHTTP(addr string) error

ServeHTTP spins up an HTTP server at the specified address

func (*Cron) Start Uses

func (c *Cron) Start(ctx context.Context) error

Start initiates the check loop, looking for updates to execute once at every iteration of the configured check interval. Start blocks until the passed context completes

func (*Cron) Unschedule Uses

func (c *Cron) Unschedule(ctx context.Context, name string) error

Unschedule removes a job from the cron scheduler, cancelling any future job executions

type DatasetOptions Uses

type DatasetOptions struct {
    Title     string
    Message   string
    Recall    string
    BodyPath  string
    FilePaths []string

    Publish             bool
    Strict              bool
    Force               bool
    ConvertFormatToPrev bool
    ShouldRender        bool

    Config  map[string]string
    Secrets map[string]string
}

DatasetOptions encapsulates options passed to `qri save` TODO (b5) - we should contribute flexbuffer support for golang & remove this entirely

func (*DatasetOptions) MarshalFlatbuffer Uses

func (o *DatasetOptions) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes to a builder

func (*DatasetOptions) UnmarshalFlatbuffer Uses

func (o *DatasetOptions) UnmarshalFlatbuffer(fbo *cronfb.DatasetOptions)

UnmarshalFlatbuffer reads flatbuffer data into DatasetOptions

type FlatbufferJobStore Uses

type FlatbufferJobStore struct {
    // contains filtered or unexported fields
}

FlatbufferJobStore is a jobstore implementation that saves to a file of flatbuffer bytes. FlatbufferJobStore is safe for concurrent use

func (*FlatbufferJobStore) CreateLogFile Uses

func (s *FlatbufferJobStore) CreateLogFile(j *Job) (f io.WriteCloser, path string, err error)

CreateLogFile creates a log file in the specified logs directory

func (*FlatbufferJobStore) DeleteJob Uses

func (s *FlatbufferJobStore) DeleteJob(ctx context.Context, name string) error

DeleteJob removes a job from the store by name. deleting a non-existent job won't return an error

func (*FlatbufferJobStore) Destroy Uses

func (s *FlatbufferJobStore) Destroy() error

Destroy removes the path entirely

func (*FlatbufferJobStore) Job Uses

func (s *FlatbufferJobStore) Job(ctx context.Context, name string) (*Job, error)

Job gets job details from the store by name

func (*FlatbufferJobStore) ListJobs Uses

func (s *FlatbufferJobStore) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs lists jobs currently in the store

func (*FlatbufferJobStore) PutJob Uses

func (s *FlatbufferJobStore) PutJob(ctx context.Context, job *Job) error

PutJob places a job in the store. If the job name matches the name of a job that already exists, it will be overwritten with the new job

func (*FlatbufferJobStore) PutJobs Uses

func (s *FlatbufferJobStore) PutJobs(ctx context.Context, add ...*Job) error

PutJobs places one or more jobs in the store. Putting a job who's name already exists must overwrite the previous job, making all job names unique

type HTTPClient Uses

type HTTPClient struct {
    Addr string
}

HTTPClient implements the Scheduler interface over HTTP, talking to a Cron HTTPServer

func (HTTPClient) Job Uses

func (c HTTPClient) Job(ctx context.Context, name string) (*Job, error)

Job gets a job by querying an HTTP server

func (HTTPClient) ListJobs Uses

func (c HTTPClient) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs jobs by querying an HTTP server

func (HTTPClient) ListLogs Uses

func (c HTTPClient) ListLogs(ctx context.Context, offset, limit int) ([]*Job, error)

ListLogs gives a log of executed jobs

func (HTTPClient) Log Uses

func (c HTTPClient) Log(ctx context.Context, logName string) (*Job, error)

Log returns a single executed job by job.LogName

func (HTTPClient) LogFile Uses

func (c HTTPClient) LogFile(ctx context.Context, logName string) (io.ReadCloser, error)

LogFile returns a reader for a file at the given name

func (HTTPClient) Ping Uses

func (c HTTPClient) Ping() error

Ping confirms client can dial the server, if a connection cannot be established at all, Ping will return ErrUnreachable

func (HTTPClient) Schedule Uses

func (c HTTPClient) Schedule(ctx context.Context, job *Job) error

Schedule adds a job to the cron scheduler via an HTTP request

func (HTTPClient) Unschedule Uses

func (c HTTPClient) Unschedule(ctx context.Context, name string) error

Unschedule removes a job from scheduling

type Job Uses

type Job struct {
    Name         string                    `json:"name"`
    Alias        string                    `json:"alias"`
    Type         JobType                   `json:"type"`
    Periodicity  iso8601.RepeatingInterval `json:"periodicity"`
    PrevRunStart time.Time                 `json:"lastRunStart,omitempty"`

    RunNumber   int64     `json:"runNumber,omitempty"`
    RunStart    time.Time `json:"runStart,omitempty"`
    RunStop     time.Time `json:"runStop,omitempty"`
    RunError    string    `json:"runError,omitempty"`
    LogFilePath string    `json:"logFilePath,omitempty"`

    RepoPath string `json:"repoPath,omitempty"`

    Options Options `json:"options,omitempty"`
}

Job represents a "cron job" that can be scheduled for repeated execution at a specified Periodicity (time interval)

a Job struct has one of three "run" states, which describe it's position in the execution lifecycle: * unexected: job.RunStart.IsZero() && job.RunStop.IsZero() * executing: !job.RunStart.IsZero() && job.RunStop.IsZero() * completed: !job.RunStart.IsZero() && !job.RunStop.IsZero()

func (*Job) Copy Uses

func (job *Job) Copy() *Job

Copy creates a copy of a job

func (*Job) FlatbufferBytes Uses

func (job *Job) FlatbufferBytes() []byte

FlatbufferBytes formats a job as a flatbuffer byte slice

func (*Job) LogName Uses

func (job *Job) LogName() string

LogName returns a canonical name string for a job that's executed and saved to a logging system

func (*Job) MarshalFlatbuffer Uses

func (job *Job) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes a job to a builder

func (*Job) NextExec Uses

func (job *Job) NextExec() time.Time

NextExec returns the next time execution horizon. If job periodicity is improperly configured, the returned time will be zero

func (*Job) UnmarshalFlatbuffer Uses

func (job *Job) UnmarshalFlatbuffer(j *cronfb.Job) error

UnmarshalFlatbuffer decodes a job from a flatbuffer

func (*Job) Validate Uses

func (job *Job) Validate() error

Validate confirms a Job contains valid details for scheduling

type JobStore Uses

type JobStore interface {
    // JobStores must implement the ReadJobs interface for fetching stored jobs
    ReadJobs
    // PutJob places one or more jobs in the store. Putting a job who's name
    // already exists must overwrite the previous job, making all job names unique
    PutJobs(context.Context, ...*Job) error
    // PutJob places a job in the store. Putting a job who's name already exists
    // must overwrite the previous job, making all job names unique
    PutJob(context.Context, *Job) error
    // DeleteJob removes a job from the store
    DeleteJob(ctx context.Context, name string) error
}

JobStore handles the persistence of Job details. JobStore implementations must be safe for concurrent use

func NewFlatbufferJobStore Uses

func NewFlatbufferJobStore(path string) JobStore

NewFlatbufferJobStore creates a job store that persists to a file

type JobType Uses

type JobType string

JobType is a type for distinguishing between two different kinds of jobs JobType should be used as a shorthand for defining how to execute a job

const (
    // JTDataset indicates a job that runs "qri update" on a dataset specified
    // by Job Name. The job periodicity is determined by the specified dataset's
    // Meta.AccrualPeriodicity field. LastRun should closely match the datasets's
    // latest Commit.Timestamp value
    JTDataset JobType = "dataset"
    // JTShellScript represents a shell script to be run locally, which might
    // update one or more datasets. A non-zero exit code from shell script
    // indicates the job failed to execute properly
    JTShellScript JobType = "shell"
)

func (JobType) Enum Uses

func (jt JobType) Enum() int8

Enum returns the enumerated representation of a JobType

type LogFileCreator Uses

type LogFileCreator interface {
    // CreateLogFile returns a file to write output to
    CreateLogFile(job *Job) (f io.WriteCloser, path string, err error)
}

LogFileCreator is an interface for generating log files to write to, JobStores should implement this interface

type MemJobStore Uses

type MemJobStore struct {
    // contains filtered or unexported fields
}

MemJobStore is an in-memory implementation of the JobStore interface Jobs stored in MemJobStore can be persisted for the duration of a process at the longest. MemJobStore is safe for concurrent use

func (*MemJobStore) DeleteJob Uses

func (s *MemJobStore) DeleteJob(ctx context.Context, name string) error

DeleteJob removes a job from the store by name. deleting a non-existent job won't return an error

func (*MemJobStore) Job Uses

func (s *MemJobStore) Job(ctx context.Context, name string) (*Job, error)

Job gets job details from the store by name

func (*MemJobStore) ListJobs Uses

func (s *MemJobStore) ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)

ListJobs lists jobs currently in the store

func (*MemJobStore) PutJob Uses

func (s *MemJobStore) PutJob(ctx context.Context, job *Job) error

PutJob places a job in the store. If the job name matches the name of a job that already exists, it will be overwritten with the new job

func (*MemJobStore) PutJobs Uses

func (s *MemJobStore) PutJobs(ctx context.Context, js ...*Job) error

PutJobs places one or more jobs in the store. Putting a job whose name already exists must overwrite the previous job, making all job names unique

type Options Uses

type Options interface {
    MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT
}

Options is an interface for job options

type ReadJobs Uses

type ReadJobs interface {
    // ListJobs should return the set of jobs sorted in reverse-chronological order
    // (newest first order) of the last time they were run. When two LastRun times
    // are equal, Jobs should alpha sort the names
    // passing a limit and offset of 0 must return the entire list of stored jobs
    ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)
    // Job gets a job by it's name. All job names in a set must be unique. It's
    // the job of the set backing ReadJobs functions to enforce uniqueness
    Job(ctx context.Context, name string) (*Job, error)
}

ReadJobs are functions for fetching a set of jobs. ReadJobs defines canoncial behavior for listing & fetching jobs

type RunJobFactory Uses

type RunJobFactory func(ctx context.Context) (runner RunJobFunc)

RunJobFactory is a function that returns a runner

type RunJobFunc Uses

type RunJobFunc func(ctx context.Context, streams ioes.IOStreams, job *Job) error

RunJobFunc is a function for executing a job. Cron takes care of scheduling job execution, and delegates the work of executing a job to a RunJobFunc implementation.

type Scheduler Uses

type Scheduler interface {
    // ListJobs lists currently scheduled jobs
    ListJobs(ctx context.Context, offset, limit int) ([]*Job, error)
    // Job gets a single scheduled job by name
    Job(ctx context.Context, name string) (*Job, error)

    // Schedule adds a job to the scheduler for execution once every period
    Schedule(ctx context.Context, job *Job) error
    // Unschedule removes a job from the scheduler
    Unschedule(ctx context.Context, name string) error

    // ListLogs gives a log of executed jobs
    ListLogs(ctx context.Context, offset, limit int) ([]*Job, error)
    // Log returns a single executed job by job.LogName
    Log(ctx context.Context, logName string) (*Job, error)
    // JobLogFile returns a reader for a file at the given name
    LogFile(ctx context.Context, logName string) (io.ReadCloser, error)
}

Scheduler is the generic interface for the Cron Scheduler, it's implemented by both Cron and HTTPClient for easier RPC communication

type ShellScriptOptions Uses

type ShellScriptOptions struct {
}

ShellScriptOptions encapsulates options for running a shell script cron job

func (*ShellScriptOptions) MarshalFlatbuffer Uses

func (o *ShellScriptOptions) MarshalFlatbuffer(builder *flatbuffers.Builder) flatbuffers.UOffsetT

MarshalFlatbuffer writes to a builder

Directories

PathSynopsis
cron_fbs

Package cron imports 18 packages (graph) and is imported by 3 packages. Updated 2019-07-17. Refresh now. Tools for package owners.