compute-image-tools: Index | Files | Directories

package daisy

import ""

Package daisy describes a daisy workflow.


Package Files

common.go disk.go disktype.go error.go firewall_rule.go forwarding_rule.go image.go instance.go license.go logger.go machinetype.go network.go project.go region.go resource.go resource_registry.go sources.go step.go step_attach_disks.go step_copy_gcs_objects.go step_create_disks.go step_create_firewall_rule.go step_create_forwarding_rule.go step_create_images.go step_create_instances.go step_create_networks.go step_create_subnetworks.go step_create_target_instance.go step_delete_resources.go step_deprecate_images.go step_detach_disks.go step_includeworkflow.go step_resize_disks.go step_start_instances.go step_stop_instances.go step_sub_workflow.go step_wait_for_instances_signal.go storage.go subnetwork.go target_instance.go validate.go workflow.go zone.go


var SleepFn = time.Sleep

SleepFn function is mocked on testing.

func JSONError Uses

func JSONError(file string, data []byte, err error) error

JSONError turns an error from json.Unmarshal and returns a more user friendly error.

type AttachDisk Uses

type AttachDisk struct {

    // Instance to attach to.
    Instance string
    // contains filtered or unexported fields

AttachDisk is used to attach a GCE disk into an instance.

type AttachDisks Uses

type AttachDisks []*AttachDisk

AttachDisks is a Daisy AttachDisks workflow step.

type CopyGCSObject Uses

type CopyGCSObject struct {
    Source, Destination string
    ACLRules            []*storage.ACLRule `json:",omitempty"`

CopyGCSObject copies a GCS object from Source to Destination.

type CopyGCSObjects Uses

type CopyGCSObjects []CopyGCSObject

CopyGCSObjects is a Daisy CopyGCSObject workflow step.

type CreateDisks Uses

type CreateDisks []*Disk

CreateDisks is a Daisy CreateDisks workflow step.

type CreateFirewallRules Uses

type CreateFirewallRules []*FirewallRule

CreateFirewallRules is a Daisy CreateFirewallRules workflow step.

type CreateForwardingRules Uses

type CreateForwardingRules []*ForwardingRule

CreateForwardingRules is a Daisy CreateForwardingRules workflow step.

type CreateImages Uses

type CreateImages struct {
    Images     []*Image
    ImagesBeta []*ImageBeta

CreateImages is a Daisy CreateImages workflow step.

func (*CreateImages) UnmarshalJSON Uses

func (ci *CreateImages) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals Image.

type CreateInstances Uses

type CreateInstances []*Instance

CreateInstances is a Daisy CreateInstances workflow step.

type CreateNetworks Uses

type CreateNetworks []*Network

CreateNetworks is a Daisy CreateNetwork workflow step.

type CreateSubnetworks Uses

type CreateSubnetworks []*Subnetwork

CreateSubnetworks is a Daisy CreateSubnetwork workflow step.

type CreateTargetInstances Uses

type CreateTargetInstances []*TargetInstance

CreateTargetInstances is a Daisy CreateTargetInstances workflow step.

type DeleteResources Uses

type DeleteResources struct {
    Disks       []string `json:",omitempty"`
    Images      []string `json:",omitempty"`
    Instances   []string `json:",omitempty"`
    Networks    []string `json:",omitempty"`
    Subnetworks []string `json:",omitempty"`
    GCSPaths    []string `json:",omitempty"`

DeleteResources deletes GCE/GCS resources.

type DeprecateImage Uses

type DeprecateImage struct {
    // Image to set deprecation status on.
    Image string
    // DeprecationStatus to set for image.
    DeprecationStatus compute.DeprecationStatus
    // Project image is in, overrides workflow Project.
    Project string `json:",omitempty"`

DeprecateImage sets the deprecation status on a GCE image.

type DeprecateImages Uses

type DeprecateImages []*DeprecateImage

DeprecateImages is a Daisy DeprecateImage workflow step.

type DetachDisk Uses

type DetachDisk struct {
    // Instance to detach diskName.
    Instance   string
    DeviceName string
    // contains filtered or unexported fields

DetachDisk is used to detach a GCE disk from an instance.

type DetachDisks Uses

type DetachDisks []*DetachDisk

DetachDisks is a Daisy DetachDisks workflow step.

type Disk Uses

type Disk struct {

    // Size of this disk.
    SizeGb string `json:"sizeGb,omitempty"`

Disk is used to create a GCE disk in a project.

func (*Disk) MarshalJSON Uses

func (d *Disk) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to prevent Disk from using compute.Disk's implementation.

type FailureMatches Uses

type FailureMatches []string

FailureMatches is a list of matching failure strings.

func (*FailureMatches) UnmarshalJSON Uses

func (fms *FailureMatches) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals FailureMatches.

type FirewallRule Uses

type FirewallRule struct {

FirewallRule is used to create a GCE firewallRule.

func (*FirewallRule) MarshalJSON Uses

func (fir *FirewallRule) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.Firewall's implementation.

type ForwardingRule Uses

type ForwardingRule struct {

ForwardingRule is used to create a GCE forwardingRule.

func (*ForwardingRule) MarshalJSON Uses

func (fr *ForwardingRule) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.ForwardingRule's implementation.

type GCSLogger Uses

type GCSLogger struct {
    // contains filtered or unexported fields

GCSLogger is a logger that writes to a GCS object.

func NewGCSLogger Uses

func NewGCSLogger(ctx context.Context, client *storage.Client, bucket, object string) *GCSLogger

NewGCSLogger creates a new GCSLogger.

func (*GCSLogger) Write Uses

func (l *GCSLogger) Write(b []byte) (int, error)

type Image Uses

type Image struct {

    // GuestOsFeatures to set for the image.
    GuestOsFeatures guestOsFeatures `json:"guestOsFeatures,omitempty"`

Image is used to create a GCE image using GA API. Supported sources are a GCE disk or a RAW image listed in Workflow.Sources.

func (*Image) MarshalJSON Uses

func (i *Image) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to prevent Image from using compute.Image's implementation.

type ImageBase Uses

type ImageBase struct {

    // Should an existing image of the same name be deleted, defaults to false
    // which will fail validation.
    OverWrite bool `json:",omitempty"`

    //Ignores license validation if 403/forbidden returned
    IgnoreLicenseValidationIfForbidden bool `json:",omitempty"`

ImageBase is a base struct for GA/Beta images. It holds the shared properties between the two.

type ImageBeta Uses

type ImageBeta struct {

    // GuestOsFeatures to set for the image.
    GuestOsFeatures guestOsFeatures `json:"guestOsFeatures,omitempty"`

ImageBeta is used to create a GCE image using Beta API. Supported sources are a GCE disk or a RAW image listed in Workflow.Sources.

type ImageInterface Uses

type ImageInterface interface {
    // contains filtered or unexported methods

ImageInterface represent abstract Image across different API stages (Alpha, Beta, API)

type IncludeWorkflow Uses

type IncludeWorkflow struct {
    Path     string
    Vars     map[string]string `json:",omitempty"`
    Workflow *Workflow         `json:",omitempty"`

IncludeWorkflow defines a Daisy workflow injection step. This step will 'include' the workflow found the path given into the parent workflow. Unlike a Subworkflow the included workflow will exist in the same namespace as the parent and have access to all its resources.

type Instance Uses

type Instance struct {

    // Additional metadata to set for the instance.
    Metadata map[string]string `json:"metadata,omitempty"`
    // OAuth2 scopes to give the instance. If left unset
    // will be added.
    Scopes []string `json:",omitempty"`
    // StartupScript is the Sources path to a startup script to use in this step.
    // This will be automatically mapped to the appropriate metadata key.
    StartupScript string `json:",omitempty"`

Instance is used to create a GCE instance. Output of serial port 1 will be streamed to the daisy logs directory.

func (*Instance) MarshalJSON Uses

func (i *Instance) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to prevent Instance from using compute.Instance's implementation.

type InstanceSignal Uses

type InstanceSignal struct {
    // Instance name to wait for.
    Name string
    // Interval to check for signal (default is 5s).
    // Must be parsable by
    Interval string `json:",omitempty"`

    // Wait for the instance to stop.
    Stopped bool `json:",omitempty"`
    // Wait for a string match in the serial output.
    SerialOutput *SerialOutput `json:",omitempty"`
    // contains filtered or unexported fields

InstanceSignal waits for a signal from an instance.

type LogEntry Uses

type LogEntry struct {
    LocalTimestamp time.Time `json:"localTimestamp"`
    WorkflowName   string    `json:"workflow"`
    StepName       string    `json:"stepName,omitempty"`
    StepType       string    `json:"stepType,omitempty"`
    SerialPort1    string    `json:"serialPort1,omitempty"`
    Message        string    `json:"message"`
    Type           string    `json:"type"`

LogEntry encapsulates a single log entry.

func (*LogEntry) String Uses

func (e *LogEntry) String() string

type Logger Uses

type Logger interface {
    WriteLogEntry(e *LogEntry)
    WriteSerialPortLogs(w *Workflow, instance string, buf bytes.Buffer)

Logger is a helper that encapsulates the logging logic for Daisy.

type Network Uses

type Network struct {
    AutoCreateSubnetworks *bool `json:"autoCreateSubnetworks,omitempty"`

Network is used to create a GCE network.

func (*Network) MarshalJSON Uses

func (n *Network) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.Network's implementation.

type ResizeDisk Uses

type ResizeDisk struct {
    // Name of the disk to be resized
    Name string

ResizeDisk is used to resize a GCE disk.

type ResizeDisks Uses

type ResizeDisks []*ResizeDisk

ResizeDisks is a Daisy ResizeDisks workflow step.

type Resource Uses

type Resource struct {
    // If this is unset Workflow.Project is used.
    Project string `json:",omitempty"`
    // Should this resource be cleaned up after the workflow?
    NoCleanup bool `json:",omitempty"`
    // If set Daisy will use this as the resource name instead of generating a name. Mutually exclusive with ExactName.
    RealName string `json:",omitempty"`
    // If set, Daisy will use the exact name as specified by the user instead of generating a name. Mutually exclusive with RealName.
    ExactName bool `json:",omitempty"`
    // contains filtered or unexported fields

Resource is the base struct for Daisy representation structs for GCE resources. This base struct defines some common user-definable fields, as well as some Daisy bookkeeping fields.

type SerialOutput Uses

type SerialOutput struct {
    Port         int64          `json:",omitempty"`
    SuccessMatch string         `json:",omitempty"`
    FailureMatch FailureMatches `json:"failureMatch,omitempty"`
    StatusMatch  string         `json:",omitempty"`

SerialOutput describes text signal strings that will be written to the serial port. A StatusMatch will print out the matching line from the StatusMatch onward. This step will not complete until a line in the serial output matches SuccessMatch or FailureMatch. A match with FailureMatch will cause the step to fail.

type StartInstances Uses

type StartInstances struct {
    Instances []string `json:",omitempty"`

StartInstances start GCE instances.

type Step Uses

type Step struct {

    //Timeout description
    TimeoutDescription string `json:",omitempty"`
    // Time to wait for this step to complete (default 10m).
    // Must be parsable by
    Timeout string `json:",omitempty"`

    // Only one of the below fields should exist for each instance of Step.
    AttachDisks            *AttachDisks            `json:",omitempty"`
    DetachDisks            *DetachDisks            `json:",omitempty"`
    CreateDisks            *CreateDisks            `json:",omitempty"`
    CreateForwardingRules  *CreateForwardingRules  `json:",omitempty"`
    CreateFirewallRules    *CreateFirewallRules    `json:",omitempty"`
    CreateImages           *CreateImages           `json:",omitempty"`
    CreateInstances        *CreateInstances        `json:",omitempty"`
    CreateNetworks         *CreateNetworks         `json:",omitempty"`
    CreateSubnetworks      *CreateSubnetworks      `json:",omitempty"`
    CreateTargetInstances  *CreateTargetInstances  `json:",omitempty"`
    CopyGCSObjects         *CopyGCSObjects         `json:",omitempty"`
    ResizeDisks            *ResizeDisks            `json:",omitempty"`
    StartInstances         *StartInstances         `json:",omitempty"`
    StopInstances          *StopInstances          `json:",omitempty"`
    DeleteResources        *DeleteResources        `json:",omitempty"`
    DeprecateImages        *DeprecateImages        `json:",omitempty"`
    IncludeWorkflow        *IncludeWorkflow        `json:",omitempty"`
    SubWorkflow            *SubWorkflow            `json:",omitempty"`
    WaitForInstancesSignal *WaitForInstancesSignal `json:",omitempty"`
    // contains filtered or unexported fields

Step is a single daisy workflow step.

func NewStep Uses

func NewStep(name string, w *Workflow, timeout time.Duration) *Step

NewStep creates a Step with given name and timeout with the specified workflow

type StopInstances Uses

type StopInstances struct {
    Instances []string `json:",omitempty"`

StopInstances stop GCE instances.

type SubWorkflow Uses

type SubWorkflow struct {
    Path     string
    Vars     map[string]string `json:",omitempty"`
    Workflow *Workflow         `json:",omitempty"`

SubWorkflow defines a Daisy sub workflow.

type Subnetwork Uses

type Subnetwork struct {

Subnetwork is used to create a GCE subnetwork.

func (*Subnetwork) MarshalJSON Uses

func (sn *Subnetwork) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.Subnetwork's implementation.

type TargetInstance Uses

type TargetInstance struct {

TargetInstance is used to create a GCE targetInstance.

func (*TargetInstance) MarshalJSON Uses

func (ti *TargetInstance) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.TargetInstance's implementation.

type TimeRecord Uses

type TimeRecord struct {
    Name      string
    StartTime time.Time
    EndTime   time.Time

TimeRecord is a type with info of a step execution time

type Var Uses

type Var struct {
    Value       string
    Required    bool   `json:",omitempty"`
    Description string `json:",omitempty"`

Var is a type with a flexible JSON representation. A Var can be represented by either a string, or by this struct definition. A Var that is represented by a string will unmarshal into the struct: {Value: <string>, Required: false, Description: ""}.

func (*Var) UnmarshalJSON Uses

func (v *Var) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals a Var.

type WaitForInstancesSignal Uses

type WaitForInstancesSignal []*InstanceSignal

WaitForInstancesSignal is a Daisy WaitForInstancesSignal workflow step.

type Workflow Uses

type Workflow struct {
    // Populated on New() construction.
    Cancel chan struct{} `json:"-"`

    // Workflow template fields.
    // Workflow name.
    Name string `json:",omitempty"`
    // Project to run in.
    Project string `json:",omitempty"`
    // Zone to run in.
    Zone string `json:",omitempty"`
    // GCS Path to use for scratch data and write logs/results to.
    GCSPath string `json:",omitempty"`
    // Path to OAuth credentials file.
    OAuthPath string `json:",omitempty"`
    // Sources used by this workflow, map of destination to source.
    Sources map[string]string `json:",omitempty"`
    // Vars defines workflow variables, substitution is done at Workflow run time.
    Vars  map[string]Var   `json:",omitempty"`
    Steps map[string]*Step `json:",omitempty"`
    // Map of steps to their dependencies.
    Dependencies map[string][]string `json:",omitempty"`
    // Default timout for each step, defaults to 10m.
    // Must be parsable by
    DefaultTimeout string `json:",omitempty"`

    Logger Logger `json:"-"`

    // Optional compute endpoint override.
    ComputeEndpoint string          `json:",omitempty"`
    ComputeClient   compute.Client  `json:"-"`
    StorageClient   *storage.Client `json:"-"`
    // contains filtered or unexported fields

Workflow is a single Daisy workflow workflow.

func New Uses

func New() *Workflow

New instantiates a new workflow.

func NewFromFile Uses

func NewFromFile(file string) (*Workflow, error)

NewFromFile reads and unmarshals a workflow file. Recursively reads subworkflow steps as well.

func (*Workflow) AddDependency Uses

func (w *Workflow) AddDependency(dependent *Step, dependencies ...*Step) error

AddDependency creates a dependency of dependent on each dependency. Returns an error if dependent or dependency are not steps in this workflow.

func (*Workflow) AddVar Uses

func (w *Workflow) AddVar(k, v string)

AddVar adds a variable set to the Workflow.

func (*Workflow) DisableCloudLogging Uses

func (w *Workflow) DisableCloudLogging()

DisableCloudLogging disables logging to Cloud Logging for this workflow.

func (*Workflow) DisableGCSLogging Uses

func (w *Workflow) DisableGCSLogging()

DisableGCSLogging disables logging to GCS for this workflow.

func (*Workflow) DisableStdoutLogging Uses

func (w *Workflow) DisableStdoutLogging()

DisableStdoutLogging disables logging to stdout for this workflow.

func (*Workflow) GetStepTimeRecords Uses

func (w *Workflow) GetStepTimeRecords() []TimeRecord

GetStepTimeRecords returns time records of each steps

func (*Workflow) ID Uses

func (w *Workflow) ID() string

ID is the unique identifyier for this Workflow.

func (*Workflow) LogStepInfo Uses

func (w *Workflow) LogStepInfo(stepName, stepType, format string, a ...interface{})

LogStepInfo logs information for the workflow step.

func (*Workflow) LogWorkflowInfo Uses

func (w *Workflow) LogWorkflowInfo(format string, a ...interface{})

LogWorkflowInfo logs information for the workflow.

func (*Workflow) NewIncludedWorkflowFromFile Uses

func (w *Workflow) NewIncludedWorkflowFromFile(file string) (*Workflow, error)

NewIncludedWorkflowFromFile reads and unmarshals a workflow with the same resources as the parent.

func (*Workflow) NewStep Uses

func (w *Workflow) NewStep(name string) (*Step, error)

NewStep instantiates a new, typeless step for this workflow. The step type must be specified before running this workflow.

func (*Workflow) NewSubWorkflow Uses

func (w *Workflow) NewSubWorkflow() *Workflow

NewSubWorkflow instantiates a new workflow as a child to this workflow.

func (*Workflow) NewSubWorkflowFromFile Uses

func (w *Workflow) NewSubWorkflowFromFile(file string) (*Workflow, error)

NewSubWorkflowFromFile reads and unmarshals a workflow as a child to this workflow.

func (*Workflow) PopulateClients Uses

func (w *Workflow) PopulateClients(ctx context.Context) error

PopulateClients populates the compute and storage clients for the workflow.

func (*Workflow) Print Uses

func (w *Workflow) Print(ctx context.Context)

Print populates then pretty prints the workflow.

func (*Workflow) Run Uses

func (w *Workflow) Run(ctx context.Context) error

Run runs a workflow.

func (*Workflow) RunWithModifiers Uses

func (w *Workflow) RunWithModifiers(
    ctx context.Context,
    preValidateWorkflowModifier WorkflowModifier,
    postValidateWorkflowModifier WorkflowModifier) error

RunWithModifiers runs a workflow with the ability to modify it before and/or after validation.

func (*Workflow) Validate Uses

func (w *Workflow) Validate(ctx context.Context) error

Validate runs validation on the workflow.

type WorkflowModifier Uses

type WorkflowModifier func(*Workflow)

WorkflowModifier is a function type for functions that can modify a Workflow object.


computePackage compute provides access to the Google Compute API.

Package daisy imports 28 packages (graph) and is imported by 11 packages. Updated 2019-08-23. Refresh now. Tools for package owners.