compute-image-tools: Index | Files | Directories

package daisy

import ""

Package daisy describes a daisy workflow.


Package Files

common.go disk.go disktype.go error.go firewall_rule.go forwarding_rule.go image.go instance.go license.go logger.go machineimage.go machinetype.go network.go project.go region.go resource.go resource_registry.go sources.go step.go step_attach_disks.go step_copy_gcs_objects.go step_create_disks.go step_create_firewall_rule.go step_create_forwarding_rule.go step_create_images.go step_create_instances.go step_create_machine_images.go step_create_networks.go step_create_subnetworks.go step_create_target_instance.go step_delete_resources.go step_deprecate_images.go step_detach_disks.go step_includeworkflow.go step_resize_disks.go step_start_instances.go step_stop_instances.go step_sub_workflow.go step_update_instances_metadata.go step_wait_for_instances_signal.go storage.go subnetwork.go target_instance.go validate.go workflow.go zone.go


var SleepFn = time.Sleep

SleepFn function is mocked on testing.

func CombineGuestOSFeatures Uses

func CombineGuestOSFeatures(features1 []*compute.GuestOsFeature,
    features2 ...string) []*compute.GuestOsFeature

CombineGuestOSFeatures merges two slices of Guest OS features and returns a new slice instance. Duplicates are removed.

func CombineGuestOSFeaturesBeta Uses

func CombineGuestOSFeaturesBeta(features1 []*computeBeta.GuestOsFeature,
    features2 ...string) []*computeBeta.GuestOsFeature

CombineGuestOSFeaturesBeta merges two slices of Beta Guest OS features and returns a new slice instance. Duplicates are removed.

func JSONError Uses

func JSONError(file string, data []byte, err error) error

JSONError turns an error from json.Unmarshal and returns a more user friendly error.

type AttachDisk Uses

type AttachDisk struct {

    // Instance to attach to.
    Instance string
    // contains filtered or unexported fields

AttachDisk is used to attach a GCE disk into an instance.

type AttachDisks Uses

type AttachDisks []*AttachDisk

AttachDisks is a Daisy AttachDisks workflow step.

type CopyGCSObject Uses

type CopyGCSObject struct {
    Source, Destination string
    ACLRules            []*storage.ACLRule `json:",omitempty"`

CopyGCSObject copies a GCS object from Source to Destination.

type CopyGCSObjects Uses

type CopyGCSObjects []CopyGCSObject

CopyGCSObjects is a Daisy CopyGCSObject workflow step.

type CreateDisks Uses

type CreateDisks []*Disk

CreateDisks is a Daisy CreateDisks workflow step.

type CreateFirewallRules Uses

type CreateFirewallRules []*FirewallRule

CreateFirewallRules is a Daisy CreateFirewallRules workflow step.

type CreateForwardingRules Uses

type CreateForwardingRules []*ForwardingRule

CreateForwardingRules is a Daisy CreateForwardingRules workflow step.

type CreateImages Uses

type CreateImages struct {
    Images     []*Image
    ImagesBeta []*ImageBeta

CreateImages is a Daisy CreateImages workflow step.

func (*CreateImages) UnmarshalJSON Uses

func (ci *CreateImages) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals Image.

type CreateInstances Uses

type CreateInstances []*Instance

CreateInstances is a Daisy CreateInstances workflow step.

type CreateMachineImages Uses

type CreateMachineImages []*MachineImage

CreateMachineImages is a Daisy workflow step for creating machine images.

type CreateNetworks Uses

type CreateNetworks []*Network

CreateNetworks is a Daisy CreateNetwork workflow step.

type CreateSubnetworks Uses

type CreateSubnetworks []*Subnetwork

CreateSubnetworks is a Daisy CreateSubnetwork workflow step.

type CreateTargetInstances Uses

type CreateTargetInstances []*TargetInstance

CreateTargetInstances is a Daisy CreateTargetInstances workflow step.

type DError Uses

type DError interface {

    AnonymizedErrs() []string
    CausedByErrType(t string) bool
    // contains filtered or unexported methods

DError is a Daisy external error type. It has: - optional error typing - multiple error aggregation - safe error messages in which privacy information is removed

Default implementation: The default DError implementation is flat, DError.add(anotherDErr) will merge the two dErrs into a single, flat DError instead of making anotherDErr a child to DError.

func Errf Uses

func Errf(format string, a ...interface{}) DError

Errf returns a DError by constructing error message with given format.

func ToDError Uses

func ToDError(e error) DError

ToDError returns a DError. ToDError is used to wrap another error as a DError. If e is already a DError, e is copied and returned. If e is a normal error, error message is reused as format. If e is nil, nil is returned.

type DeleteResources Uses

type DeleteResources struct {
    Disks         []string `json:",omitempty"`
    Images        []string `json:",omitempty"`
    MachineImages []string `json:",omitempty"`
    Instances     []string `json:",omitempty"`
    Networks      []string `json:",omitempty"`
    Subnetworks   []string `json:",omitempty"`
    GCSPaths      []string `json:",omitempty"`

DeleteResources deletes GCE/GCS resources.

type DeprecateImage Uses

type DeprecateImage struct {
    // Image to set deprecation status on.
    Image string
    // DeprecationStatus to set for image.
    DeprecationStatus compute.DeprecationStatus
    // Project image is in, overrides workflow Project.
    Project string `json:",omitempty"`

DeprecateImage sets the deprecation status on a GCE image.

type DeprecateImages Uses

type DeprecateImages []*DeprecateImage

DeprecateImages is a Daisy DeprecateImage workflow step.

type DetachDisk Uses

type DetachDisk struct {
    // Instance to detach diskName.
    Instance   string
    DeviceName string
    // contains filtered or unexported fields

DetachDisk is used to detach a GCE disk from an instance.

type DetachDisks Uses

type DetachDisks []*DetachDisk

DetachDisks is a Daisy DetachDisks workflow step.

type Disk Uses

type Disk struct {

    // If this is enabled, then WINDOWS will be added to the
    // disk's guestOsFeatures. This is a string since daisy
    // replaces variables after JSON has been parsed.
    // (If it were boolean, the JSON marshaller throws
    // an error when it sees something like `${is_windows}`)
    IsWindows string `json:"isWindows,omitempty"`

    // Size of this disk.
    SizeGb string `json:"sizeGb,omitempty"`

    // Fallback to pd-standard when quota is not enough for higher-level pd
    FallbackToPdStandard bool `json:"fallbackToPdStandard,omitempty"`

Disk is used to create a GCE disk in a project.

func (*Disk) MarshalJSON Uses

func (d *Disk) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to prevent Disk from using compute.Disk's implementation.

type FailureMatches Uses

type FailureMatches []string

FailureMatches is a list of matching failure strings.

func (*FailureMatches) UnmarshalJSON Uses

func (fms *FailureMatches) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals FailureMatches.

type FirewallRule Uses

type FirewallRule struct {

FirewallRule is used to create a GCE firewallRule.

func (*FirewallRule) MarshalJSON Uses

func (fir *FirewallRule) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.Firewall's implementation.

type ForwardingRule Uses

type ForwardingRule struct {

ForwardingRule is used to create a GCE forwardingRule.

func (*ForwardingRule) MarshalJSON Uses

func (fr *ForwardingRule) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.ForwardingRule's implementation.

type GCSLogger Uses

type GCSLogger struct {
    // contains filtered or unexported fields

GCSLogger is a logger that writes to a GCS object.

func NewGCSLogger Uses

func NewGCSLogger(ctx context.Context, client *storage.Client, bucket, object string) *GCSLogger

NewGCSLogger creates a new GCSLogger.

func (*GCSLogger) Write Uses

func (l *GCSLogger) Write(b []byte) (int, error)

type Image Uses

type Image struct {

    // GuestOsFeatures to set for the image.
    GuestOsFeatures guestOsFeatures `json:"guestOsFeatures,omitempty"`

Image is used to create a GCE image using GA API. Supported sources are a GCE disk or a RAW image listed in Workflow.Sources.

func (*Image) MarshalJSON Uses

func (i *Image) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to prevent Image from using compute.Image's implementation.

type ImageBase Uses

type ImageBase struct {

    // Should an existing image of the same name be deleted, defaults to false
    // which will fail validation.
    OverWrite bool `json:",omitempty"`

    //Ignores license validation if 403/forbidden returned
    IgnoreLicenseValidationIfForbidden bool `json:",omitempty"`

ImageBase is a base struct for GA/Beta images. It holds the shared properties between the two.

type ImageBeta Uses

type ImageBeta struct {

    // GuestOsFeatures to set for the image.
    GuestOsFeatures guestOsFeatures `json:"guestOsFeatures,omitempty"`

ImageBeta is used to create a GCE image using Beta API. Supported sources are a GCE disk or a RAW image listed in Workflow.Sources.

type ImageInterface Uses

type ImageInterface interface {
    // contains filtered or unexported methods

ImageInterface represent abstract Image across different API stages (Alpha, Beta, API)

type IncludeWorkflow Uses

type IncludeWorkflow struct {
    Path     string
    Vars     map[string]string `json:",omitempty"`
    Workflow *Workflow         `json:",omitempty"`

IncludeWorkflow defines a Daisy workflow injection step. This step will 'include' the workflow found the path given into the parent workflow. Unlike a Subworkflow the included workflow will exist in the same namespace as the parent and have access to all its resources.

type Instance Uses

type Instance struct {

    // Additional metadata to set for the instance.
    Metadata map[string]string `json:"metadata,omitempty"`
    // OAuth2 scopes to give the instance. If left unset
    // will be added.
    Scopes []string `json:",omitempty"`
    // StartupScript is the Sources path to a startup script to use in this step.
    // This will be automatically mapped to the appropriate metadata key.
    StartupScript string `json:",omitempty"`

Instance is used to create a GCE instance. Output of serial port 1 will be streamed to the daisy logs directory.

func (*Instance) MarshalJSON Uses

func (i *Instance) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to prevent Instance from using compute.Instance's implementation.

type InstanceSignal Uses

type InstanceSignal struct {
    // Instance name to wait for.
    Name string
    // Interval to check for signal (default is 5s).
    // Must be parsable by
    Interval string `json:",omitempty"`

    // Wait for the instance to stop.
    Stopped bool `json:",omitempty"`
    // Wait for a string match in the serial output.
    SerialOutput *SerialOutput `json:",omitempty"`
    // contains filtered or unexported fields

InstanceSignal waits for a signal from an instance.

type LogEntry Uses

type LogEntry struct {
    LocalTimestamp time.Time `json:"localTimestamp"`
    WorkflowName   string    `json:"workflow"`
    StepName       string    `json:"stepName,omitempty"`
    StepType       string    `json:"stepType,omitempty"`
    SerialPort1    string    `json:"serialPort1,omitempty"`
    Message        string    `json:"message"`
    Type           string    `json:"type"`

LogEntry encapsulates a single log entry.

func (*LogEntry) String Uses

func (e *LogEntry) String() string

type Logger Uses

type Logger interface {
    WriteLogEntry(e *LogEntry)
    WriteSerialPortLogs(w *Workflow, instance string, buf bytes.Buffer)
    ReadSerialPortLogs() []string

Logger is a helper that encapsulates the logging logic for Daisy.

type MachineImage Uses

type MachineImage struct {

    // Should an existing machine image of the same name be deleted.
    // Defaults to false which will fail validation if a machine image with the
    // same name exists in the project.
    OverWrite bool `json:",omitempty"`

MachineImage is used to create a GCE machine image.

func (*MachineImage) MarshalJSON Uses

func (mi *MachineImage) MarshalJSON() ([]byte, error)

MarshalJSON is a workaround to prevent MachineImage from using computeBeta.MachineImage's implementation.

type Network Uses

type Network struct {
    AutoCreateSubnetworks *bool `json:"autoCreateSubnetworks,omitempty"`

Network is used to create a GCE network.

func (*Network) MarshalJSON Uses

func (n *Network) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.Network's implementation.

type ResizeDisk Uses

type ResizeDisk struct {
    // Name of the disk to be resized
    Name string

ResizeDisk is used to resize a GCE disk.

type ResizeDisks Uses

type ResizeDisks []*ResizeDisk

ResizeDisks is a Daisy ResizeDisks workflow step.

type Resource Uses

type Resource struct {
    // If this is unset Workflow.Project is used.
    Project string `json:",omitempty"`
    // Should this resource be cleaned up after the workflow?
    NoCleanup bool `json:",omitempty"`
    // If set Daisy will use this as the resource name instead of generating a name. Mutually exclusive with ExactName.
    RealName string `json:",omitempty"`
    // If set, Daisy will use the exact name as specified by the user instead of generating a name. Mutually exclusive with RealName.
    ExactName bool `json:",omitempty"`
    // contains filtered or unexported fields

Resource is the base struct for Daisy representation structs for GCE resources. This base struct defines some common user-definable fields, as well as some Daisy bookkeeping fields.

type SerialOutput Uses

type SerialOutput struct {
    Port         int64          `json:",omitempty"`
    SuccessMatch string         `json:",omitempty"`
    FailureMatch FailureMatches `json:"failureMatch,omitempty"`
    StatusMatch  string         `json:",omitempty"`

SerialOutput describes text signal strings that will be written to the serial port. A StatusMatch will print out the matching line from the StatusMatch onward. This step will not complete until a line in the serial output matches SuccessMatch or FailureMatch. A match with FailureMatch will cause the step to fail.

type StartInstances Uses

type StartInstances struct {
    Instances []string `json:",omitempty"`

StartInstances start GCE instances.

type Step Uses

type Step struct {

    //Timeout description
    TimeoutDescription string `json:",omitempty"`
    // Time to wait for this step to complete (default 10m).
    // Must be parsable by
    Timeout string `json:",omitempty"`

    // Only one of the below fields should exist for each instance of Step.
    AttachDisks             *AttachDisks             `json:",omitempty"`
    DetachDisks             *DetachDisks             `json:",omitempty"`
    CreateDisks             *CreateDisks             `json:",omitempty"`
    CreateForwardingRules   *CreateForwardingRules   `json:",omitempty"`
    CreateFirewallRules     *CreateFirewallRules     `json:",omitempty"`
    CreateImages            *CreateImages            `json:",omitempty"`
    CreateMachineImages     *CreateMachineImages     `json:",omitempty"`
    CreateInstances         *CreateInstances         `json:",omitempty"`
    CreateNetworks          *CreateNetworks          `json:",omitempty"`
    CreateSubnetworks       *CreateSubnetworks       `json:",omitempty"`
    CreateTargetInstances   *CreateTargetInstances   `json:",omitempty"`
    CopyGCSObjects          *CopyGCSObjects          `json:",omitempty"`
    ResizeDisks             *ResizeDisks             `json:",omitempty"`
    StartInstances          *StartInstances          `json:",omitempty"`
    StopInstances           *StopInstances           `json:",omitempty"`
    DeleteResources         *DeleteResources         `json:",omitempty"`
    DeprecateImages         *DeprecateImages         `json:",omitempty"`
    IncludeWorkflow         *IncludeWorkflow         `json:",omitempty"`
    SubWorkflow             *SubWorkflow             `json:",omitempty"`
    WaitForInstancesSignal  *WaitForInstancesSignal  `json:",omitempty"`
    UpdateInstancesMetadata *UpdateInstancesMetadata `json:",omitempty"`
    // contains filtered or unexported fields

Step is a single daisy workflow step.

func NewStep Uses

func NewStep(name string, w *Workflow, timeout time.Duration) *Step

NewStep creates a Step with given name and timeout with the specified workflow. If timeout is less or equal to zero, defaultTimeout from the workflow will be used

func NewStepDefaultTimeout Uses

func NewStepDefaultTimeout(name string, w *Workflow) *Step

NewStepDefaultTimeout creates a Step with given name using default timeout from the workflow

type StopInstances Uses

type StopInstances struct {
    Instances []string `json:",omitempty"`

StopInstances stop GCE instances.

type SubWorkflow Uses

type SubWorkflow struct {
    Path     string
    Vars     map[string]string `json:",omitempty"`
    Workflow *Workflow         `json:",omitempty"`

SubWorkflow defines a Daisy sub workflow.

type Subnetwork Uses

type Subnetwork struct {

Subnetwork is used to create a GCE subnetwork.

func (*Subnetwork) MarshalJSON Uses

func (sn *Subnetwork) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.Subnetwork's implementation.

type TargetInstance Uses

type TargetInstance struct {

TargetInstance is used to create a GCE targetInstance.

func (*TargetInstance) MarshalJSON Uses

func (ti *TargetInstance) MarshalJSON() ([]byte, error)

MarshalJSON is a hacky workaround to compute.TargetInstance's implementation.

type TimeRecord Uses

type TimeRecord struct {
    Name      string
    StartTime time.Time
    EndTime   time.Time

TimeRecord is a type with info of a step execution time

type UpdateInstanceMetadata Uses

type UpdateInstanceMetadata struct {
    // Metadata
    Metadata map[string]string `json:"metadata,omitempty"`

    // Instance to attach to.
    Instance string
    // contains filtered or unexported fields

UpdateInstanceMetadata is used to update an instance metadata.

type UpdateInstancesMetadata Uses

type UpdateInstancesMetadata []*UpdateInstanceMetadata

UpdateInstancesMetadata is a Daisy UpdateInstancesMetadata workflow step.

type Var Uses

type Var struct {
    Value       string
    Required    bool   `json:",omitempty"`
    Description string `json:",omitempty"`

Var is a type with a flexible JSON representation. A Var can be represented by either a string, or by this struct definition. A Var that is represented by a string will unmarshal into the struct: {Value: <string>, Required: false, Description: ""}.

func (*Var) UnmarshalJSON Uses

func (v *Var) UnmarshalJSON(b []byte) error

UnmarshalJSON unmarshals a Var.

type WaitForInstancesSignal Uses

type WaitForInstancesSignal []*InstanceSignal

WaitForInstancesSignal is a Daisy WaitForInstancesSignal workflow step.

type Workflow Uses

type Workflow struct {
    // Populated on New() construction.
    Cancel chan struct{} `json:"-"`

    // Workflow template fields.
    // Workflow name.
    Name string `json:",omitempty"`
    // Project to run in.
    Project string `json:",omitempty"`
    // Zone to run in.
    Zone string `json:",omitempty"`
    // GCS Path to use for scratch data and write logs/results to.
    GCSPath string `json:",omitempty"`
    // Path to OAuth credentials file.
    OAuthPath string `json:",omitempty"`
    // Sources used by this workflow, map of destination to source.
    Sources map[string]string `json:",omitempty"`
    // Vars defines workflow variables, substitution is done at Workflow run time.
    Vars  map[string]Var   `json:",omitempty"`
    Steps map[string]*Step `json:",omitempty"`
    // Map of steps to their dependencies.
    Dependencies map[string][]string `json:",omitempty"`
    // Default timout for each step, defaults to 10m.
    // Must be parsable by
    DefaultTimeout string `json:",omitempty"`

    Logger Logger `json:"-"`

    // Optional compute endpoint override.
    ComputeEndpoint string          `json:",omitempty"`
    ComputeClient   compute.Client  `json:"-"`
    StorageClient   *storage.Client `json:"-"`

    //Forces cleanup on error of all resources, including those marked with NoCleanup
    ForceCleanupOnError bool
    // contains filtered or unexported fields

Workflow is a single Daisy workflow workflow.

func New Uses

func New() *Workflow

New instantiates a new workflow.

func NewFromFile Uses

func NewFromFile(file string) (*Workflow, error)

NewFromFile reads and unmarshals a workflow file. Recursively reads subworkflow steps as well.

func (*Workflow) AddDependency Uses

func (w *Workflow) AddDependency(dependent *Step, dependencies ...*Step) error

AddDependency creates a dependency of dependent on each dependency. Returns an error if dependent or dependency are not steps in this workflow.

func (*Workflow) AddSerialConsoleOutputValue Uses

func (w *Workflow) AddSerialConsoleOutputValue(k, v string)

AddSerialConsoleOutputValue adds an serial-output key-value pair to the Workflow.

func (*Workflow) AddVar Uses

func (w *Workflow) AddVar(k, v string)

AddVar adds a variable set to the Workflow.

func (*Workflow) DisableCloudLogging Uses

func (w *Workflow) DisableCloudLogging()

DisableCloudLogging disables logging to Cloud Logging for this workflow.

func (*Workflow) DisableGCSLogging Uses

func (w *Workflow) DisableGCSLogging()

DisableGCSLogging disables logging to GCS for this workflow.

func (*Workflow) DisableStdoutLogging Uses

func (w *Workflow) DisableStdoutLogging()

DisableStdoutLogging disables logging to stdout for this workflow.

func (*Workflow) GetSerialConsoleOutputValue Uses

func (w *Workflow) GetSerialConsoleOutputValue(k string) string

GetSerialConsoleOutputValue gets an serial-output value by key.

func (*Workflow) GetStepTimeRecords Uses

func (w *Workflow) GetStepTimeRecords() []TimeRecord

GetStepTimeRecords returns time records of each steps

func (*Workflow) ID Uses

func (w *Workflow) ID() string

ID is the unique identifyier for this Workflow.

func (*Workflow) IterateWorkflowSteps Uses

func (w *Workflow) IterateWorkflowSteps(cb func(step *Step))

IterateWorkflowSteps iterates over all workflow steps, including included workflow steps, and calls cb callback function

func (*Workflow) LogStepInfo Uses

func (w *Workflow) LogStepInfo(stepName, stepType, format string, a ...interface{})

LogStepInfo logs information for the workflow step.

func (*Workflow) LogWorkflowInfo Uses

func (w *Workflow) LogWorkflowInfo(format string, a ...interface{})

LogWorkflowInfo logs information for the workflow.

func (*Workflow) NewIncludedWorkflowFromFile Uses

func (w *Workflow) NewIncludedWorkflowFromFile(file string) (*Workflow, error)

NewIncludedWorkflowFromFile reads and unmarshals a workflow with the same resources as the parent.

func (*Workflow) NewStep Uses

func (w *Workflow) NewStep(name string) (*Step, error)

NewStep instantiates a new, typeless step for this workflow. The step type must be specified before running this workflow.

func (*Workflow) NewSubWorkflow Uses

func (w *Workflow) NewSubWorkflow() *Workflow

NewSubWorkflow instantiates a new workflow as a child to this workflow.

func (*Workflow) NewSubWorkflowFromFile Uses

func (w *Workflow) NewSubWorkflowFromFile(file string) (*Workflow, error)

NewSubWorkflowFromFile reads and unmarshals a workflow as a child to this workflow.

func (*Workflow) PopulateClients Uses

func (w *Workflow) PopulateClients(ctx context.Context) error

PopulateClients populates the compute and storage clients for the workflow.

func (*Workflow) Print Uses

func (w *Workflow) Print(ctx context.Context)

Print populates then pretty prints the workflow.

func (*Workflow) Run Uses

func (w *Workflow) Run(ctx context.Context) error

Run runs a workflow.

func (*Workflow) RunWithModifiers Uses

func (w *Workflow) RunWithModifiers(
    ctx context.Context,
    preValidateWorkflowModifier WorkflowModifier,
    postValidateWorkflowModifier WorkflowModifier) (err DError)

RunWithModifiers runs a workflow with the ability to modify it before and/or after validation.

func (*Workflow) SetLogProcessHook Uses

func (w *Workflow) SetLogProcessHook(hook func(string) string)

SetLogProcessHook sets a hook function to process log string

func (*Workflow) Validate Uses

func (w *Workflow) Validate(ctx context.Context) DError

Validate runs validation on the workflow.

type WorkflowModifier Uses

type WorkflowModifier func(*Workflow)

WorkflowModifier is a function type for functions that can modify a Workflow object.


cliDaisy is a GCE workflow tool.
computePackage compute provides access to the Google Compute API.

Package daisy imports 29 packages (graph) and is imported by 22 packages. Updated 2020-01-25. Refresh now. Tools for package owners.