build

package
v0.1.23 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 2, 2023 License: GPL-3.0 Imports: 11 Imported by: 0

README

build-a-build

Build pipeline automation which generates GitHub Actions workflow definitions to run itself.

See the example pipeline config input and the workflow definitions output.

Usage

# generate example
go run ./cmd/build generate pkg/build/example/workflow.yaml --config pkg/build/example/pipeline.yaml

# use in other repos
go run github.com/itura/fun/cmd/build generate ./.github/workflows/ci-cd.yaml

Concepts

  • Artifact: set of Docker images defined by multiple targets in a single Dockerfile
    • "verify" image used to test & lint source code
    • "app" image which runs the process defined by the source code
    • build-artifact command
      1. builds verify image
      2. runs verify image (terminate if failure)
      3. builds app image
      4. tags and pushes app image
  • Application: Helm chart or Terraform config
    • can be dependent on Artifacts or other Applications
    • deploy-application command either:
      1. runs helm update --install, setting values defined in config as well as repo and tag values for Artifact images.
      2. runs terraform apply
  • Pipeline: set of Artifact and Application definitions
    • commands for both Artifacts and Applications will run in parallel based on dependencies using GH Actions job dependencies

Features

Helm value injection

For an Application, you can specify values to be set for the helm upgrade. This can take 3 forms:

# pipeline.yaml
applications:
  - id: api
    path: helm/app
    values:
    # static value
    - key: app.name
      value: cool-api
    # load value from GH Actions `env`. 
    - key: app.lifecycle
      envValue: LIFECYCLE
    # load value from GH Actions `secret`
    - key: postgresql.auth.postgresPassword
      secretValue: DB_PASSWORD

Environment variables and secrets referenced by values must be populated separately.

Change detection

Commands for each Artifact and Application will only perform their full actions if associated paths have changed since the previous commit.

If an Artifact's source has not changed, the image will not be built, but a new tag for the current commit will be added to the previous image. This makes it so that Applications can use the same tag for all Artifacts in that build.

A current limitation/bug is that when pushing multiple commits at once, change detection will only operate on the most recent 2 commits .

Prerequisites

  • GCP
  • Workload identity for SA link
  • Artifact Registry API enabled
  • GKE cluster
  • GitHub Actions
    • envs:
      • PROJECT_ID
      • WORKLOAD_IDENTITY_PROVIDER
      • SERVICE_ACCOUNT

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	CloudProviderTypeEnum = NewEnum[CloudProviderType](map[CloudProviderType]string{
		cloudProviderTypeGcp: "gcp",
	})
)
View Source
var (
	SecretProviderTypeEnum = NewEnum[SecretProviderType](map[SecretProviderType]string{
		secretProviderTypeGcp:    "gcp",
		secretProviderTypeGithub: "github-actions",
	})
)

Functions

func CloudProviderMissingField added in v0.1.20

func CloudProviderMissingField(t string) error

func CreateApplications added in v0.1.20

func CreateApplications(args ActionArgs, previousSha string, config PipelineConfigRaw, artifacts map[string]Artifact, artifactRepository string) (map[string]Application, error)

func CreateArtifacts added in v0.1.20

func CreateArtifacts(args ActionArgs, previousSha string, config PipelineConfigRaw, artifactRepository string) map[string]Artifact

func GetDeployRunCommand added in v0.1.20

func GetDeployRunCommand(applicationId string, cmd string, configPath string) string

func Run

func Run() int

Types

type ActionArgs

type ActionArgs struct {
	CommonArgs
	Id         string `arg:"positional,required"`
	CurrentSha string `arg:"--current-sha,required" help:"current git sha, used for change detection"`
	Force      bool   `arg:"--force" help:"Ignore change detection"`
}

func TestArgs added in v0.1.20

func TestArgs(configPath string) ActionArgs

func (ActionArgs) CreatePipeline

func (a ActionArgs) CreatePipeline() (Pipeline, error)

type Application

type Application struct {
	Id                string
	Path              string
	Repository        string
	KubernetesCluster ClusterConfig
	CurrentSha        string
	Namespace         string
	Values            []HelmValue
	Upstreams         []Job
	Type              ApplicationType
	Secrets           map[string][]HelmSecretValue
	SecretProviders   map[string]SecretProvider

	CloudProvider CloudProviderConfig
	// contains filtered or unexported fields
}

func PostgresHelmChart added in v0.1.20

func PostgresHelmChart(builder TestBuilder, upstreams ...Job) Application

func TerraformConfig added in v0.1.23

func TerraformConfig(builder TestBuilder, id string, path string) Application

func WebsiteHelmChart added in v0.1.23

func WebsiteHelmChart(builder TestBuilder, upstreams ...Job) Application

func (Application) AddValue added in v0.1.20

func (a Application) AddValue(key, value string) Application

func (Application) GetSteps added in v0.1.20

func (a Application) GetSteps(cmd string, configPath string) []GitHubActionsStep

func (Application) HasDependencies

func (a Application) HasDependencies() bool

func (Application) JobId

func (a Application) JobId() string

func (Application) PrepareBuild

func (a Application) PrepareBuild() Build

func (Application) ResolveSecrets added in v0.1.20

func (a Application) ResolveSecrets() map[string]string

func (Application) SetNamespace added in v0.1.23

func (a Application) SetNamespace(namespace string) Application

func (Application) SetSecret added in v0.1.20

func (a Application) SetSecret(key, provider, name string) Application

func (Application) ToGitHubActionsJob added in v0.1.20

func (a Application) ToGitHubActionsJob(cmd string, configPath string) GitHubActionsJob

type ApplicationType added in v0.1.12

type ApplicationType string

type Artifact

type Artifact struct {
	Id         string
	Path       string
	Repository string
	Host       string
	CurrentSha string
	Type       ArtifactType
	Upstreams  []Job

	CloudProvider CloudProviderConfig
	// contains filtered or unexported fields
}

func (Artifact) AppImageBase

func (a Artifact) AppImageBase() string

func (Artifact) AppImageName

func (a Artifact) AppImageName(tag string) string

func (Artifact) AppTarget

func (a Artifact) AppTarget() string

func (Artifact) GetSteps added in v0.1.20

func (a Artifact) GetSteps(cmd string, configPath string) []GitHubActionsStep

func (Artifact) GreenTag

func (a Artifact) GreenTag() string

func (Artifact) HasDependencies

func (a Artifact) HasDependencies() bool

func (Artifact) JobId

func (a Artifact) JobId() string

func (Artifact) PrepareBuild

func (a Artifact) PrepareBuild() (Build, error)

func (Artifact) ToGitHubActionsJob added in v0.1.20

func (a Artifact) ToGitHubActionsJob(cmd string, configPath string) GitHubActionsJob

func (Artifact) VerifyImageName

func (a Artifact) VerifyImageName() string

func (Artifact) VerifyTarget

func (a Artifact) VerifyTarget() string

type ArtifactRepository added in v0.1.20

type ArtifactRepository struct {
	Host string `validate:"required"`
	Name string `validate:"required"`
}

func (ArtifactRepository) Validate added in v0.1.20

func (a ArtifactRepository) Validate(key string) ValidationErrors

type ArtifactType

type ArtifactType string

type Build

type Build interface {
	Build() (SideEffects, error)
}

type BuildArtifactCommand

type BuildArtifactCommand struct {
	ActionArgs
}

func (BuildArtifactCommand) Run

func (c BuildArtifactCommand) Run() error

type ChangeDetection

type ChangeDetection interface {
	HasChanged() bool
}

func NewAlwaysChanged added in v0.1.9

func NewAlwaysChanged() ChangeDetection

func NewNeverChanged added in v0.1.9

func NewNeverChanged() ChangeDetection

type CloudProvider added in v0.1.20

type CloudProvider interface {
	AuthStep() GitHubActionsStep
	Validate(key string) ValidationErrors
}

type CloudProviderConfig added in v0.1.20

type CloudProviderConfig struct {
	Type   CloudProviderType
	Config map[string]string
}

func (CloudProviderConfig) Impl added in v0.1.20

func (CloudProviderConfig) Validate added in v0.1.20

type CloudProviderType added in v0.1.20

type CloudProviderType uint

func (*CloudProviderType) UnmarshalYAML added in v0.1.20

func (s *CloudProviderType) UnmarshalYAML(
	unmarshal func(interface{}) error,
) error

type ClusterConfig added in v0.1.17

type ClusterConfig struct {
	Name     string `validate:"required"`
	Location string `validate:"required"`
}

func (ClusterConfig) Validate added in v0.1.20

func (c ClusterConfig) Validate(key string) ValidationErrors

type Command

type Command struct {
	Name      string
	Arguments []string
}

func NewCommand added in v0.1.23

func NewCommand(name string, args ...string) Command

type CommandRunner added in v0.1.23

type CommandRunner interface {
	Run(name string, args ...string) error
	RunSilent(name string, args ...string) error
}

type CommonArgs

type CommonArgs struct {
	ConfigPath string `arg:"--config" default:"pipeline.yaml" help:"path to pipeline definition yaml"`
	Self       bool   `arg:"--self" help:"Run the tool in its own source repo."`
}

type DeployApplicationCommand

type DeployApplicationCommand struct {
	ActionArgs
}

func (DeployApplicationCommand) Run

type DockerImage

type DockerImage struct {
	PackageVerifier
}

func NewDockerImage

func NewDockerImage(a Artifact) DockerImage

func (DockerImage) Build

func (b DockerImage) Build() (SideEffects, error)

type Enum added in v0.1.20

type Enum[T comparable] struct {
	// contains filtered or unexported fields
}

func NewEnum added in v0.1.20

func NewEnum[T comparable](keyToValue map[T]string) Enum[T]

func (Enum[T]) FromString added in v0.1.20

func (e Enum[T]) FromString(value string) (T, bool)

func (Enum[T]) InvalidEnumValue added in v0.1.20

func (e Enum[T]) InvalidEnumValue(value string) error

func (Enum[T]) Marshal added in v0.1.20

func (e Enum[T]) Marshal(key T) interface{}

func (Enum[T]) ToString added in v0.1.20

func (e Enum[T]) ToString(key T) (string, bool)

func (Enum[T]) Unmarshal added in v0.1.20

func (e Enum[T]) Unmarshal(
	unmarshal func(interface{}) error,
	result *T,
) error

type GCP added in v0.1.20

type GCP struct {
	Config map[string]string
}

func (GCP) AuthStep added in v0.1.20

func (g GCP) AuthStep() GitHubActionsStep

func (GCP) Type added in v0.1.20

func (g GCP) Type() string

func (GCP) Validate added in v0.1.20

func (g GCP) Validate(key string) ValidationErrors

type GenerateArgs

type GenerateArgs struct {
	CommonArgs
	OutputPath string `arg:"positional,required" help:"path to write generated Github Actions definition to"`
}

func (GenerateArgs) CreatePipeline

func (g GenerateArgs) CreatePipeline() (Pipeline, error)

type GenerateCommand

type GenerateCommand struct {
	GenerateArgs
}

func (GenerateCommand) Run

func (c GenerateCommand) Run() error

type GitChangeDetection

type GitChangeDetection struct {
	// contains filtered or unexported fields
}

func NewGitChangeDetection

func NewGitChangeDetection(previousSha string) GitChangeDetection

func (GitChangeDetection) AddPaths

func (g GitChangeDetection) AddPaths(paths ...string) GitChangeDetection

func (GitChangeDetection) HasChanged

func (g GitChangeDetection) HasChanged() bool

type GitHubActionsJob added in v0.1.20

type GitHubActionsJob struct {
	Name        string
	RunsOn      string `yaml:"runs-on"`
	Permissions map[string]string
	Needs       []string `yaml:"needs,omitempty"`
	Steps       []GitHubActionsStep
}

type GitHubActionsStep added in v0.1.20

type GitHubActionsStep struct {
	Id   string                 `yaml:"id,omitempty"`
	Name string                 `yaml:"name,omitempty"`
	Uses string                 `yaml:"uses,omitempty"`
	With map[string]interface{} `yaml:"with,omitempty"`
	Env  map[string]string      `yaml:"env,omitempty"`
	Run  string                 `yaml:"run,omitempty"`
}

func GetDeployStep added in v0.1.20

func GetDeployStep(applicationId string, values []HelmValue, resolvedSecrets map[string]string, runCommand string) GitHubActionsStep

func GetHelmSteps added in v0.1.20

func GetHelmSteps(cluster ClusterConfig) []GitHubActionsStep

func GetTerraformSteps added in v0.1.20

func GetTerraformSteps() []GitHubActionsStep

type GitHubActionsTriggerEvent added in v0.1.20

type GitHubActionsTriggerEvent struct {
	Branches []string
}

type GitHubActionsWorkflow added in v0.1.20

func (GitHubActionsWorkflow) WriteYaml added in v0.1.20

func (g GitHubActionsWorkflow) WriteYaml(path string) error

type HelmDeployment

type HelmDeployment struct {
	Application
}

func NewHelm added in v0.1.12

func NewHelm(a Application) HelmDeployment

func (HelmDeployment) Build

func (b HelmDeployment) Build() (SideEffects, error)

type HelmSecretValue added in v0.1.20

type HelmSecretValue struct {
	HelmKey    string
	SecretName string
}

type HelmValue

type HelmValue struct {
	Key   string
	Value string
}

type InvalidCloudProvider added in v0.1.20

type InvalidCloudProvider struct{ Message string }

func (InvalidCloudProvider) Error added in v0.1.20

func (m InvalidCloudProvider) Error() string

type Job

type Job interface {
	JobId() string
}

type MissingSecretProvider added in v0.1.20

type MissingSecretProvider struct{}

func (MissingSecretProvider) Error added in v0.1.20

func (m MissingSecretProvider) Error() string

type NullBuild

type NullBuild struct{}

func (NullBuild) Build

func (b NullBuild) Build() (SideEffects, error)

type PackageVerifier

type PackageVerifier struct {
	Artifact
	// contains filtered or unexported fields
}

func NewPackageVerifier

func NewPackageVerifier(a Artifact) PackageVerifier

func (PackageVerifier) Build

func (b PackageVerifier) Build() (SideEffects, error)

type Pipeline

type Pipeline struct {
	ConfigPath string
	Name       string
	Cmd        string
	// contains filtered or unexported fields
}

func NewPipeline

func NewPipeline(result PipelineConfig, configPath string, _cmd string) Pipeline

func ParsePipeline added in v0.1.20

func ParsePipeline(args ActionArgs, previousSha string) (Pipeline, error)

func (Pipeline) BuildArtifact

func (p Pipeline) BuildArtifact(id string) (SideEffects, error)

func (Pipeline) DeployApplication

func (p Pipeline) DeployApplication(id string) (SideEffects, error)

func (Pipeline) ToGitHubWorkflow added in v0.1.20

func (p Pipeline) ToGitHubWorkflow() GitHubActionsWorkflow

type PipelineCommand

type PipelineCommand interface {
	Run() error
}

type PipelineConfig

type PipelineConfig struct {
	Artifacts    fun.Config[Artifact]
	Applications fun.Config[Application]
	BuildName    string
	Error        error
}

func FailedParse added in v0.1.20

func FailedParse(name string, err error) PipelineConfig

func NewParsedConfig added in v0.1.20

func NewParsedConfig() PipelineConfig

func SuccessfulParse added in v0.1.20

func SuccessfulParse(name string, artifacts fun.Config[Artifact], applications fun.Config[Application]) PipelineConfig

func ValidPipelineConfig added in v0.1.23

func ValidPipelineConfig(builder TestBuilder) PipelineConfig

func (PipelineConfig) ListApplications added in v0.1.23

func (o PipelineConfig) ListApplications() []Application

func (PipelineConfig) ListArtifacts added in v0.1.23

func (o PipelineConfig) ListArtifacts() []Artifact

func (PipelineConfig) SetApplications added in v0.1.23

func (o PipelineConfig) SetApplications(applications fun.Config[Application]) PipelineConfig

func (PipelineConfig) SetArtifacts added in v0.1.23

func (o PipelineConfig) SetArtifacts(artifacts fun.Config[Artifact]) PipelineConfig

func (PipelineConfig) SetBuildName added in v0.1.23

func (o PipelineConfig) SetBuildName(name string) PipelineConfig

func (PipelineConfig) SetError added in v0.1.23

func (o PipelineConfig) SetError(err error) PipelineConfig

type PipelineConfigRaw added in v0.1.23

type PipelineConfigRaw struct {
	Name      string    `validate:"required"`
	Resources Resources `validate:"required"`
	Artifacts []struct {
		Id           string
		Path         string
		Dependencies []string
		Type         ArtifactType
	}
	Applications []struct {
		Id           string
		Path         string
		Namespace    string
		Artifacts    []string
		Values       []HelmValue
		Secrets      []SecretConfig
		Dependencies []string
		Type         ApplicationType
	}
}

func (PipelineConfigRaw) Validate added in v0.1.23

func (p PipelineConfigRaw) Validate(key string) ValidationErrors

type Resources added in v0.1.20

type Resources struct {
	ArtifactRepository ArtifactRepository  `yaml:"artifactRepository" validate:"required"`
	KubernetesCluster  ClusterConfig       `yaml:"kubernetesCluster"  validate:"required"`
	SecretProviders    SecretProviders     `yaml:"secretProviders"    validate:"required"`
	CloudProvider      CloudProviderConfig `yaml:"cloudProvider"      validate:"required"`
}

func (Resources) Validate added in v0.1.20

func (r Resources) Validate(key string) ValidationErrors

type SecretConfig added in v0.1.20

type SecretConfig struct {
	HelmKey    string "yaml:\"helmKey\""
	SecretName string "yaml:\"secretName\""
	Provider   string
}

type SecretProvider added in v0.1.20

type SecretProvider struct {
	Type   SecretProviderType `validate:"required"`
	Config map[string]string
}

func (SecretProvider) Validate added in v0.1.20

func (s SecretProvider) Validate(key string) ValidationErrors

type SecretProviderType added in v0.1.20

type SecretProviderType uint

func (*SecretProviderType) UnmarshalYAML added in v0.1.20

func (s *SecretProviderType) UnmarshalYAML(
	unmarshal func(interface{}) error,
) error

type SecretProviders added in v0.1.20

type SecretProviders fun.Config[SecretProvider]

func (SecretProviders) Validate added in v0.1.20

func (s SecretProviders) Validate(key string) ValidationErrors

type Secrets added in v0.1.23

type Secrets struct {
	Secrets         map[string][]HelmSecretValue
	SecretProviders map[string]SecretProvider
}

func (Secrets) Resolve added in v0.1.23

func (b Secrets) Resolve() map[string]string

func (Secrets) SercetsToCLIArgs added in v0.1.23

func (b Secrets) SercetsToCLIArgs() []string

func (Secrets) ToGitHubActionsSteps added in v0.1.23

func (b Secrets) ToGitHubActionsSteps() []GitHubActionsStep

type ShellCommandRunner added in v0.1.23

type ShellCommandRunner struct{}

func (ShellCommandRunner) Run added in v0.1.23

func (c ShellCommandRunner) Run(name string, args ...string) error

func (ShellCommandRunner) RunSilent added in v0.1.23

func (c ShellCommandRunner) RunSilent(name string, args ...string) error

type SideEffects added in v0.1.23

type SideEffects struct {
	Commands []Command
	Runner   CommandRunner
}

func (SideEffects) Add added in v0.1.23

func (s SideEffects) Add(commands ...Command) SideEffects

func (SideEffects) Apply added in v0.1.23

func (s SideEffects) Apply() error

type StaticChangeDetection added in v0.1.9

type StaticChangeDetection struct {
	// contains filtered or unexported fields
}

func (StaticChangeDetection) HasChanged added in v0.1.9

func (c StaticChangeDetection) HasChanged() bool

type TestBuilder added in v0.1.20

type TestBuilder struct {
	// contains filtered or unexported fields
}

func NewTestBuilder added in v0.1.20

func NewTestBuilder(currentSha string) TestBuilder

func (TestBuilder) Application added in v0.1.20

func (b TestBuilder) Application(id string, path string, appType ApplicationType, upstreams ...Job) Application

func (TestBuilder) Artifact added in v0.1.20

func (b TestBuilder) Artifact(id string, path string, upstreams ...Job) Artifact

type TfConfig added in v0.1.12

type TfConfig struct {
	Application
}

func NewTerraform added in v0.1.12

func NewTerraform(a Application) TfConfig

func (TfConfig) Build added in v0.1.12

func (b TfConfig) Build() (SideEffects, error)

type ValidationErrors added in v0.1.20

type ValidationErrors struct {
	// contains filtered or unexported fields
}

func NewValidationErrors added in v0.1.20

func NewValidationErrors(key string) ValidationErrors

func (ValidationErrors) Error added in v0.1.20

func (v ValidationErrors) Error() string

func (ValidationErrors) IsPresent added in v0.1.20

func (v ValidationErrors) IsPresent() bool

func (ValidationErrors) Put added in v0.1.20

func (ValidationErrors) PutChild added in v0.1.20

func (ValidationErrors) Validate added in v0.1.20

func (v ValidationErrors) Validate(parent interface{}) ValidationErrors

func (ValidationErrors) ValidateNested added in v0.1.20

func (v ValidationErrors) ValidateNested(parent interface{}) ValidationErrors

func (ValidationErrors) ValidateTags added in v0.1.20

func (v ValidationErrors) ValidateTags(parent interface{}) ValidationErrors

type Validator added in v0.1.20

type Validator interface {
	Validate(key string) ValidationErrors
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL