vignet

package module
v0.5.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 5, 2024 License: MIT Imports: 27 Imported by: 0

README

Vignet

The missing GitOps piece: expose a Git repository behind an authenticated API to perform updates with authorization.

Where does it fit in?

GitOps tools already handle many aspects of syncing infrastructure resources (e.g. in Kubernetes) from Git repositories. To fully integrate into the delivery workflow, updates to the Git repo should be able to be performed via an API for automation (e.g. set a new image tag for a release). Since a Git repo can store the complete infrastructure, updates should be protected - an application pipeline should only be allowed to update its own declaration.

This is why we created Vignet:

  • It runs as a standalone service in your infrastructure
  • It will get access to your GitOps repositories
  • It exposes an authenticated Rest API for patching YAML declarations via commands
  • It integrates flexible authorization via OPA (Open Policy Agent) rules to decide if a command should be allowed
  • It is easy to integrate into GitLab CI, GitHub Actions and other systems
  • Works perfectly with Flux, ArgoCD or other GitOps tools

Design principles

  • Vignet is stateless, repositories and authorization are configured via configuration files
  • Policies are customizable via Open Policy Agent (OPA) rules

Current state

It is in the early stages of development, but it should already be usable for integration in GitLab CI. Configuration and API is subject to change. Use in production at your own risk.

Command reference

NAME:
   vignet - The missing GitOps piece: expose Git repositories for automation via an authenticated HTTP API

USAGE:
   vignet [global options] command [command options] [arguments...]

DESCRIPTION:
   The default command starts an HTTP server that handles commands.

COMMANDS:
   help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h  show help (default: false)

   authorization
   --policy value  Path to an OPA policy bundle path, uses the built-in by default [$VIGNET_POLICY]

   configuration
   --config value, -c value  Path to the configuration file (default: "config.yaml") [$VIGNET_CONFIG]

   http
   --address value  Address for HTTP server to listen on (default: ":8080") [$VIGNET_ADDRESS]

   logging
   --force-logfmt  Force logging to use logfmt (default: false) [$VIGNET_FORCE_LOGFMT]
   --verbose       Enable verbose logging (default: false) [$VIGNET_VERBOSE]

Configuration

Vignet is configured via flags or env vars and a YAML configuration file. The configuration file makes it easier to manage multiple repository configurations. A custom Open Policy Agent policy bundle can be used to customize authorization via the policy flag.

Example configuration
authenticationProvider:
  # Use a GitLab job token authentication provider
  type: gitlab

  # Configuration for the GitLab authentication provider
  gitlab:
    # URL to the GitLab instance
    url: https://gitlab.example.com

# Configure repositories that can be accessed by Vignet
repositories:
  # Repository name
  my-project:
    # URL to the repository
    url: https://gitlab.example.com/my-group/my-project.git
    basicAuth:
      # Username doesn't matter for GitLab
      username: gitlab
      # Use an access token with scopes "read_repository", "write_repository"
      password: an-access-token

commit:
  # Default message to use for a commit if none is specified in a request
  defaultMessage: "Automated update"
  # Default author to use for a commit if none is specified in a request
  defaultAuthor:
    name: Git autopilot
    email: bot@example.com

Rest API

POST /patch/{repository}

Pulls the repository, patches files according to commands, creates a commit and pushes to the repository.

Responds with status code 200 on success.

Body
  • commit object Commit options (optional)
    • message string Commit message (optional)
    • committer object Committer for the commit (optional)
      • name string
      • email string
    • author object Author for the commit (optional)
      • name string
      • email string
  • commands array Commands to perform, one of setField and n.n. must be set
    • path string Path to the file to patch (relative from repository root)
    • setField object Perform a set field command (optional)
      • field string Field to set with dot path syntax, JSONPath features are supported (see examples)
      • value mixed Value to set the field to
      • create boolean Create the field (and intermediate path) if it doesn't exist (optional, defaults to false)
    • createFile object Perform a create file command to create a new file (optional)
      • content string Content of the file to create
    • deleteFile object Perform a delete file command to delete a file (optional)
Examples
Setting a field in a YAML file
POST http://localhost:8080/patch/infra-test
Authorization: Bearer [CI_JOB_JWT]
Content-Type: application/json

{
  "commit": {
    "message": "Bump image to 1.2.5"
  },
  "commands": [
    {
      "path": "my-group/my-project/release.yml",
      "setField": {
        "field": "spec.values.image.tag",
        "value": "1.2.5"
      }
    }
  ]
}
Using JSONPath

JSONPath can be used to reference a field by array index, filter expression or other features:

POST http://localhost:8080/patch/infra-test
Authorization: Bearer [CI_JOB_JWT]
Content-Type: application/json

{
  "commit": {
    "message": "Bump image to 1.2.5, update BUILD_ID"
  },
  "commands": [
    {
      "path": "my-group/my-project/deployment.yml",
      "setField": {
        "field": "spec.template.spec.containers[0].image",
        "value": "registry.example.com/my/image:1.2.5"
      }
    },
    {
      "path": "my-group/my-project/deployment.yml",
      "setField": {
        "field": "spec.template.spec.containers[0].env[?(@.name == 'BUILD_ID')].value",
        "value": "987654"
      }
    }
  ]
}

Using Curl is a convenient way to integrate Vignet into GitLab CI:

curl -s --fail-with-body -H "Authorization: Bearer $CI_JOB_JWT" -H "Content-Type: application/json" -d @- http://localhost:8080/patch/infra-test << JSON
{
  "commit": {
    "message": "${CI_PROJECT_PATH}: Release ${CI_REGISTRY_TAG} to ${CI_ENVIRONMENT_SLUG}"
  },
  "commands": [
    {
      "path": "projects/${CI_PROJECT_PATH}/release-${CI_ENVIRONMENT_SLUG}.yaml",
      "setField": {
        "field": "spec.values.app.image.tag",
        "value": "$CI_REGISTRY_TAG"
      }
    }
  ]
}
JSON
Writing a new file
POST http://localhost:8080/patch/infra-test
Authorization: Bearer [CI_JOB_JWT]
Content-Type: application/json

{
  "commit": {
    "message": "Bump image to 1.2.5"
  },
  "commands": [
    {
      "path": "my-group/my-project/new.yml",
      "createFile": {
        "content": "---\nversion: 1.2.3\n"
      }
    }
  ]
}

Curl can be paired with jq to read an existing file and convert it to a JSON string:

# Prepare a YAML file for the new release before this command
yaml_file=new-release.yml

curl --fail-with-body -H "Authorization: Bearer $CI_JOB_JWT"  http://localhost:8080/patch/infra-test -H "Content-Type: application/json" -d \
@<(jq -n --arg yaml "$(jq -sR . $yaml_file)" "$(cat <<JSON
{
  "commit": {
    "message": "${CI_PROJECT_PATH}: Release ${CI_REGISTRY_TAG} to ${CI_ENVIRONMENT_SLUG}"
  },
  "commands": [
    {
      "path": "projects/${CI_PROJECT_PATH}/new-${CI_ENVIRONMENT_SLUG}.yaml",
      "createFile": {
        "content": \$yaml
      }
    }
  ]
}
JSON
)")

Authentication

GitLab
  • GitLab CI generates a job token env var CI_JOB_JWT for each job. It contains claims about the user, project and repository.
  • This token needs to be passed via Authorization: Bearer [CI_JOB_JWT] header to Vignet.
  • Requests are denied if the token is invalid or missing.
  • Claims in the token are passed to the authorization policy to check if the request should be allowed.

Authorization

Vignet will pass the authentication context and request information to the policy for decision.

Default policy
Patch request
  • path Accepts only .yml and .yaml files

The further policy behavior depends on the authentication provider:

GitLab

  • path Requires a prefix of the GitLab project path (of the job passing the job token).

    E.g. a job token with project_path: "my-group/my-project" will only authorize requests for my-group/my-project/**/*.{yml,yaml}.

Known limitations

  • Currently, only authentication via a GitLab job token is supported
  • There is only a setField command for now

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var DefaultConfig = Config{
	Commit: CommitConfig{
		DefaultMessage: "Automated patch by vignet",
		DefaultAuthor: SignatureConfig{
			Name:  "vignet",
			Email: "bot@vignet",
		},
	},
}

DefaultConfig is the default configuration that will be overwritten by the configuration file.

Functions

func AuthenticateRequest 

func AuthenticateRequest(authenticationProvider AuthenticationProvider) func(http.Handler) http.Handler

AuthenticateRequest is a middleware to set the AuthCtx from the given request on the request context.

Types

type AuthCtx 

type AuthCtx struct {
	// Error is set if the authentication failed.
	Error error `json:"error"`
	// GitLabClaims is set for GitLab authentication provider if no authenticated error occurred.
	GitLabClaims *GitLabClaims `json:"gitLabClaims"`
}

type AuthenticationProvider 

type AuthenticationProvider interface {
	// AuthCtxFromRequest builds an authentication context from the given requests.
	//
	// If a client error concerning the authentication is encountered or the request could not be authenticated, the error is set in AuthCtx.
	// If an internal error is encountered, the error is returned as error return value.
	AuthCtxFromRequest(r *http.Request) (AuthCtx, error)
}

type AuthenticationProviderType 

type AuthenticationProviderType string
const (
	AuthenticationProviderGitLab AuthenticationProviderType = "gitlab"
)

func (AuthenticationProviderType) IsValid 

func (p AuthenticationProviderType) IsValid() bool

type Authorizer 

type Authorizer interface {
	AllowPatch(ctx context.Context, authCtx AuthCtx, repo string, req patchRequest) error
}

type BasicAuthConfig 

type BasicAuthConfig struct {
	Username string `yaml:"username"`
	Password string `yaml:"password"`
}

type CommitConfig 

type CommitConfig struct {
	DefaultMessage string          `yaml:"defaultMessage"`
	DefaultAuthor  SignatureConfig `yaml:"defaultAuthor"`
}

type Config 

type Config struct {
	// AuthenticationProvider configures the authentication provider to use for authenticating requests.
	AuthenticationProvider struct {
		Type AuthenticationProviderType `yaml:"type"`
		// GitLab must be set for type `gitlab`
		GitLab *struct {
			URL string `yaml:"url"`
		} `yaml:"gitlab"`
	} `yaml:"authenticationProvider"`

	// Repositories indexed by an identifier.
	Repositories RepositoriesConfig `yaml:"repositories"`

	// Commit configures commit options when creating a new commit.
	Commit CommitConfig `yaml:"commit"`
}

func (Config) BuildAuthenticationProvider 

func (c Config) BuildAuthenticationProvider(ctx context.Context) (AuthenticationProvider, error)

func (Config) Validate 

func (c Config) Validate() error

type GitLabAuthenticationProvider 

type GitLabAuthenticationProvider struct {
	// contains filtered or unexported fields
}

func NewGitLabAuthenticationProvider 

func NewGitLabAuthenticationProvider(ctx context.Context, url string) (*GitLabAuthenticationProvider, error)

NewGitLabAuthenticationProvider creates a new GitLabAuthenticationProvider.

It takes the GitLab instance URL as an argument. The context is used to cancel the refreshing of keys.

func (*GitLabAuthenticationProvider) AuthCtxFromRequest 

func (p *GitLabAuthenticationProvider) AuthCtxFromRequest(r *http.Request) (AuthCtx, error)

type GitLabClaims 

type GitLabClaims struct {
	jwt.RegisteredClaims

	NamespaceID    string `json:"namespace_id"`
	NamespacePath  string `json:"namespace_path"`
	ProjectID      string `json:"project_id"`
	ProjectPath    string `json:"project_path"`
	UserID         string `json:"user_id"`
	UserLogin      string `json:"user_login"`
	UserEmail      string `json:"user_email"`
	PipelineID     string `json:"pipeline_id"`
	PipelineSource string `json:"pipeline_source"`
	JobID          string `json:"job_id"`
	Ref            string `json:"ref"`
	RefType        string `json:"ref_type"`
	RefProtected   string `json:"ref_protected"`
}

type Handler 

type Handler struct {
	// contains filtered or unexported fields
}

func NewHandler 

func NewHandler(
	authenticationProvider AuthenticationProvider,
	authorizer Authorizer,
	config Config,
) *Handler

func (*Handler) ServeHTTP 

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

type RegoAuthorizer 

type RegoAuthorizer struct {
	// contains filtered or unexported fields
}

func NewRegoAuthorizer 

func NewRegoAuthorizer(ctx context.Context, bundle *bundle.Bundle) (*RegoAuthorizer, error)

func (*RegoAuthorizer) AllowPatch 

func (r *RegoAuthorizer) AllowPatch(ctx context.Context, authCtx AuthCtx, repo string, req patchRequest) error

type RepositoriesConfig 

type RepositoriesConfig map[string]RepositoryConfig

type RepositoryConfig 

type RepositoryConfig struct {
	URL       string           `yaml:"url"`
	BasicAuth *BasicAuthConfig `yaml:"basicAuth"`
}

type SignatureConfig 

type SignatureConfig struct {
	Name  string `yaml:"name"`
	Email string `yaml:"email"`
}

func (SignatureConfig) Valid 

func (c SignatureConfig) Valid() error

type ViolationsResolver 

type ViolationsResolver interface {
	Violations() []string
}

Source Files

View all Source files

Directories

Path Synopsis
header
Package header provides functions for parsing HTTP headers.
 Package header provides functions for parsing HTTP headers.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.