keppel

package
v0.0.0-...-479019f Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 6, 2024 License: Apache-2.0 Imports: 50 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var AuthDriverRegistry pluggable.Registry[AuthDriver]

AuthDriverRegistry is a pluggable.Registry for AuthDriver implementations.

View Source
var ErrAuthDriverMismatch = errors.New("given AuthDriver is not supported by this driver")

ErrAuthDriverMismatch is returned by Init() methods on most driver interfaces, to indicate that the driver in question does not work with the selected AuthDriver.

View Source
var ErrCannotGenerateURL = errors.New("URLForBlob() is not supported")

ErrCannotGenerateURL is returned by StorageDriver.URLForBlob() when the StorageDriver does not support blob URLs.

View Source
var (
	ErrIncompatibleReplicationPolicy = errors.New("cannot change replication policy on existing account")
)
View Source
var ErrNoSuchPrimaryAccount = errors.New("no such primary account")

ErrNoSuchPrimaryAccount is returned by FederationDriver.FindPrimaryAccount if no peer has the given primary account.

View Source
var FederationDriverRegistry pluggable.Registry[FederationDriver]

FederationDriverRegistry is a pluggable.Registry for FederationDriver implementations.

View Source
var InboundCacheDriverRegistry pluggable.Registry[InboundCacheDriver]

InboundCacheDriverRegistry is a pluggable.Registry for InboundCacheDriver implementations.

View Source
var RateLimitDriverRegistry pluggable.Registry[RateLimitDriver]

RateLimitDriverRegistry is a pluggable.Registry for RateLimitDriver implementations.

View Source
var StorageDriverRegistry pluggable.Registry[StorageDriver]

StorageDriverRegistry is a pluggable.Registry for StorageDriver implementations.

View Source
var UserIdentityRegistry pluggable.Registry[UserIdentity]

UserIdentityRegistry is a pluggable.Registry for UserIdentity implementations.

Functions

func AppendQuery

func AppendQuery(urlStr string, query url.Values) string

AppendQuery adds additional query parameters to an existing unparsed URL.

func BuildBasicAuthHeader

func BuildBasicAuthHeader(userName, password string) string

BuildBasicAuthHeader constructs the value of an "Authorization" HTTP header for the given basic auth credentials.

func CompressTokenPayload

func CompressTokenPayload(payload []byte) ([]byte, error)

CompressTokenPayload can be used by types implementing the UserIdentity interface to compress large token payloads with GZip or similar. (The exact compression format is an implementation detail.) The result is a valid JSON message that self-documents the compression algorithm that was used.

func DecompressTokenPayload

func DecompressTokenPayload(payload []byte) ([]byte, error)

DecompressTokenPayload is the exact reverse of CompressTokenPayload.

func FindAccount

func FindAccount(db gorp.SqlExecutor, name string) (*models.Account, error)

FindAccount works similar to db.SelectOne(), but returns nil instead of sql.ErrNoRows if no account exists with this name.

func FindBlobByAccountName

func FindBlobByAccountName(db gorp.SqlExecutor, blobDigest digest.Digest, account models.Account) (*models.Blob, error)

FindBlobByAccountName is a convenience wrapper around db.SelectOne(). If the blob in question does not exist, sql.ErrNoRows is returned.

func FindBlobByRepository

func FindBlobByRepository(db gorp.SqlExecutor, blobDigest digest.Digest, repo models.Repository) (*models.Blob, error)

FindBlobByRepository is a convenience wrapper around db.SelectOne(). If the blob in question does not exist, sql.ErrNoRows is returned.

func FindBlobByRepositoryName

func FindBlobByRepositoryName(db gorp.SqlExecutor, blobDigest digest.Digest, repoName string, account models.Account) (*models.Blob, error)

FindBlobByRepositoryName is a convenience wrapper around db.SelectOne(). If the blob in question does not exist, sql.ErrNoRows is returned.

func FindManifest

func FindManifest(db gorp.SqlExecutor, repo models.Repository, manifestDigest digest.Digest) (*models.Manifest, error)

FindManifest is a convenience wrapper around db.SelectOne(). If the manifest in question does not exist, sql.ErrNoRows is returned.

func FindManifestByRepositoryName

func FindManifestByRepositoryName(db gorp.SqlExecutor, repoName string, account models.Account, manifestDigest digest.Digest) (*models.Manifest, error)

FindManifestByRepositoryName is a convenience wrapper around db.SelectOne(). If the manifest in question does not exist, sql.ErrNoRows is returned.

func FindOrCreateRepository

func FindOrCreateRepository(db gorp.SqlExecutor, name string, account models.Account) (*models.Repository, error)

FindOrCreateRepository works similar to db.SelectOne(), but autovivifies a Repository record when none exists yet.

func FindQuotas

func FindQuotas(db gorp.SqlExecutor, authTenantID string) (*models.Quotas, error)

FindQuotas works similar to db.SelectOne(), but returns nil instead of sql.ErrNoRows if no quota set exists for this auth tenant.

func FindRepository

func FindRepository(db gorp.SqlExecutor, name string, account models.Account) (*models.Repository, error)

FindRepository is a convenience wrapper around db.SelectOne(). If the repository in question does not exist, sql.ErrNoRows is returned.

func FindRepositoryByID

func FindRepositoryByID(db gorp.SqlExecutor, id int64) (*models.Repository, error)

FindRepositoryByID is a convenience wrapper around db.SelectOne(). If the repository in question does not exist, sql.ErrNoRows is returned.

func FindUploadByRepository

func FindUploadByRepository(db gorp.SqlExecutor, uuid string, repo models.Repository) (*models.Upload, error)

FindUploadByRepository is a convenience wrapper around db.SelectOne(). If the upload in question does not exist, sql.ErrNoRows is returned.

func GenerateStorageID

func GenerateStorageID() string

GenerateStorageID generates a new random storage ID for use with keppel.StorageDriver.AppendToBlob().

func GetManifestUsage

func GetManifestUsage(db gorp.SqlExecutor, quotas models.Quotas) (uint64, error)

GetManifestUsage returns how many manifests currently exist in repos in accounts connected to this quota set's auth tenant.

func GetPeerFromAccount

func GetPeerFromAccount(db gorp.SqlExecutor, account models.Account) (models.Peer, error)

GetPeerFromAccount returns the peer of the account given.

Returns sql.ErrNoRows if the configured peer does not exist.

func GetRedisOptions

func GetRedisOptions(prefix string) (*redis.Options, error)

GetRedisOptions returns a redis.Options by getting the required parameters from environment variables:

REDIS_PASSWORD, REDIS_HOSTNAME, REDIS_PORT, and REDIS_DB_NUM.

The environment variable keys are prefixed with the provided prefix.

func GetSecurityInfo

func GetSecurityInfo(db gorp.SqlExecutor, repoID int64, manifestDigest digest.Digest) (*models.TrivySecurityInfo, error)

func MaxMaybeTime

func MaxMaybeTime(t1, t2 *time.Time) *time.Time

func MaybeTimeToUnix

func MaybeTimeToUnix(t *time.Time) *int64

MaybeTimeToUnix casts a time.Time instance into its UNIX timestamp while preserving nil-ness.

func MinMaybeTime

func MinMaybeTime(t1, t2 *time.Time) *time.Time

func MountBlobIntoRepo

func MountBlobIntoRepo(db gorp.SqlExecutor, blob models.Blob, repo models.Repository) error

MountBlobIntoRepo creates an entry in the blob_mounts database table.

func OriginalRequestURL

func OriginalRequestURL(r *http.Request) url.URL

OriginalRequestURL returns the URL that the original requester used when sending an HTTP request. This inspects the X-Forwarded-* set of headers to identify reverse proxying.

func ParseIssuerKey

func ParseIssuerKey(in string) (crypto.PrivateKey, error)

ParseIssuerKey parses the contents of the KEPPEL_ISSUER_KEY variable.

func SetTaskName

func SetTaskName(taskName string)

func SetupHTTPClient

func SetupHTTPClient()

Types

type Account

type Account struct {
	Name              string                `json:"name"`
	AuthTenantID      string                `json:"auth_tenant_id"`
	GCPolicies        []GCPolicy            `json:"gc_policies,omitempty"`
	InMaintenance     bool                  `json:"in_maintenance"`
	Metadata          map[string]string     `json:"metadata"`
	RBACPolicies      []RBACPolicy          `json:"rbac_policies"`
	ReplicationPolicy *ReplicationPolicy    `json:"replication,omitempty"`
	ValidationPolicy  *ValidationPolicy     `json:"validation,omitempty"`
	PlatformFilter    models.PlatformFilter `json:"platform_filter,omitempty"`
}

Account represents an account in the API.

func RenderAccount

func RenderAccount(dbAccount models.Account) (Account, error)

RenderAccount converts an account model from the DB into the API representation.

type AuditContext

type AuditContext struct {
	UserIdentity UserIdentity
	Request      *http.Request
}

AuditContext collects arguments that business logic methods need only for generating audit events.

type Auditor

type Auditor interface {
	// Record forwards the given audit event to the audit log.
	// EventParameters.Observer will be filled by the auditor.
	Record(params audittools.EventParameters)
}

Auditor is a component that forwards audit events to the appropriate logs. It is used by some of the API modules.

func InitAuditTrail

func InitAuditTrail() Auditor

InitAuditTrail initializes a Auditor from the configuration variables found in the environment.

type AuthDriver

type AuthDriver interface {
	pluggable.Plugin
	// Init is called before any other interface methods, and allows the plugin to
	// perform first-time initialization. The supplied *redis.Client can be stored
	// for caching authorizations, but only if it is non-nil.
	Init(*redis.Client) error

	// AuthenticateUser authenticates the user identified by the given username
	// and password. Note that usernames may not contain colons, because
	// credentials are encoded by clients in the "username:password" format.
	AuthenticateUser(ctx context.Context, userName, password string) (UserIdentity, *RegistryV2Error)
	// AuthenticateUserFromRequest reads credentials from the given incoming HTTP
	// request to authenticate the user which makes this request. The
	// implementation shall follow the conventions of the concrete backend, e.g. a
	// OAuth backend could try to read a Bearer token from the Authorization
	// header, whereas an OpenStack auth driver would look for a Keystone token in the
	// X-Auth-Token header.
	//
	// If the request contains no auth headers at all, (nil, nil) shall be
	// returned to trigger the codepath for anonymous users.
	AuthenticateUserFromRequest(r *http.Request) (UserIdentity, *RegistryV2Error)
}

AuthDriver represents an authentication backend that supports multiple tenants. A tenant is a scope where users can be authorized to perform certain actions. For example, in OpenStack, a Keppel tenant is a Keystone project.

func NewAuthDriver

func NewAuthDriver(pluginTypeID string, rc *redis.Client) (AuthDriver, error)

NewAuthDriver creates a new AuthDriver using one of the plugins registered with AuthDriverRegistry.

type ClaimResult

type ClaimResult int

ClaimResult is an enum returned by FederationDriver.ClaimAccountName().

const (
	// ClaimSucceeded indicates that ClaimAccountName() returned with a nil error.
	ClaimSucceeded ClaimResult = iota
	// ClaimFailed indicates that ClaimAccountName() returned with an error
	// because the user did not have permission to claim the account in question.
	ClaimFailed
	// ClaimErrored indicates that ClaimAccountName() returned with an error
	// because of an unexpected problem on the server side.
	ClaimErrored
)

type Configuration

type Configuration struct {
	APIPublicHostname        string
	AnycastAPIPublicHostname string
	DatabaseURL              *url.URL
	JWTIssuerKeys            []crypto.PrivateKey
	AnycastJWTIssuerKeys     []crypto.PrivateKey
	Trivy                    *trivy.Config
}

Configuration contains all configuration values that are not specific to a certain driver.

func ParseConfiguration

func ParseConfiguration() Configuration

ParseConfiguration obtains a keppel.Configuration instance from the corresponding environment variables. Aborts on error.

func (Configuration) ReverseProxyAnycastRequestToPeer

func (cfg Configuration) ReverseProxyAnycastRequestToPeer(w http.ResponseWriter, r *http.Request, peerHostName string) error

ReverseProxyAnycastRequestToPeer takes a http.Request for the anycast API and reverse-proxies it to a different keppel-api in this Keppel's peer group.

If an error is returned, no response has been written and the caller is responsible for producing the error response.

type DB

type DB struct {
	gorp.DbMap
}

DB adds convenience functions on top of gorp.DbMap.

func InitDB

func InitDB(dbURL *url.URL) (*DB, error)

InitDB connects to the Postgres database.

func (*DB) SelectBool

func (db *DB) SelectBool(query string, args ...any) (bool, error)

SelectBool is analogous to the other SelectFoo() functions from gorp.DbMap like SelectFloat, SelectInt, SelectStr, etc.

type Duration

type Duration time.Duration

Duration is a time.Duration with custom JSON marshalling/unmarshalling logic.

func (Duration) MarshalJSON

func (d Duration) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

func (*Duration) UnmarshalJSON

func (d *Duration) UnmarshalJSON(src []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

type FederationDriver

type FederationDriver interface {
	pluggable.Plugin
	// Init is called before any other interface methods, and allows the plugin to
	// perform first-time initialization.
	//
	// Implementations should inspect the auth driver to ensure that the
	// federation driver can work with this authentication method, or return
	// ErrAuthDriverMismatch otherwise.
	Init(AuthDriver, Configuration) error

	// ClaimAccountName is called when creating a new account, and returns nil if
	// and only if this Keppel is allowed to use `account.Name` for the given new
	// `account`.
	//
	// For some drivers, creating a replica account requires confirmation from the
	// Keppel hosting the primary account. This is done by issuing a sublease
	// token secret on the primary account using IssueSubleaseTokenSecret(), then
	// presenting this `subleaseTokenSecret` to this method.
	//
	// The implementation MUST be idempotent. If a call returned nil, a subsequent
	// call with the same `account` must also return nil unless
	// ForfeitAccountName() was called in between.
	ClaimAccountName(ctx context.Context, account models.Account, subleaseTokenSecret string) (ClaimResult, error)

	// IssueSubleaseTokenSecret may only be called on existing primary accounts,
	// not on replica accounts. It generates a secret one-time token that other
	// Keppels can use to verify that the caller is allowed to create a replica
	// account for this primary account.
	//
	// Sublease tokens are optional. If ClaimAccountName does not inspect its
	// `subleaseTokenSecret` parameter, this method shall return ("", nil).
	IssueSubleaseTokenSecret(ctx context.Context, account models.Account) (string, error)

	// ForfeitAccountName is the inverse operation of ClaimAccountName. It is used
	// when deleting an account and releases this Keppel's claim on the account
	// name.
	ForfeitAccountName(ctx context.Context, account models.Account) error

	// RecordExistingAccount is called regularly for each account in our database.
	// The driver implementation can use this call to ensure that the existence of
	// this account is tracked in its storage. (We don't expect this to require
	// any actual work during normal operation. The purpose of this mechanism is
	// to aid in switching between federation drivers.)
	//
	// The `now` argument contains the value of time.Now(). It may refer to an
	// artificial wall clock during unit tests.
	RecordExistingAccount(ctx context.Context, account models.Account, now time.Time) error

	// FindPrimaryAccount is used to redirect anycast requests for accounts that
	// do not exist locally. It shell return the hostname of the peer that hosts
	// the primary account. If no account with this name exists anywhere,
	// ErrNoSuchPrimaryAccount shall be returned.
	FindPrimaryAccount(ctx context.Context, accountName string) (peerHostName string, err error)
}

FederationDriver is the abstract interface for a strategy that coordinates the claiming of account names across Keppel deployments.

func NewFederationDriver

func NewFederationDriver(pluginTypeID string, ad AuthDriver, cfg Configuration) (FederationDriver, error)

NewFederationDriver creates a new FederationDriver using one of the plugins registered with FederationDriverRegistry.

type GCPolicy

type GCPolicy struct {
	RepositoryRx         regexpext.BoundedRegexp `json:"match_repository"`
	NegativeRepositoryRx regexpext.BoundedRegexp `json:"except_repository,omitempty"`
	TagRx                regexpext.BoundedRegexp `json:"match_tag,omitempty"`
	NegativeTagRx        regexpext.BoundedRegexp `json:"except_tag,omitempty"`
	OnlyUntagged         bool                    `json:"only_untagged,omitempty"`
	TimeConstraint       *GCTimeConstraint       `json:"time_constraint,omitempty"`
	Action               string                  `json:"action"`
}

GCPolicy is a policy enabling optional garbage collection runs in an account. It is stored in serialized form in the GCPoliciesJSON field of type Account.

func ParseGCPolicies

func ParseGCPolicies(account models.Account) ([]GCPolicy, error)

ParseGCPolicies parses the GC policies for the given account.

func (GCPolicy) MatchesRepository

func (g GCPolicy) MatchesRepository(repoName string) bool

MatchesRepository evaluates the repository regexes in this policy.

func (GCPolicy) MatchesTags

func (g GCPolicy) MatchesTags(tagNames []string) bool

MatchesTags evaluates the tag regexes in this policy for a complete set of tag names belonging to a single manifest.

func (GCPolicy) MatchesTimeConstraint

func (g GCPolicy) MatchesTimeConstraint(manifest models.Manifest, allManifestsInRepo []models.Manifest, now time.Time) bool

MatchesTimeConstraint evaluates the time constraint in this policy for the given manifest. A full list of all manifests in this repo must be supplied in order to evaluate "newest" and "oldest" time constraints. The final argument must be equivalent to time.Now(); it is given explicitly to allow for simulated clocks during unit tests.

func (GCPolicy) Validate

func (g GCPolicy) Validate() error

Validate returns an error if this policy is invalid.

type GCStatus

type GCStatus struct {
	// True if the manifest was uploaded less than 10 minutes ago and is therefore
	// protected from GC.
	ProtectedByRecentUpload bool `json:"protected_by_recent_upload,omitempty"`
	// If a parent manifest references this manifest and thus protects it from GC,
	// contains the parent manifest's digest.
	ProtectedByParentManifest string `json:"protected_by_parent,omitempty"`
	// If a policy with action "protect" applies to this image, contains the
	// definition of the policy.
	ProtectedByPolicy *GCPolicy `json:"protected_by_policy,omitempty"`
	// If the image is not protected, contains all policies with action "delete"
	// that could delete this image in the future.
	RelevantPolicies []GCPolicy `json:"relevant_policies,omitempty"`
}

GCStatus documents the current status of a manifest with regard to image GC. It is stored in serialized form in the GCStatusJSON field of type Manifest.

Since GCStatus objects describe images that currently exist in the DB, they only describe policy decisions that led to no cleanup.

func (GCStatus) IsProtected

func (s GCStatus) IsProtected() bool

IsProtected returns whether any of the ProtectedBy... fields is filled.

type GCTimeConstraint

type GCTimeConstraint struct {
	FieldName   string   `json:"on"`
	OldestCount uint64   `json:"oldest,omitempty"`
	NewestCount uint64   `json:"newest,omitempty"`
	MinAge      Duration `json:"older_than,omitempty"`
	MaxAge      Duration `json:"newer_than,omitempty"`
}

GCTimeConstraint appears in type GCPolicy.

type InboundCacheDriver

type InboundCacheDriver interface {
	pluggable.Plugin
	// Init is called before any other interface methods, and allows the plugin to
	// perform first-time initialization.
	Init(Configuration) error

	// LoadManifest pulls a manifest from the cache. If the given manifest is not
	// cached, or if the cache entry has expired, sql.ErrNoRows shall be returned.
	//
	// time.Now() is given in the second argument to allow for tests to use an
	// artificial wall clock.
	LoadManifest(location models.ImageReference, now time.Time) (contents []byte, mediaType string, err error)
	// StoreManifest places a manifest in the cache for later retrieval.
	//
	// time.Now() is given in the last argument to allow for tests to use an
	// artificial wall clock.
	StoreManifest(location models.ImageReference, contents []byte, mediaType string, now time.Time) error
}

InboundCacheDriver is the abstract interface for a caching strategy for manifests and tags residing in an external registry.

func NewInboundCacheDriver

func NewInboundCacheDriver(pluginTypeID string, cfg Configuration) (InboundCacheDriver, error)

NewInboundCacheDriver creates a new InboundCacheDriver using one of the plugins registered with InboundCacheDriverRegistry.

type ManifestForSync

type ManifestForSync struct {
	Digest       digest.Digest `json:"digest"`
	LastPulledAt *int64        `json:"last_pulled_at,omitempty"`
	Tags         []TagForSync  `json:"tags,omitempty"`
}

ManifestForSync represents a manifest in the _sync_replica API endpoint.

(This type is declared in this package because it gets used in both internal/api/peer and internal/tasks.)

type ParsedManifest

type ParsedManifest interface {
	// FindImageConfigBlob returns the descriptor of the blob containing this
	// manifest's image configuration, or nil if the manifest does not have an image
	// configuration.
	FindImageConfigBlob() *distribution.Descriptor
	// FindImageLayerBlobs returns the descriptors of the blobs containing this
	// manifest's image layers, or an empty list if the manifest does not have layers.
	FindImageLayerBlobs() []distribution.Descriptor
	// BlobReferences returns all blobs referenced by this manifest.
	BlobReferences() []distribution.Descriptor
	// ManifestReferences returns all manifests referenced by this manifest.
	ManifestReferences(pf models.PlatformFilter) []manifestlist.ManifestDescriptor
	// AcceptableAlternates returns the subset of ManifestReferences() that is
	// acceptable as alternate representations of this manifest. When a client
	// asks for this manifest, but the Accept header does not match the manifest
	// itself, the API will look for an acceptable alternate to serve instead.
	AcceptableAlternates(pf models.PlatformFilter) []manifestlist.ManifestDescriptor
}

ParsedManifest is an interface that can interrogate manifests about the blobs and submanifests referenced therein.

func ParseManifest

func ParseManifest(mediaType string, contents []byte) (ParsedManifest, distribution.Descriptor, error)

ParseManifest parses a manifest. It also returns a Descriptor describing the manifest itself.

type Permission

type Permission string

Permission is an enum used by AuthDriver.

const (
	// CanViewAccount is the permission for viewing account metadata.
	CanViewAccount Permission = "view"
	// CanPullFromAccount is the permission for pulling images from this account.
	CanPullFromAccount Permission = "pull"
	// CanPushToAccount is the permission for pushing images to this account.
	CanPushToAccount Permission = "push"
	// CanDeleteFromAccount is the permission for deleting manifests from this account.
	CanDeleteFromAccount Permission = "delete"
	// CanChangeAccount is the permission for creating and updating accounts.
	CanChangeAccount Permission = "change"
	// CanViewQuotas is the permission for viewing an auth tenant's quotas.
	CanViewQuotas Permission = "viewquota"
	// CanChangeQuotas is the permission for changing an auth tenant's quotas.
	CanChangeQuotas Permission = "changequota"
)

type RBACPermission

type RBACPermission string

RBACPermission enumerates permissions that can be granted by an RBAC policy.

const (
	GrantsPull               RBACPermission = "pull"
	GrantsPush               RBACPermission = "push"
	GrantsDelete             RBACPermission = "delete"
	GrantsAnonymousPull      RBACPermission = "anonymous_pull"
	GrantsAnonymousFirstPull RBACPermission = "anonymous_first_pull"
)

type RBACPolicy

type RBACPolicy struct {
	CidrPattern       string                  `json:"match_cidr,omitempty"`
	RepositoryPattern regexpext.BoundedRegexp `json:"match_repository,omitempty"`
	UserNamePattern   regexpext.BoundedRegexp `json:"match_username,omitempty"`
	Permissions       []RBACPermission        `json:"permissions"`
}

RBACPolicy is a policy granting user-defined access to repos in an account. It is stored in serialized form in the RBACPoliciesJSON field of type Account.

func ParseRBACPolicies

func ParseRBACPolicies(account models.Account) ([]RBACPolicy, error)

ParseRBACPolicies parses the RBAC policies for the given account.

func (RBACPolicy) Matches

func (r RBACPolicy) Matches(ip, repoName, userName string) bool

Matches evaluates the cidr and regexes in this policy.

func (*RBACPolicy) ValidateAndNormalize

func (r *RBACPolicy) ValidateAndNormalize(strategy ReplicationStrategy) error

ValidateAndNormalize performs some normalizations and returns an error if this policy is invalid.

type RateLimitDriver

type RateLimitDriver interface {
	pluggable.Plugin
	// Init is called before any other interface methods, and allows the plugin to
	// perform first-time initialization.
	//
	// Implementations should inspect the auth driver to ensure that the
	// federation driver can work with this authentication method, or return
	// ErrAuthDriverMismatch otherwise.
	Init(AuthDriver, Configuration) error

	// GetRateLimit shall return nil if the given action has no rate limit.
	GetRateLimit(account models.Account, action RateLimitedAction) *redis_rate.Limit
}

RateLimitDriver is a pluggable strategy that determines the rate limits of each account.

func NewRateLimitDriver

func NewRateLimitDriver(pluginTypeID string, ad AuthDriver, cfg Configuration) (RateLimitDriver, error)

NewRateLimitDriver creates a new RateLimitDriver using one of the plugins registered with RateLimitDriverRegistry.

type RateLimitEngine

type RateLimitEngine struct {
	Driver RateLimitDriver
	Client *redis.Client
}

RateLimitEngine provides the rate-limiting interface used by the API implementation.

func (RateLimitEngine) RateLimitAllows

func (e RateLimitEngine) RateLimitAllows(ctx context.Context, remoteAddr string, account models.Account, action RateLimitedAction, amount uint64) (bool, *redis_rate.Result, error)

RateLimitAllows checks whether the given action on the given account is allowed by the account's rate limit.

type RateLimitedAction

type RateLimitedAction string

RateLimitedAction is an enum of all actions that can be rate-limited.

const (
	// BlobPullAction is a RateLimitedAction.
	BlobPullAction RateLimitedAction = "pullblob"
	// BlobPushAction is a RateLimitedAction.
	BlobPushAction RateLimitedAction = "pushblob"
	// ManifestPullAction is a RateLimitedAction.
	ManifestPullAction RateLimitedAction = "pullmanifest"
	// ManifestPushAction is a RateLimitedAction.
	ManifestPushAction RateLimitedAction = "pushmanifest"
	// AnycastBlobBytePullAction is a RateLimitedAction.
	// It refers to blobs being pulled from other regions via anycast.
	// The `amount` given to RateLimitAllows() shall be the blob size in bytes.
	AnycastBlobBytePullAction RateLimitedAction = "pullblobbytesanycast"
	// TrivyReportRetrieveAction is a RateLimitedAction.
	// It refers to reports being retrieved from keppel through the trivy proxy from trivy itself.
	TrivyReportRetrieveAction RateLimitedAction = "retrievetrivyreport"
)

type RegistryV2Error

type RegistryV2Error struct {
	Code    RegistryV2ErrorCode `json:"code"`
	Message string              `json:"message"`
	// Detail is always a string for errors generated by Keppel, but may be a JSON
	// object (i.e. map[string]any or similar) for errors coming from
	// keppel-registry.
	Detail  any         `json:"detail"`
	Status  int         `json:"-"`
	Headers http.Header `json:"-"`
}

RegistryV2Error is the error type expected by clients of the docker-registry v2 API.

func AsRegistryV2Error

func AsRegistryV2Error(err error) *RegistryV2Error

AsRegistryV2Error tries to cast `err` into RegistryV2Error. If `err` is not a RegistryV2Error, it gets wrapped in ErrUnknown instead.

func (*RegistryV2Error) Error

func (e *RegistryV2Error) Error() string

Error implements the builtin/error interface.

func (*RegistryV2Error) WithDetail

func (e *RegistryV2Error) WithDetail(detail any) *RegistryV2Error

WithDetail adds detail information to this error.

func (*RegistryV2Error) WithHeader

func (e *RegistryV2Error) WithHeader(key string, values ...string) *RegistryV2Error

WithHeader adds a HTTP response header to this error.

func (*RegistryV2Error) WithStatus

func (e *RegistryV2Error) WithStatus(status int) *RegistryV2Error

WithStatus changes the HTTP status code for this error.

func (*RegistryV2Error) WriteAsAuthResponseTo

func (e *RegistryV2Error) WriteAsAuthResponseTo(w http.ResponseWriter)

WriteAsAuthResponseTo reports this error in the format used by the Auth API endpoint.

func (*RegistryV2Error) WriteAsRegistryV2ResponseTo

func (e *RegistryV2Error) WriteAsRegistryV2ResponseTo(w http.ResponseWriter, r *http.Request)

WriteAsRegistryV2ResponseTo reports this error in the format used by the Registry V2 API.

func (*RegistryV2Error) WriteAsTextTo

func (e *RegistryV2Error) WriteAsTextTo(w http.ResponseWriter)

WriteAsTextTo reports this error in a plain text format.

type RegistryV2ErrorCode

type RegistryV2ErrorCode string

RegistryV2ErrorCode is the closed set of error codes that can appear in type RegistryV2Error.

const (
	ErrBlobUnknown         RegistryV2ErrorCode = "BLOB_UNKNOWN"
	ErrBlobUploadInvalid   RegistryV2ErrorCode = "BLOB_UPLOAD_INVALID"
	ErrBlobUploadUnknown   RegistryV2ErrorCode = "BLOB_UPLOAD_UNKNOWN"
	ErrDigestInvalid       RegistryV2ErrorCode = "DIGEST_INVALID"
	ErrManifestBlobUnknown RegistryV2ErrorCode = "MANIFEST_BLOB_UNKNOWN"
	ErrManifestInvalid     RegistryV2ErrorCode = "MANIFEST_INVALID"
	ErrManifestUnknown     RegistryV2ErrorCode = "MANIFEST_UNKNOWN"
	ErrManifestUnverified  RegistryV2ErrorCode = "MANIFEST_UNVERIFIED"
	ErrNameInvalid         RegistryV2ErrorCode = "NAME_INVALID"
	ErrNameUnknown         RegistryV2ErrorCode = "NAME_UNKNOWN"
	ErrSizeInvalid         RegistryV2ErrorCode = "SIZE_INVALID"
	ErrTagInvalid          RegistryV2ErrorCode = "TAG_INVALID"
	ErrUnauthorized        RegistryV2ErrorCode = "UNAUTHORIZED"
	ErrDenied              RegistryV2ErrorCode = "DENIED"
	ErrUnsupported         RegistryV2ErrorCode = "UNSUPPORTED"

	// not in opencontainers/distribution-spec, but appears in github.com/docker/distribution
	ErrUnknown         RegistryV2ErrorCode = "UNKNOWN"
	ErrUnavailable     RegistryV2ErrorCode = "UNAVAILABLE"
	ErrTooManyRequests RegistryV2ErrorCode = "TOOMANYREQUESTS"
)

Possible values for RegistryV2ErrorCode.

func (RegistryV2ErrorCode) With

func (c RegistryV2ErrorCode) With(msg string, args ...any) *RegistryV2Error

With is a convenience function for constructing type RegistryV2Error.

type ReplicaSyncPayload

type ReplicaSyncPayload struct {
	Manifests []ManifestForSync `json:"manifests"`
}

ReplicaSyncPayload is the format for request bodies and response bodies of the sync-replica API endpoint.

(This type is declared in this package because it gets used in both internal/api/peer and internal/tasks.)

func (ReplicaSyncPayload) DigestForTag

func (p ReplicaSyncPayload) DigestForTag(name string) digest.Digest

DigestForTag returns the digest of the manifest that this tag points to, or the empty string if the tag does not exist in this payload.

func (ReplicaSyncPayload) HasManifest

func (p ReplicaSyncPayload) HasManifest(manifestDigest digest.Digest) bool

HasManifest returns whether there is a manifest with the given digest in this payload.

type ReplicationExternalPeerSpec

type ReplicationExternalPeerSpec struct {
	URL      string `json:"url"`
	UserName string `json:"username,omitempty"`
	Password string `json:"password,omitempty"`
}

ReplicationExternalPeerSpec appears in type ReplicationPolicy.

type ReplicationPolicy

type ReplicationPolicy struct {
	Strategy ReplicationStrategy `json:"strategy"`
	// only for `on_first_use`
	UpstreamPeerHostName string `json:"upstream_peer_hostname"`
	// only for `from_external_on_first_use`
	ExternalPeer ReplicationExternalPeerSpec `json:"external_peer"`
}

ReplicationPolicy represents a replication policy in the API.

func RenderReplicationPolicy

func RenderReplicationPolicy(account models.Account) *ReplicationPolicy

RenderReplicationPolicy builds a ReplicationPolicy object out of the information in the given account model.

func (ReplicationPolicy) ApplyToAccount

func (r ReplicationPolicy) ApplyToAccount(account *models.Account) error

ApplyToAccount validates this policy and stores it in the given account model.

WARNING 1: For existing accounts, the caller must ensure that the policy uses the same replication strategy as the given account already does.

WARNING 2: For internal replica accounts, the caller must ensure that the UpstreamPeerHostName refers to a known peer. This method does not do it itself because callers often need to do other things with the peer, too.

func (ReplicationPolicy) MarshalJSON

func (r ReplicationPolicy) MarshalJSON() ([]byte, error)

MarshalJSON implements the json.Marshaler interface.

func (*ReplicationPolicy) UnmarshalJSON

func (r *ReplicationPolicy) UnmarshalJSON(buf []byte) error

UnmarshalJSON implements the json.Unmarshaler interface.

type ReplicationStrategy

type ReplicationStrategy string

ReplicationStrategy is an enum that appears in type ReplicationPolicy.

const (
	NoReplicationStrategy          ReplicationStrategy = ""
	OnFirstUseStrategy             ReplicationStrategy = "on_first_use"
	FromExternalOnFirstUseStrategy ReplicationStrategy = "from_external_on_first_use"
)

type SecurityScanPolicy

type SecurityScanPolicy struct {
	//NOTE: We have code that uses slices.Contains() to locate policies. Be careful
	// when adding fields that cannot be meaningfully compared with the == operator.
	ManagingUserName          string                   `json:"managed_by_user,omitempty"`
	RepositoryRx              regexpext.BoundedRegexp  `json:"match_repository"`
	NegativeRepositoryRx      regexpext.BoundedRegexp  `json:"except_repository,omitempty"`
	VulnerabilityIDRx         regexpext.BoundedRegexp  `json:"match_vulnerability_id"`
	NegativeVulnerabilityIDRx regexpext.BoundedRegexp  `json:"except_vulnerability_id,omitempty"`
	ExceptFixReleased         bool                     `json:"except_fix_released,omitempty"`
	Action                    SecurityScanPolicyAction `json:"action"`
}

SecurityScanPolicy is a policy enabling user-defined adjustments to vulnerability reports generated by Trivy.

func (SecurityScanPolicy) MatchesRepository

func (p SecurityScanPolicy) MatchesRepository(repo models.Repository) bool

MatchesRepository evaluates the repository regexes in this policy.

func (SecurityScanPolicy) MatchesVulnerability

func (p SecurityScanPolicy) MatchesVulnerability(vuln types.DetectedVulnerability) bool

MatchesVulnerability evaluates the vulnerability regexes and checkin this policy.

func (SecurityScanPolicy) String

func (p SecurityScanPolicy) String() string

String returns the JSON representation of this policy (for use in log and error messages).

func (SecurityScanPolicy) Validate

func (p SecurityScanPolicy) Validate(path string) (errs errext.ErrorSet)

Validate returns errors if this policy is invalid.

When constructing error messages, `path` is prepended to all field names. This allows identifying the location of the policy within a larger data structure.

func (SecurityScanPolicy) VulnerabilityStatus

func (p SecurityScanPolicy) VulnerabilityStatus() models.VulnerabilityStatus

VulnerabilityStatus returns the status that this policy forces for matching vulnerabilities in matching repos.

type SecurityScanPolicyAction

type SecurityScanPolicyAction struct {
	Assessment string                     `json:"assessment"`
	Ignore     bool                       `json:"ignore,omitempty"`
	Severity   models.VulnerabilityStatus `json:"severity,omitempty"`
}

SecurityScanPolicyAction appears in type SecurityScanPolicy.

type SecurityScanPolicySet

type SecurityScanPolicySet []SecurityScanPolicy

SecurityScanPolicySet contains convenience functions for operating on a list of SecurityScanPolicy (like those found in Account.SecurityScanPoliciesJSON).

func GetSecurityScanPolicies

func GetSecurityScanPolicies(account models.Account, repo models.Repository) (SecurityScanPolicySet, error)

SecurityScanPoliciesFor deserializes this account's security scan policies and returns the subset that match the given repository.

func (SecurityScanPolicySet) EnrichReport

func (s SecurityScanPolicySet) EnrichReport(payload *trivy.ReportPayload) error

EnrichReport computes and inserts the "X-Keppel-Applicable-Policies" field if the report is `--format json`. Other formats are not altered.

func (SecurityScanPolicySet) PolicyForVulnerability

func (s SecurityScanPolicySet) PolicyForVulnerability(vuln types.DetectedVulnerability) *SecurityScanPolicy

PolicyForVulnerability returns the first policy from this set that matches the vulnerability, or nil if no policy matches.

type StorageDriver

type StorageDriver interface {
	pluggable.Plugin
	// Init is called before any other interface methods, and allows the plugin to
	// perform first-time initialization.
	//
	// Implementations should inspect the auth driver to ensure that the
	// federation driver can work with this authentication method, or return
	// ErrAuthDriverMismatch otherwise.
	Init(AuthDriver, Configuration) error

	// `storageID` identifies blobs within an account. (The storage ID is
	// different from the digest: The storage ID gets chosen at the start of the
	// upload, when we don't know the full digest yet.) `chunkNumber` identifies
	// how often AppendToBlob() has already been called for this account and
	// storageID. For the first call to AppendToBlob(), `chunkNumber` will be 1.
	// The second call will have a `chunkNumber` of 2, and so on.
	//
	// If `chunkLength` is non-nil, the implementation may assume that `chunk`
	// will yield that many bytes, and return keppel.ErrSizeInvalid when that
	// turns out not to be true.
	AppendToBlob(account models.Account, storageID string, chunkNumber uint32, chunkLength *uint64, chunk io.Reader) error
	// FinalizeBlob() is called at the end of the upload, after the last
	// AppendToBlob() call for that blob. `chunkCount` identifies how often
	// AppendToBlob() was called.
	FinalizeBlob(account models.Account, storageID string, chunkCount uint32) error
	// AbortBlobUpload() is used to clean up after an error in AppendToBlob() or
	// FinalizeBlob(). It is the counterpart of DeleteBlob() for when any part of
	// the blob upload failed.
	AbortBlobUpload(account models.Account, storageID string, chunkCount uint32) error

	ReadBlob(account models.Account, storageID string) (contents io.ReadCloser, sizeBytes uint64, err error)
	// If the blob can be retrieved by a publicly accessible URL, URLForBlob shall
	// return it. Otherwise ErrCannotGenerateURL shall be returned to instruct the
	// caller fall back to ReadBlob().
	URLForBlob(account models.Account, storageID string) (string, error)
	// DeleteBlob may assume that FinalizeBlob() has been called. If an error
	// occurred before or during FinalizeBlob(), AbortBlobUpload() will be called
	// instead.
	DeleteBlob(account models.Account, storageID string) error

	ReadManifest(account models.Account, repoName string, digest digest.Digest) ([]byte, error)
	WriteManifest(account models.Account, repoName string, digest digest.Digest, contents []byte) error
	DeleteManifest(account models.Account, repoName string, digest digest.Digest) error

	// This method shall only be used as a positive signal for the existence of a
	// blob or manifest in the storage, not as a negative signal: If we expect a
	// blob or manifest to be in the storage, but it does not show up in these
	// lists, that does not necessarily mean it does not exist in the storage.
	// This is because storage implementations may be backed by object stores with
	// eventual consistency.
	ListStorageContents(account models.Account) (blobs []StoredBlobInfo, manifests []StoredManifestInfo, err error)

	// This method is called before a new account is set up in the DB. The
	// StorageDriver can use this opportunity to check for any reasons why the
	// account would not be functional once it is persisted in our DB.
	CanSetupAccount(account models.Account) error
	// This method can be used by the StorageDriver to perform last-minute cleanup
	// on an account that we are about to delete. This cleanup should be
	// reversible; we might bail out of the account deletion afterwards if the
	// deletion in the DB fails.
	CleanupAccount(account models.Account) error
}

StorageDriver is the abstract interface for a multi-tenant-capable storage backend.

func NewStorageDriver

func NewStorageDriver(pluginTypeID string, ad AuthDriver, cfg Configuration) (StorageDriver, error)

NewStorageDriver creates a new StorageDriver using one of the factory functions registered with RegisterStorageDriver().

type StoredBlobInfo

type StoredBlobInfo struct {
	StorageID string
	// ChunkCount is 0 for finalized blobs (that can be deleted with DeleteBlob)
	// or >0 for ongoing uploads (that can be deleted with AbortBlobUpload).
	ChunkCount uint32
}

StoredBlobInfo is returned by StorageDriver.ListStorageContents().

type StoredManifestInfo

type StoredManifestInfo struct {
	RepoName string
	Digest   digest.Digest
}

StoredManifestInfo is returned by StorageDriver.ListStorageContents().

type TagForSync

type TagForSync struct {
	Name         string `json:"name"`
	LastPulledAt *int64 `json:"last_pulled_at,omitempty"`
}

TagForSync represents a tag in the _sync_replica API endpoint.

(This type is declared in this package because it gets used in both internal/api/peer and internal/tasks.)

type UserIdentity

type UserIdentity interface {
	pluggable.Plugin

	// Returns whether the given auth tenant grants the given permission to this user.
	// The AnonymousUserIdentity always returns false.
	HasPermission(perm Permission, tenantID string) bool

	// Identifies the type of user that was authenticated.
	UserType() UserType
	// Returns the name of the user that was authenticated. This should be the
	// same format that is given as the first argument of AuthenticateUser().
	// The AnonymousUserIdentity always returns the empty string.
	UserName() string
	// If this identity is backed by a Keystone token, return a UserInfo for that
	// token. Returns nil otherwise, especially for all anonymous and peer users.
	//
	// If non-nil, the Keppel API will submit OpenStack CADF audit events.
	UserInfo() audittools.UserInfo

	// SerializeToJSON serializes this UserIdentity instance into JSON for
	// inclusion in a token payload.
	SerializeToJSON() (payload []byte, err error)
	// DeserializeFromJSON deserializes the given token payload (as returned by
	// SerializeToJSON) into the callee. This is always called on a fresh
	// instance created by UserIdentityFactory.Instantiate().
	DeserializeFromJSON(payload []byte, ad AuthDriver) error
}

UserIdentity describes the identity and access rights of a user. For regular users, it is returned by methods in the AuthDriver interface. For all other types of users, it is implicitly created in helper methods higher up in the stack.

func DeserializeUserIdentity

func DeserializeUserIdentity(typeID string, payload []byte, ad AuthDriver) (UserIdentity, error)

DeserializeUserIdentity deserializes a UserIdentity payload. This is the reverse of UserIdentity.SerializeToJSON().

type UserType

type UserType int

UserType is an enum that identifies the general type of user. User types are important because certain API endpoints or certain behavior is restricted to specific user types. For example, anonymous users may not cause implicit replications to occur, and peer users are exempt from rate limits.

const (
	// RegularUser is the UserType for regular users that authenticated via the AuthDriver.
	RegularUser UserType = iota
	// AnonymousUser is the UserType for unauthenticated users.
	AnonymousUser
	// PeerUser is the UserType for peer users, i.e. other Keppel instances using the API as a peer.
	PeerUser
	// TrivyUser is the UserType for tokens issued to Trivy.
	TrivyUser
	// JanitorUser is a dummy UserType for when the janitor needs an Authorization for audit logging purposes.
	JanitorUser
)

type ValidationPolicy

type ValidationPolicy struct {
	RequiredLabels []string `json:"required_labels,omitempty"`
}

ValidationPolicy represents a validation policy in the API.

func RenderValidationPolicy

func RenderValidationPolicy(account models.Account) *ValidationPolicy

RenderValidationPolicy builds a ValidationPolicy object out of the information in the given account model.

func (ValidationPolicy) ApplyToAccount

func (v ValidationPolicy) ApplyToAccount(account *models.Account) *RegistryV2Error

ApplyToAccount validates this policy and stores it in the given account model.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL