model

package
v0.0.0-...-d7353a2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 3, 2024 License: Apache-2.0 Imports: 51 Imported by: 0

Documentation

Overview

Package model contains datastore model definitions.

Package model contains model definitions for Auth Service.

This file contains functionality related to the "direct push" method of AuthDB replication.

Index

Constants

View Source
const (
	DryRunAPIChangesEnvVar    = "DRY_RUN_API_CHANGES"
	DryRunCronConfigEnvVar    = "DRY_RUN_CRON_CONFIG"
	DryRunCronRealmsEnvVar    = "DRY_RUN_CRON_REALMS"
	DryRunCronStaleAuthEnvVar = "DRY_RUN_CRON_STALE_AUTH"
	DryRunTQChangelogEnvVar   = "DRY_RUN_TQ_CHANGELOG"
	DryRunTQReplicationEnvVar = "DRY_RUN_TQ_REPLICATION"
	EnableGroupImportsEnvVar  = "ENABLE_GROUP_IMPORTS"
)

Names of enviroment variables which control component functionality while transitioning from Auth Service v1 (Python) to Auth Service v2 (Go).

Each environment variable should be either "true" or "false" (case insensitive).

View Source
const (
	// AdminGroup defines a group whose members are allowed to create new groups.
	AdminGroup = "administrators"

	// TrustedServicesGroup defines a group whose members are allowed to
	// subscribe to group change notifications and fetch all groups at
	// once.
	TrustedServicesGroup = "auth-trusted-services"

	// Max size of AuthDBShard.Blob.
	MaxShardSize = 900 * 1024 // 900 kB.

	// The prefix to use for datastore entities' Kind field when in dry
	// run mode.
	DryRunPrefix = "V2"
)
View Source
const (
	// Object ACLs can have at most 100 entries. We limit them to 80 to
	// have some breathing room before the hard limit is reached. When
	// this happens, either some old services should be deauthorized or
	// GCS ACL management should be reimplemented in some different way.
	//
	// See https://cloud.google.com/storage/quotas.
	MaxReaders = 80

	// Limit for an AuthDBReader email, because the maximum size of an
	// indexed string property's UTF-8 encoding in Datastore is 1500 B.
	//
	// See https://cloud.google.com/datastore/docs/concepts/limits.
	MaxReaderEmailLength = 375
)
View Source
const (
	// Defines version of Auth API.
	//
	// Should be increased on any API or protocol changes.
	AuthAPIVersion = "2.0.0"
)
View Source
const RealmsAPIVersion = 1

RealmsAPIVersion is the currently acceptable version of Realms API. See api_version in realms.proto.

Variables

View Source
var (
	// ErrAlreadyExists is returned when an entity already exists.
	ErrAlreadyExists = stderrors.New("entity already exists")
	// ErrPermissionDenied is returned when the user does not have permission for
	// the requested entity operation.
	ErrPermissionDenied = stderrors.New("permission denied")
	// ErrInvalidArgument is a generic error returned when an argument is invalid.
	ErrInvalidArgument = stderrors.New("invalid argument")
	// ErrInvalidName is returned when a supplied entity name is invalid.
	ErrInvalidName = stderrors.New("invalid entity name")
	// ErrInvalidReference is returned when a referenced entity name is invalid.
	ErrInvalidReference = stderrors.New("invalid reference")
	// ErrInvalidIdentity is returned when a referenced identity or glob is invalid.
	ErrInvalidIdentity = stderrors.New("invalid identity")
	// ErrConcurrentModification is returned when an entity is modified by two concurrent operations.
	ErrConcurrentModification = stderrors.New("concurrent modification")
	// ErrReferencedEntity is returned when an entity cannot be deleted because it is referenced elsewhere.
	ErrReferencedEntity = stderrors.New("cannot delete referenced entity")
	// ErrCyclicDependency is returned when an update would create a cyclic dependency.
	ErrCyclicDependency = stderrors.New("circular dependency")
)
View Source
var (
	FatalReplicaUpdateError = errors.New("fatal replica update error")
)
View Source
var GroupNameRe = regexp.MustCompile(`^([a-z\-]+/)?[0-9a-z_\-\.@]{1,100}$`)

Functions

func AuthorizeReader

func AuthorizeReader(ctx context.Context, email string) error

AuthorizeReader records the given email as an AuthDBReader, then updates Auth Service's Google Storage object ACLs.

func ChangeID

func ChangeID(ctx context.Context, change *AuthDBChange) (string, error)

ChangeID returns the ID of an AuthDBChange entity based on its properties.

Returns error when change.Target is invalid (doesn't contain '$' or contains '!').

func ChangeLogRevisionKey

func ChangeLogRevisionKey(ctx context.Context, authDBRev int64, dryRun bool) *datastore.Key

ChangeLogRevisionKey returns the key of entity subgroup that keeps AuthDB change log for a revision.

func ChangeLogRootKey

func ChangeLogRootKey(ctx context.Context, dryRun bool) *datastore.Key

ChangeLogRootKey returns the root key of an entity group with change log.

func CompareV2Entities

func CompareV2Entities(ctx context.Context) error

CompareV2Entities compares the entities generated by Auth Service v1 (Python) to those generated by Auth Service v2 (Go) for the latest AuthDB revision. The comparison result is logged.

Returns an annotated error if one occurred.

func DeauthorizeReader

func DeauthorizeReader(ctx context.Context, email string) error

DeauthorizeReader removes the given email as an AuthDBReader, then updates Auth Service's Google Storage object ACLs.

func DeleteAuthGroup

func DeleteAuthGroup(ctx context.Context, groupName string, etag string, external bool, historicalComment string, dryRun bool) error

DeleteAuthGroup deletes the specified auth group.

Possible errors:

datastore.ErrNoSuchEntity if the specified group does not exist.
ErrPermissionDenied if the caller is not allowed to delete the group.
ErrConcurrentModification if the provided etag is not up-to-date.
ErrReferencedEntity if the group is referenced by another group.
Annotated error for other errors.

func DeleteAuthProjectRealms

func DeleteAuthProjectRealms(ctx context.Context, project string, dryRun bool, historicalComment string) error

DeleteAuthProjectRealms deletes an AuthProjectRealms entity from datastore. The caller is expected to handle the error.

Returns error if Get from datastore failed Returns error if transaction delete failed

func EnqueueProcessChangeTask

func EnqueueProcessChangeTask(ctx context.Context, authdbrev int64) error

EnqueueProcessChangeTask adds a ProcessChangeTask task to the cloud task queue.

func EnqueueReplicationTask

func EnqueueReplicationTask(ctx context.Context, authdbrev int64) error

EnqueueReplicationTask adds a ReplicationTask task to the cloud task queue.

func GetAuthorizedEmails

func GetAuthorizedEmails(ctx context.Context) (stringset.Set, error)

GetAuthorizedEmails returns the emails of all AuthDBReaders.

func HistoricalRevisionKey

func HistoricalRevisionKey(ctx context.Context, authDBRev int64) *datastore.Key

HistoricalRevisionKey gets the key for an entity subgroup that holds changes done in a concrete revision.

func IngestTarball

func IngestTarball(ctx context.Context, name string, content io.Reader) ([]string, int64, error)

IngestTarball handles upload of tarball's specified in 'tarball_upload' config entries. expected to be called in an auth context of the upload PUT request.

returns

[]string - list of modified groups
int64 - authDBRevision
error
	proto translation error
	entry is nil
	entry not found in tarball upload config
	unauthorized uploader
	bad tarball structure

func IsAuthorizedReader

func IsAuthorizedReader(ctx context.Context, email string) (bool, error)

IsAuthorizedReader returns whether the given email is authorized to read the AuthDB from Google Storage.

func IsExternalAuthGroupName

func IsExternalAuthGroupName(name string) bool

IsExternalAuthGroupName checks if the given name is a valid external auth group name. This also implies that the auth group is not editable.

func MergeRealms

func MergeRealms(
	realmsGlobals *AuthRealmsGlobals,
	allAuthProjectRealms []*AuthProjectRealms,
	useV1Perms bool) (*protocol.Realms, error)

MergeRealms merges all the project realms into one realms definition, using the permissions in the given AuthRealmsGlobals as an authoritative source of valid permissions.

If some realm uses a permission not in the list, it will be silently dropped from the bindings. This can potentially happen due to the asynchronous nature of realms config updates (e.g. a role change that deletes some permissions can be committed into the AuthDB before realms are reevaluated). Eventually, the state should converge to be 100% consistent.

Returns a single Realms object, which represents all projects' realms if there were no issues when merging, otherwise nil and an annotated error.

An error may occur when: * unmarshalling project realms; * merging conditions into a single set; * processing a single realm that doesn't have the project ID as a prefix; * relabelling condition indices relative to the new set of all conditions.

func ParseDryRunEnvVar

func ParseDryRunEnvVar(envVar string) bool

ParseDryRunEnvVar parses the dry run flag from the given environment variable, defaulting to true.

func ParseEnableEnvVar

func ParseEnableEnvVar(envVar string) bool

ParseEnableEnvVar parses the enable flag from the given environment variable, defaulting to false.

func PreviousHistoricalRevisionKey

func PreviousHistoricalRevisionKey(ctx context.Context, pm datastore.PropertyMap) *datastore.Key

PreviousHistoricalRevisionKey gets the key for the previous history entity of the given historical entity PropertyMap.

func ReplicasRootKey

func ReplicasRootKey(ctx context.Context) *datastore.Key

ReplicasRootKey is the root key for AuthReplicaState entities. The entity itself doesn't exist.

func ReplicateToReplica

func ReplicateToReplica(ctx context.Context, r *AuthReplicaState,
	authDBRev int64, authDBBlob []byte, keyName, encodedSig string) error

ReplicateToReplica pushes a revision of the AuthDB to the given replica, then updates the last known state of the replica.

Returns an error if: - the push to the replica cannot be considered successful; or - there was an error updating the last known state of the replica.

func RevokeStaleReaderAccess

func RevokeStaleReaderAccess(ctx context.Context, trustedGroup string, dryRun bool) error

RevokeStaleReaderAccess deletes any AuthDBReaders that are no longer in the given trusted group, then updates Auth Service's Google Storage object ACLs.

func RootKey

func RootKey(ctx context.Context) *datastore.Key

RootKey gets the root key of the entity group with all AuthDB entities.

func StoreAuthDBSnapshot

func StoreAuthDBSnapshot(ctx context.Context, replicationState *AuthReplicationState, authDBBlob []byte, dryRun bool) (err error)

StoreAuthDBSnapshot stores the AuthDB blob (serialized proto) into Datastore.

Args:

  • replicationState: the AuthReplicationState corresponding to the given authDBBlob.
  • authDBBlob: serialized protocol.ReplicationPushRequest message (has AuthDB inside).

func UpdateAllowlistEntities

func UpdateAllowlistEntities(ctx context.Context, subnetMap map[string][]string, dryRun bool, historicalComment string) error

UpdateAllowlistEntities updates all the entities in datastore from ip_allowlist.cfg.

TODO(crbug/1336135): Remove dryrun checks when turning off Python Auth Service.

func UpdateAuthGlobalConfig

func UpdateAuthGlobalConfig(ctx context.Context, oauthcfg *configspb.OAuthConfig, seccfg *protocol.SecurityConfig, dryRun bool, historicalComment string) error

UpdateAuthGlobalConfig updates the AuthGlobalConfig datastore entity. If there is no AuthGlobalConfig entity present in the datastore, one will be created.

TODO(crbug/1336135): Remove dryrun checks when turning off Python Auth Service.

func UpdateAuthProjectRealms

func UpdateAuthProjectRealms(ctx context.Context, eRealms []*ExpandedRealms, permsRev string, dryRun bool, historicalComment string) error

UpdateAuthProjectRealms updates all the realms for a specific LUCI Project. If an entity does not exist in datastore for a given AuthProjectRealm, one will be created.

Returns

datastore.ErrNoSuchEntity -- entity does not exist
Annotated Error
	Failed to create new AuthProjectRealm entity
	Failed to update AuthProjectRealm entity
	Failed to put AuthProjectRealmsMeta entity

func UpdateAuthRealmsGlobals

func UpdateAuthRealmsGlobals(ctx context.Context, permsCfg *configspb.PermissionsConfig, dryRun bool, historicalComment string) error

UpdateAuthRealmsGlobals updates the AuthRealmsGlobals singleton entity in datastore, creating the entity if necessary.

Returns

formatted error if stored permissions and permissions.cfg don't match
proto unmarshalling error if format in datastore is incorrect.
internal annotated error for all other

NOTE: Since the original property that is used to store permissions was of type []*protocol.Permission, we have to convert twice. This is because slices of pointers are not supported in luci-go gae/datastore package and it converts to a weird format when using datastore.Get. As a result we will also use the dryRun flag to just log the results until we don't depend on it, then once we switch we'll just stop writing to the entity.

TODO(crbug/1336137): Remove all references and checks to old permissions once Python version knows how to work with permissions.cfg.

Types

type AuthDBChange

type AuthDBChange struct {
	Kind   string         `gae:"$kind,AuthDBChange"`
	ID     string         `gae:"$id"`
	Parent *datastore.Key `gae:"$parent"`

	Class []string `gae:"class"` // A list of "class" names giving the NDB Polymodel hierarchy

	// Fields common across all change types.
	ChangeType     ChangeType `gae:"change_type" json:"change_type"` // What kind of a change this is (see Change*)
	Target         string     `gae:"target" json:"target"`           // Entity (or subentity) that was changed: kind$id[$subid] (subid is optional).
	AuthDBRev      int64      `gae:"auth_db_rev" json:"auth_db_rev"` // AuthDB revision at which the change was made.
	Who            string     `gae:"who" json:"who"`                 // Who made the change.
	When           time.Time  `gae:"when" json:"when"`               // When the change was made.
	Comment        string     `gae:"comment" json:"comment"`         // Comment passed to record_revision or record_deletion.
	AppVersion     string     `gae:"app_version" json:"app_version"` // GAE application version at which the change was made.
	Description    string     `gae:"description,noindex" json:"description"`
	OldDescription string     `gae:"old_description,noindex" json:"old_description"`

	// Fields specific to AuthDBGroupChange.
	Owners    string   `gae:"owners" json:"owners"`         // Valid for ChangeGroupCreated and ChangeGroupOwnersChanged.
	OldOwners string   `gae:"old_owners" json:"old_owners"` // Valid for ChangeGroupOwnersChanged and ChangeGroupDeleted.
	Members   []string `gae:"members" json:"members"`       // Valid for ChangeGroupMembersAdded and ChangeGroupMembersRemoved.
	Globs     []string `gae:"globs" json:"globs"`           // Valid for ChangeGroupGlobsAdded and ChangeGroupGlobsRemoved.
	Nested    []string `gae:"nested" json:"nested"`         // Valid for ChangeGroupNestedAdded and ChangeGroupNestedRemoved.

	// Fields specific to AuthDBIPAllowlistChange.
	Subnets []string `gae:"subnets" json:"subnets"` // Valid for ChangeIPWLSubnetsAdded and ChangeIPWLSubnetsRemoved.

	// Fields specific to AuthDBIPAllowlistAssignmentChange.
	Identity    string `gae:"identity"` // Valid for ChangeIPWLAssignSet and ChangeIPWLAssignUnset.
	IPAllowlist string `gae:"ip_whitelist"`

	// Fields specific to AuthDBConfigChange.
	OauthClientID            string   `gae:"oauth_client_id" json:"oauth_client_id"`                         // Valid for ChangeConfOauthClientChanged.
	OauthClientSecret        string   `gae:"oauth_client_secret" json:"oauth_client_secret"`                 // Valid for ChangeConfOauthClientChanged.
	OauthAdditionalClientIDs []string `gae:"oauth_additional_client_ids" json:"oauth_additional_client_ids"` // Valid for ChangeConfClientIDsAdded and ChangeConfClientIDsRemoved.
	TokenServerURLOld        string   `gae:"token_server_url_old" json:"token_server_url_old"`               // Valid for ChangeConfTokenServerURLChanged.
	TokenServerURLNew        string   `gae:"token_server_url_new" json:"token_server_url_new"`               // Valid for ChangeConfTokenServerURLChanged.
	SecurityConfigOld        []byte   `gae:"security_config_old,noindex" json:"security_config_old"`         // Valid for ChangeConfSecurityConfigChanged.
	SecurityConfigNew        []byte   `gae:"security_config_new,noindex" json:"security_config_new"`         // Valid for ChangeConfSecurityConfigChanged.

	// Fields specific to AuthRealmsGlobalsChange.
	PermissionsAdded   []string `gae:"permissions_added" json:"permissions_added"`
	PermissionsChanged []string `gae:"permissions_changed" json:"permissions_changed"`
	PermissionsRemoved []string `gae:"permissions_removed" json:"permissions_removed"`

	// Fields specific to AuthProjectRealmsChange.
	ConfigRevOld string `gae:"config_rev_old,noindex"`
	ConfigRevNew string `gae:"config_rev_new,noindex"`
	PermsRevOld  string `gae:"perms_rev_old,noindex"`
	PermsRevNew  string `gae:"perms_rev_new,noindex"`
}

AuthDBChange is the base (embedded) struct for change log entries. Has a change type and a bunch of common change properties (like who and when made the change). Change type order is important, it is used in UI when sorting changes introduces by single AuthDB commit.

Change types represent minimal indivisible changes. Large AuthDB change is usually represented by (unordered) set of indivisible changes. For example, an act of creation of a new AuthDB group produces following changes:

ChangeGroupCreated
ChangeGroupMembersAdded
ChangeGroupGlobsAdded
ChangeGroupNestedAdded

They are unordered, but UI sorts them based on change type integer. Thus ChangeGroupCreated appears on top: it's represented by the smallest integer.

Entity id has following format:

<original_entity_kind>$<original_id>[$<subentity_id>]!<change_type>

where:

original_entity_kind: a kind of modified AuthDB entity (e.g. 'AuthGroup')
original_id: ID of modified AuthDB entity (e.g. 'Group name')
subentity_id: optional identified of modified part of the entity, used for
  IP allowlist assignments entity (since it's just one big singleton).
change_type: integer Change* (see below), e.g. '1100'.

Such key structure makes 'diff_entity_by_key' operation idempotent. A hash of entity body could have been used too, but having readable (and sortable) keys are nice.

Parent entity is AuthDBRevision(ChangeLogRevisionKey).

Note: '$' and '!' are not likely to appear in entity names since they are forbidden in AuthDB names. Code here also asserts this.

func GetAllAuthDBChange

func GetAllAuthDBChange(ctx context.Context, target string, authDBRev int64, pageSize int32, pageToken string) (changes []*AuthDBChange, nextPageToken string, err error)

GetAllAuthDBChange returns all the AuthDBChange entities with given target and authDBRev. If target is an empty string/authDBRev equals to 0, no target/authDBRev is specified.

Returns an annotated error.

func (*AuthDBChange) ToProto

func (change *AuthDBChange) ToProto() *rpcpb.AuthDBChange

type AuthDBLogRev

type AuthDBLogRev struct {
	Kind   string         `gae:"$kind,AuthDBLogRev"`
	ID     int64          `gae:"$id"` // The AuthDB revision.
	Parent *datastore.Key `gae:"$parent"`

	When       time.Time `gae:"when" json:"when"`               // When the changes were processed.
	AppVersion string    `gae:"app_version" json:"app_version"` // GAE application version that processed the change.
}

AuthDBLogRev is used to record that the changelog was generated for an AuthDB revision.

type AuthDBReader

type AuthDBReader struct {
	// Account that should be able to read AuthDB Google Storage dump.
	//
	// These are accounts that have explicitly requested access to the
	// AuthDB via API call to
	// `/auth_service/api/v1/authdb/subscription/authorization`.
	//
	// They all belong to the `auth-trusted-services` group.
	// Note that we can't just authorize all members of
	// `auth-trusted-services` since we can't enumerate them
	// (for example, there's no way to enumerate glob entries like
	// *@example.com).
	Kind   string         `gae:"$kind,AuthDBReader"`
	Parent *datastore.Key `gae:"$parent"`

	// The account email.
	ID string `gae:"$id"`

	//AuthorizedTS is the time this account was last authorized.
	AuthorizedTS time.Time `gae:"authorized_at,noindex"`
}

type AuthDBShard

type AuthDBShard struct {
	Kind string `gae:"$kind,AuthDBShard"`
	ID   string `gae:"$id"`

	// Blob represents a sharded part of the full AuthDB blob.
	Blob []byte `gae:"blob,noindex"`
}

AuthDBShard is a shard of deflated serialized ReplicationPushRequest. Root entity. ID is "<auth_db_revision:<blob hash>

type AuthDBSnapshot

type AuthDBSnapshot struct {
	Kind string `gae:"$kind,AuthDBSnapshot"`
	ID   int64  `gae:"$id"`

	// AuthDBDeflated is the deflated serialized ReplicationPushRequest proto message.
	AuthDBDeflated []byte `gae:"auth_db_deflated,noindex"`

	// ShardIDs is a list of shard IDs if sharded or empty if auth_db_deflated should be used.
	ShardIDs []string `gae:"shard_ids,noindex"`

	// AuthDBSha256 is the SHA256 hex digest of auth_db (before compression).
	AuthDBSha256 string `gae:"auth_db_sha256,noindex"`

	// CreatedTS is when this snapshot was created.
	CreatedTS time.Time `gae:"created_ts"`
}

AuthDBSnapshot Contains deflated serialized ReplicationPushRequest for some revision. Root entity. ID is corresponding revision number (as integer). Immutable.

func GetAuthDBSnapshot

func GetAuthDBSnapshot(ctx context.Context, rev int64, skipBody bool, dryRun bool) (*AuthDBSnapshot, error)

GetAuthDBSnapshot returns the AuthDBSnapshot datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthDBSnapshot is not present. Returns an annotated error for other errors.

func (*AuthDBSnapshot) ToProto

func (snapshot *AuthDBSnapshot) ToProto() *rpcpb.Snapshot

ToProto converts the AuthDBSnapshot entity to the protobuffer equivalent Snapshot.

type AuthDBSnapshotLatest

type AuthDBSnapshotLatest struct {
	Kind string `gae:"$kind,AuthDBSnapshotLatest"`
	ID   string `gae:"$id"`

	// AuthDBRev is the revision number of latest stored AuthDBSnaphost. Monotonically increases.
	AuthDBRev int64 `gae:"auth_db_rev,noindex"`

	// AuthDBSha256 is the SHA256 hex digest of auth_db (before compression).
	AuthDBSha256 string `gae:"auth_db_sha256,noindex"`

	// CreatedTS is when this snapshot was created.
	ModifiedTS time.Time `gae:"modified_ts,noindex"`
}

AuthDBSnapshotLatest is a Pointer to latest stored AuthDBSnapshot entity. Exists in single instance with key ('AuthDBSnapshotLatest', 'latest').

func GetAuthDBSnapshotLatest

func GetAuthDBSnapshotLatest(ctx context.Context, dryRun bool) (*AuthDBSnapshotLatest, error)

GetAuthDBSnapshotLatest returns the AuthDBSnapshotLatest datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthDBSnapshotLatest is not present. Returns an annotated error for other errors.

type AuthGlobalConfig

type AuthGlobalConfig struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthGlobalConfig"`
	ID   string `gae:"$id,root"`

	// OAuthClientID is an OAuth2 client_id to use to mint new OAuth2 tokens.
	OAuthClientID string `gae:"oauth_client_id,noindex"`

	// OAuthAdditionalClientIDs are the additional client ID's allowed to access
	// the service.
	OAuthAdditionalClientIDs []string `gae:"oauth_additional_client_ids,noindex"`

	// OAuthClientSecret is passed to clients.
	OAuthClientSecret string `gae:"oauth_client_secret,noindex"`

	// TokenServerURL is the URL of the token server to use to generate delegation
	// tokens.
	TokenServerURL string `gae:"token_server_url,noindex"`

	// SecurityConfig is serialized security config from security_config.proto.
	//
	// See https://source.chromium.org/chromium/infra/infra/+/main:luci/appengine/components/components/auth/proto/security_config.proto.
	SecurityConfig []byte `gae:"security_config,noindex"`
}

AuthGlobalConfig is the root entity for auth models. There should be only one instance of this model in datastore.

func GetAuthGlobalConfig

func GetAuthGlobalConfig(ctx context.Context) (*AuthGlobalConfig, error)

GetAuthGlobalConfig returns the AuthGlobalConfig datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthGlobalConfig is not present. Returns an annotated error for other errors.

type AuthGroup

type AuthGroup struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthGroup"`
	ID   string `gae:"$id"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// Members is the list of members that are explicitly in this group.
	Members []string `gae:"members"`

	// Globs is the list of identity-glob expressions in this group.
	//
	// Example: ('user@example.com').
	Globs []string `gae:"globs"`

	// Nested is the list of nested group names.
	Nested []string `gae:"nested"`

	// Description is a human readable description of this group.
	Description string `gae:"description,noindex"`

	// Owners is the name of the group that owns this group.
	// Owners can modify or delete this group.
	Owners string `gae:"owners"`

	// CreatedTS is the time when this group was created, in UTC.
	CreatedTS time.Time `gae:"created_ts"`

	// CreatedBy is the email of the user who created this group.
	CreatedBy string `gae:"created_by"`
}

AuthGroup is a group of identities, the entity id is the group name.

func AuthGroupFromProto

func AuthGroupFromProto(ctx context.Context, grouppb *rpcpb.AuthGroup) *AuthGroup

AuthGroupFromProto allocates a new AuthGroup using the values in the supplied protobuffer.

func CreateAuthGroup

func CreateAuthGroup(ctx context.Context, group *AuthGroup, external bool, historicalComment string, dryRun bool) (*AuthGroup, error)

CreateAuthGroup creates a new AuthGroup and writes it to the datastore. Only the following fields will be read from the input: ID, Description, Owners, Members, Globs, Nested.

func GetAllAuthGroups

func GetAllAuthGroups(ctx context.Context) ([]*AuthGroup, error)

GetAllAuthGroups returns all the AuthGroups from the datastore.

Returns an annotated error.

func GetAuthGroup

func GetAuthGroup(ctx context.Context, groupName string) (*AuthGroup, error)

GetAuthGroup gets the AuthGroup with the given id(groupName).

Returns datastore.ErrNoSuchEntity if the group given is not present. Returns an annotated error for other errors.

func UpdateAuthGroup

func UpdateAuthGroup(ctx context.Context, groupUpdate *AuthGroup, updateMask *fieldmaskpb.FieldMask, etag string, fromExternal bool, historicalComment string, dryRun bool) (*AuthGroup, error)

UpdateAuthGroup updates the given auth group. The ID of the AuthGroup is used to determine which group to update. The field mask determines which fields of the AuthGroup will be updated. If the field mask is nil, all fields will be updated.

Possible errors:

ErrInvalidArgument if the field mask provided is invalid.
ErrInvalidIdentity if any identities or globs specified in the update are invalid.
datastore.ErrNoSuchEntity if the specified group does not exist.
ErrPermissionDenied if the caller is not allowed to update the group.
ErrConcurrentModification if the provided etag is not up-to-date.
Annotated error for other errors.

func (*AuthGroup) ToProto

func (group *AuthGroup) ToProto(includeMemberships bool) *rpcpb.AuthGroup

ToProto converts the AuthGroup entity to the protobuffer equivalent.

Set includeMemberships to true if we also want to include the group members, or false if we only want the metadata (e.g. if we're just listing groups).

type AuthIPAllowlist

type AuthIPAllowlist struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthIPWhitelist"`
	ID   string `gae:"$id"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// Subnets is the list of subnets for this allowlist.
	Subnets []string `gae:"subnets"`

	// Description is a human readable description of this allowlist.
	Description string `gae:"description,noindex"`

	// CreatedTS is the time when this allowlist was created, in UTC.
	CreatedTS time.Time `gae:"created_ts"`

	// CreatedBy is the email of the user who created this allowlist.
	CreatedBy string `gae:"created_by"`
}

func GetAllAuthIPAllowlists

func GetAllAuthIPAllowlists(ctx context.Context) ([]*AuthIPAllowlist, error)

GetAllAuthIPAllowlists returns all the AuthIPAllowlist entities from the datastore, these are stored as AuthIPWhitelist in the datastore.

Returns an annotated error.

func GetAuthIPAllowlist

func GetAuthIPAllowlist(ctx context.Context, allowlistName string) (*AuthIPAllowlist, error)

GetAuthIPAllowlist gets the AuthIPAllowlist with the given allowlist name.

Returns datastore.ErrNoSuchEntity if the allowlist given is not present. Returns an annotated error for other errors.

func (*AuthIPAllowlist) ToProto

func (allowlist *AuthIPAllowlist) ToProto() *rpcpb.Allowlist

ToProto converts the AuthIPAllowlist entity to the protobuffer equivalent Allowlist.

type AuthProjectRealms

type AuthProjectRealms struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthProjectRealms"`
	ID   string `gae:"$id"`

	Parent *datastore.Key `gae:"$parent"`

	// Realms is all the realms of a project, see comment for AuthProjectRealms.
	Realms []byte `gae:"realms,noindex"`

	// ConfigRev is the git revision the config was picked up from.
	ConfigRev string `gae:"config_rev,noindex"`

	// PermsRev is the revision of permissions DB used to expand roles.
	PermsRev string `gae:"perms_rev,noindex"`
}

AuthProjectRealms is all realms of a single LUCI project.

They are defined as Realms proto message reduced to a single project:

  • Only project's realms are listed in `realms` field.
  • Only permissions used by the project are listed in `permissions` field.
  • Permissions have their metadata stripped, they have only names.

func GetAllAuthProjectRealms

func GetAllAuthProjectRealms(ctx context.Context) ([]*AuthProjectRealms, error)

GetAllAuthProjectRealms returns all the AuthProjectRealms entities in datastore.

Returns an annotated error.

func GetAuthProjectRealms

func GetAuthProjectRealms(ctx context.Context, project string) (*AuthProjectRealms, error)

GetAuthProjectRealms returns the AuthProjectRealms datastore entity for a given project.

Returns datastore.ErrNoSuchEntity if the AuthProjectRealms entity is not present. Returns an annotated error for other errors.

func (*AuthProjectRealms) RealmsToProto

func (apr *AuthProjectRealms) RealmsToProto() (projectRealms *protocol.Realms, err error)

RealmsToProto processes the Realms field and returns it as a proto.

Returns an annotated error if one occurs.

type AuthProjectRealmsMeta

type AuthProjectRealmsMeta struct {
	Kind string `gae:"$kind,AuthProjectRealmsMeta"`
	ID   string `gae:"$id,meta"`

	Parent *datastore.Key `gae:"$parent"`

	// ConfigRev is the revision the config was picked up from.
	ConfigRev string `gae:"config_rev,noindex"`

	// PermsRev is the revision of permissions.cfg used to expand roles.
	PermsRev string `gae:"perms_rev,noindex"`

	// ConfigDigest is a SHA256 digest of the raw config body.
	ConfigDigest string `gae:"config_digest,noindex"`

	// ModifiedTS is when it was last updated, (mostly FYI).
	ModifiedTS time.Time `gae:"modified_ts,noindex"`
}

AuthProjectRealmsMeta is the metadata of some AuthProjectRealms entity.

Always created/deleted/updated transactionally with the corresponding AuthProjectRealms entity, but it is not part of AuthDB itself.

Used to hold bookkeeping state related to realms.cfg processing. Can be fetched very efficiently (compared to fetching AuthProjectRealms).

ID is always meta, the parent entity is the corresponding AuthProjectRealms.

func GetAllAuthProjectRealmsMeta

func GetAllAuthProjectRealmsMeta(ctx context.Context) ([]*AuthProjectRealmsMeta, error)

GetAllAuthProjectRealmsMeta returns all the AuthProjectRealmsMeta entities in datastore.

Returns an annotated error.

func GetAuthProjectRealmsMeta

func GetAuthProjectRealmsMeta(ctx context.Context, project string) (*AuthProjectRealmsMeta, error)

GetAuthProjectRealmsMeta returns the AuthProjectRealmsMeta datastore entity for a given project.

Returns datastore.ErrNoSuchEntity if the AuthProjectRealmsMeta is not present. Returns an annotated error for other errors.

func (*AuthProjectRealmsMeta) ProjectID

func (aprm *AuthProjectRealmsMeta) ProjectID() (string, error)

ProjectID returns the project that an AuthProjectRealmsMeta is connected to.

type AuthRealmsGlobals

type AuthRealmsGlobals struct {
	// AuthVersionedEntityMixin is embedded
	// to include modification details related to this entity.
	AuthVersionedEntityMixin

	Kind string `gae:"$kind,AuthRealmsGlobal"`
	ID   string `gae:"$id,globals"`

	Parent *datastore.Key `gae:"$parent"`

	// TODO(cjacomet): Remove this once Python version no longer depends on
	// this.
	//
	// Permissions and PermissionsList are the same, they are just represented
	// differently in datastore from Python to Golang. In Python lists of pointers
	// to protobuf types would serialize automagically and store as an array of
	// serialized strings in datastore. In Go this is not supported in the luci datastore
	// package. It is better to define the list as repeated protobuf.Msg instead of
	// []*protobuf.Msg. This will yield the same results in the end but with a bit different steps.
	// Permissions will not be unmarshalled in this version, the declaration
	// is to avoid any issues when fetching the entity from datastore.
	//
	// Permissions is all globally defined permissions, in alphabetical order.
	Permissions []string `gae:"permissions,noindex"`

	// PermissionsList is all globally defined permissions, in alphabetical order.
	PermissionsList *permissions.PermissionsList `gae:"permissionslist"`
}

AuthRealmsGlobals is a singleton entity with global portions of realms configuration.

Data here does not relate to any individual realm or project. Currently contains a list of all defined permissions (with their metadata).

func GetAuthRealmsGlobals

func GetAuthRealmsGlobals(ctx context.Context) (*AuthRealmsGlobals, error)

GetAuthRealmsGlobals returns the AuthRealmsGlobals singleton datastore entity.

Returns datastore.ErrNoSuchEntity if the AuthRealmsGlobals entity is not present. Returns an annotated error for other errors.

type AuthReplicaState

type AuthReplicaState struct {
	Kind string `gae:"$kind,AuthReplicaState"`
	// ID is GAE Application ID of the Replica.
	ID string `gae:"$id,self"`
	// Parent is replicasRootKey().
	Parent *datastore.Key `gae:"$parent"`

	// URL of a host to push updates to.
	ReplicaURL string `gae:"replica_url,noindex"`
	//Revision of auth DB replica is synced to.
	AuthDBRev int64 `gae:"auth_db_rev,noindex"`
	// Time when AuthDBRev was created (by primary clock).
	RevModifiedTS time.Time `gae:"rev_modified_ts,noindex"`
	// Value of components.auth.version.__version__ used by replica.
	AuthCodeVersion string `gae:"auth_code_version,noindex"`

	// Timestamp of when last push attempt started.
	PushStartedTS time.Time `gae:"push_started_ts,noindex"`
	// Timestamp of when last push attempt finished (successfully or not).
	PushFinishedTS time.Time `gae:"push_finished_ts,noindex"`
	// Status of last push attempt. See ReplicaPushStatus* enumeration.
	PushStatus ReplicaPushStatus `gae:"push_status,noindex"`
	// Error message of last push attempt, or empty string if it was successful.
	PushError string `gae:"push_error,noindex"`
}

AuthReplicaState is the last known state of a Replica as known by Auth Service.

func GetAllReplicas

func GetAllReplicas(ctx context.Context) ([]*AuthReplicaState, error)

GetAllReplicas gets all the AuthReplicaState entities.

func GetAllStaleReplicas

func GetAllStaleReplicas(ctx context.Context, authDBRev int64) ([]*AuthReplicaState, error)

GetAllStaleReplicas gets all the AuthReplicaState entities that are behind the given authDBRev.

func (*AuthReplicaState) ToProto

func (r *AuthReplicaState) ToProto() *rpcpb.ReplicaState

type AuthReplicationState

type AuthReplicationState struct {
	Kind string `gae:"$kind,AuthReplicationState"`
	ID   string `gae:"$id,self"`

	// Parent is RootKey().
	Parent *datastore.Key `gae:"$parent"`

	// AuthDBRev is increased by 1 with every change to the AuthDB that should be
	// distributed to linked services.
	AuthDBRev int64 `gae:"auth_db_rev,noindex"`

	// ModifiedTS is the time this entity was last modified.
	ModifiedTS time.Time `gae:"modified_ts,noindex"`

	PrimaryID  string   `gae:"primary_id,noindex"`
	PrimaryURL string   `gae:"primary_url,noindex"`
	ShardIDs   []string `gae:"shard_ids,noindex"`
}

AuthReplicationState contains revision number incremented each time a part of AuthDB changes.

func GetReplicationState

func GetReplicationState(ctx context.Context) (*AuthReplicationState, error)

GetReplicationState fetches AuthReplicationState from the datastore.

Returns datastore.ErrNoSuchEntity if it is missing.

type AuthVersionedEntityMixin

type AuthVersionedEntityMixin struct {
	// ModifiedTS is the time this entity was last modified.
	ModifiedTS time.Time `gae:"modified_ts"`

	// ModifiedBy specifies who mode the modification to this entity.
	ModifiedBy string `gae:"modified_by"`

	// AuthDBRev is the revision number that this entity was last modified at.
	AuthDBRev int64 `gae:"auth_db_rev"`

	// AuthDBPrevRev is revision number of the previous version of this entity.
	AuthDBPrevRev int64 `gae:"auth_db_prev_rev"`
}

AuthVersionedEntityMixin is for AuthDB entities that keep track of when they change.

type ChangeType

type ChangeType int64

ChangeType is the enum for AuthDBChange.ChangeType.

const (
	// AuthDBGroupChange change types.
	ChangeGroupCreated            ChangeType = 1000
	ChangeGroupDescriptionChanged ChangeType = 1100
	ChangeGroupOwnersChanged      ChangeType = 1150
	ChangeGroupMembersAdded       ChangeType = 1200
	ChangeGroupMembersRemoved     ChangeType = 1300
	ChangeGroupGlobsAdded         ChangeType = 1400
	ChangeGroupGlobsRemoved       ChangeType = 1500
	ChangeGroupNestedAdded        ChangeType = 1600
	ChangeGroupNestedRemoved      ChangeType = 1700
	ChangeGroupDeleted            ChangeType = 1800

	// AuthDBIPAllowlistChange change types.
	ChangeIPALCreated            ChangeType = 3000
	ChangeIPALDescriptionChanged ChangeType = 3100
	ChangeIPALSubnetsAdded       ChangeType = 3200
	ChangeIPALSubnetsRemoved     ChangeType = 3300
	ChangeIPALDeleted            ChangeType = 3400

	// AuthDBIPAllowlistAssignmentChange change types.
	ChangeIPALAssignSet   ChangeType = 5000
	ChangeIPALAssignUnset ChangeType = 5100

	// AuthDBConfigChange change types.
	ChangeConfOauthClientChanged    ChangeType = 7000
	ChangeConfClientIDsAdded        ChangeType = 7100
	ChangeConfClientIDsRemoved      ChangeType = 7200
	ChangeConfTokenServerURLChanged ChangeType = 7300
	ChangeConfSecurityConfigChanged ChangeType = 7400

	// AuthRealmsGlobalsChange change types.
	ChangeRealmsGlobalsChanged ChangeType = 9000

	// AuthProjectRealmsChange change types.
	ChangeProjectRealmsCreated     ChangeType = 10000
	ChangeProjectRealmsChanged     ChangeType = 10100
	ChangeProjectRealmsReevaluated ChangeType = 10200
	ChangeProjectRealmsRemoved     ChangeType = 10300
)

func (ChangeType) ToString

func (ct ChangeType) ToString() string

type ExpandedRealms

type ExpandedRealms struct {
	CfgRev *RealmsCfgRev
	// Realms is the expanded form of the realms.cfg for a project
	Realms *protocol.Realms
}

ExpandedRealms contains the expanded realms and information about the expanded realms.

type GroupBundle

type GroupBundle = map[string][]identity.Identity

GroupBundle is a map where k: groupName, v: list of identities belonging to group k.

type GroupImporterConfig

type GroupImporterConfig struct {
	Kind string `gae:"$kind,GroupImporterConfig"`
	ID   string `gae:"$id,config"`

	// ConfigProto is the plaintext copy of the config found at imports.cfg.
	ConfigProto string `gae:"config_proto"`

	// ConfigRevision is revision version of the config found at imports.cfg.
	ConfigRevision []byte `gae:"config_revision"`

	// ModifiedBy is the email of the user who modified the cfg.
	ModifiedBy string `gae:"modified_by"`

	// ModifiedTS is the time when this entity was last modified.
	ModifiedTS time.Time `gae:"modified_ts"`
}

GroupImporterConfig is a singleton entity that contains the contents of the imports.cfg file.

func GetGroupImporterConfig

func GetGroupImporterConfig(ctx context.Context) (*GroupImporterConfig, error)

GetGroupImporterConfig fetches the GroupImporterConfig entity from the datastore.

Returns GroupImporterConfig entity if present.
Returns datastore.ErrNoSuchEntity if the entity is not present.
Returns annotated error for all other errors.

func (*GroupImporterConfig) ToProto

ToProto converts the GroupImporterConfig entity to the proto equivalent.

type RealmsCfgRev

type RealmsCfgRev struct {
	ProjectID    string
	ConfigRev    string
	ConfigDigest string

	// These two are mutually exclusive.
	ConfigBody []byte
	PermsRev   string
}

RealmsCfgRev is information about fetched or previously processed realms.cfg. Comes either from LUCI Config (then `ConfigBody` is set, but `PermsRev` isn't) or from the datastore (then `PermsRev` is set, but `ConfigBody` isn't). All other fields are always set.

type ReplicaPushStatus

type ReplicaPushStatus int

ReplicaPushStatus is an enum for the replica push attempt status.

const (
	ReplicaPushStatusSuccess ReplicaPushStatus = iota
	ReplicaPushStatusTransientError
	ReplicaPushStatusFatalError
)

The possible values for the PushStatus field of AuthReplicaState.

type ReplicaUpdateError

type ReplicaUpdateError struct {
	RootErr error
	IsFatal bool
}

ReplicaUpdateError wraps an error that occurred when attempting to directly push a new AuthDB revision to a replica.

func (*ReplicaUpdateError) Error

func (r *ReplicaUpdateError) Error() string

func (*ReplicaUpdateError) Is

func (r *ReplicaUpdateError) Is(target error) bool

func (*ReplicaUpdateError) Unwrap

func (r *ReplicaUpdateError) Unwrap() error

type Snapshot

type Snapshot struct {
	ReplicationState *AuthReplicationState
	GlobalConfig     *AuthGlobalConfig
	Groups           []*AuthGroup
	IPAllowlists     []*AuthIPAllowlist
	RealmsGlobals    *AuthRealmsGlobals
	ProjectRealms    []*AuthProjectRealms
}

Snapshot contains transactionally captured AuthDB entities.

func TakeSnapshot

func TakeSnapshot(ctx context.Context) (snap *Snapshot, err error)

TakeSnapshot takes a consistent snapshot of the replicated subset of AuthDB entities.

Runs a read-only transaction internally.

func (*Snapshot) ToAuthDB

func (s *Snapshot) ToAuthDB(useV1Perms bool) (*authdb.SnapshotDB, error)

ToAuthDB converts the snapshot to an authdb.SnapshotDB.

It then can be used by the auth service itself to make ACL checks.

func (*Snapshot) ToAuthDBProto

func (s *Snapshot) ToAuthDBProto(useV1Perms bool) (*protocol.AuthDB, error)

ToAuthDBProto converts the snapshot to an AuthDB proto message.

Directories

Path Synopsis
Package graph contains groups graph definitions and operations.
Package graph contains groups graph definitions and operations.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL