README
¶
Firestore Events backend implementation for Teleport.
Introduction
This package enables Teleport auth server to store secrets in Firestore on GCP.
WARNING: Using Firestore events involves recurring charge from GCP.
Building
Firestore events is not enabled by default. To enable it you have to
compile Teleport with firestore build flag.
To build Teleport with Firestore enabled, run:
ADDFLAGS='-tags firestore' make teleport
Quick Start
There are currently two Firestore mode options for any given GCP Project; Native mode and
Datastore Mode. This storage backend uses Real-time updates to keep individual auth instances
in sync and requires Firestore configured in Native mode.
Add this storage configuration in teleport section of the config file (by default it's /etc/teleport.yaml):
teleport:
storage:
audit_events_uri: 'firestore://events?projectID=gcp-proj-with-firestore-enabled&credentialsPath=/var/lib/teleport/gcs_creds'
Collections are automatically created by the Firestore APIs and the required indexes are created by the event backend on first start, if they do not exist.
Full Properties
The full list of configurable properties for this backend are:
- host portion of URI is the Firestore collection used to persist stored events
credentialsPath(string, path to GCP creds for Firestore, not-required)projectID(string, project ID, required)purgeInterval(time duration, poll interval to sweep expired documents, not-required, defaults toonce per minute)retryPeriod(time duration, retry period for all background tasks, not-required, defaults to10 seconds)disableExpiredDocumentPurge(bool, disables expired document purging, not-required, defaults tofalse)eventRetentionPeriod(int, buffer size for watched events, not-required, defaults to1024)endpoint(string, firestore client endpoint, not-required, ex:localhost:8618)
Firestore Client Authentication Options
There are three authentication/authorization modes available;
- With no
credentialsPathand noendpointdefined, the Firestore clients will use Google Application Default Credentials for authentication. This only works in cases where Teleport is installed on GCE instances and have service accounts with IAM role/profile associations authorizing that GCE instance to use Firestore. - With
endpointdefined, Firestore will create clients no auth, GRPC in-secure, clients pointed at the specified endpoint. This is only used for tests, seeTestssection below. - With
credentialsPathdefined, Firestore will create clients authenticating against live systems with the Service Account bound to the JSON key file referenced in the option.
Implementation Details
Firestore Document IDs must be unique, cannot start with periods, and cannot contain forward
slashes. In order to support more straight forward fetching but work within the requirements
of Firestore, Document IDs are the concatenation of the session ID (a UUID) and event type joined with a dash -,
ex: 13498a42-69a8-4fa2-b39d-b0c49e346713-user.login.
Expired event purging should be enabled on as few instances as possible to reduce query costs,
though there's no harm in having every instance query and purge. Purging is enabled based on
the purgeExpiredDocuments property, which defaults to true. Purging is done based on the
configurable eventRetentionPeriod property, which defaults to a year. Add this property to
the URI to change the retention period.
Tests
Tests must execute one of two ways:
- With
gcloudinstalled in test infrastructure and thefirestoreemulator enabled and running to a dynamic port a pre-defined port used in the config. Ex:gcloud beta emulators firestore start --host-port=localhost:8618. This is where the Firestore config parameterendpointis used. - With a service account pointed a test GCP project and or test collections.
Get Help
This backend has been contributed by https://github.com/joshdurbin
Documentation
¶
Overview ¶
Package firestoreeventsLog implements Firestore storage backend for Teleport event storage.
firestoreevents package implements the Log storage back-end for the auth server. Originally contributed by https://github.com/joshdurbin
Index ¶
- type EventsConfig
- type Log
- func (l *Log) Close() error
- func (l *Log) EmitAuditEvent(ctx context.Context, in apievents.AuditEvent) error
- func (l *Log) GetSessionChunk(namespace string, sid session.ID, offsetBytes, maxBytes int) ([]byte, error)
- func (l *Log) GetSessionEvents(namespace string, sid session.ID, after int, inlcudePrintEvents bool) ([]events.EventFields, error)
- func (l *Log) SearchEvents(fromUTC, toUTC time.Time, namespace string, eventTypes []string, limit int, ...) ([]apievents.AuditEvent, string, error)
- func (l *Log) SearchSessionEvents(fromUTC, toUTC time.Time, limit int, order types.EventOrder, startKey string, ...) ([]apievents.AuditEvent, string, error)
- func (l *Log) StreamSessionEvents(ctx context.Context, sessionID session.ID, startIndex int64) (chan apievents.AuditEvent, chan error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type EventsConfig ¶
type EventsConfig struct {
firestorebk.Config
// RetentionPeriod is a default retention period for events
RetentionPeriod time.Duration
// Clock is a clock interface, used in tests
Clock clockwork.Clock
// UIDGenerator is unique ID generator
UIDGenerator utils.UID
}
Config structure represents Firestore configuration as appears in `storage` section of Teleport YAML
func (*EventsConfig) SetFromParams ¶
func (cfg *EventsConfig) SetFromParams(params backend.Params) error
SetFromParams establishes values on an EventsConfig from the supplied params
func (*EventsConfig) SetFromURL ¶
func (cfg *EventsConfig) SetFromURL(url *url.URL) error
SetFromURL establishes values on an EventsConfig from the supplied URI
type Log ¶
type Log struct {
// Entry is a log entry
*log.Entry
// Config is a backend configuration
EventsConfig
// contains filtered or unexported fields
}
Log is a firestore-db backed storage of events
func New ¶
func New(cfg EventsConfig) (*Log, error)
New returns new instance of Firestore backend. It's an implementation of backend API's NewFunc
func (*Log) EmitAuditEvent ¶
EmitAuditEvent emits audit event
func (*Log) GetSessionChunk ¶
func (l *Log) GetSessionChunk(namespace string, sid session.ID, offsetBytes, maxBytes int) ([]byte, error)
GetSessionChunk returns a reader which can be used to read a byte stream of a recorded session starting from 'offsetBytes' (pass 0 to start from the beginning) up to maxBytes bytes.
If maxBytes > MaxChunkBytes, it gets rounded down to MaxChunkBytes
func (*Log) GetSessionEvents ¶
func (l *Log) GetSessionEvents(namespace string, sid session.ID, after int, inlcudePrintEvents bool) ([]events.EventFields, error)
Returns all events that happen during a session sorted by time (oldest first).
after tells to use only return events after a specified cursor Id
This function is usually used in conjunction with GetSessionReader to replay recorded session streams.
func (*Log) SearchEvents ¶
func (l *Log) SearchEvents(fromUTC, toUTC time.Time, namespace string, eventTypes []string, limit int, order types.EventOrder, startKey string) ([]apievents.AuditEvent, string, error)
SearchEvents is a flexible way to find events.
Event types to filter can be specified and pagination is handled by an iterator key that allows a query to be resumed.
The only mandatory requirement is a date range (UTC).
This function may never return more than 1 MiB of event data.
func (*Log) SearchSessionEvents ¶
func (l *Log) SearchSessionEvents(fromUTC, toUTC time.Time, limit int, order types.EventOrder, startKey string, cond *types.WhereExpr, sessionID string) ([]apievents.AuditEvent, string, error)
SearchSessionEvents returns session related events only. This is used to find completed sessions.
func (*Log) StreamSessionEvents ¶
func (l *Log) StreamSessionEvents(ctx context.Context, sessionID session.ID, startIndex int64) (chan apievents.AuditEvent, chan error)
StreamSessionEvents streams all events from a given session recording. An error is returned on the first channel if one is encountered. Otherwise the event channel is closed when the stream ends. The event channel is not closed on error to prevent race conditions in downstream select statements.
Source Files