server

package
v0.0.0-...-4a11b79 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 2, 2020 License: Apache-2.0 Imports: 59 Imported by: 0

Documentation

Overview

Package server implements an environment for running LUCI servers.

It interprets command line flags and initializes the serving environment with the following base services available via context.Context:

  • go.chromium.org/luci/common/logging: Logging.
  • go.chromium.org/luci/common/trace: Tracing.
  • go.chromium.org/luci/server/caching: Process cache.
  • go.chromium.org/luci/server/auth: Making authenticated calls.

Other functionality is optional and provided by modules (objects implementing module.Module interface). They should be passed to the server when it starts.

Usage example: 

func main() {
  // When copying, please pick only modules you need.
  modules := []module.Module{
    gaeemulation.NewModuleFromFlags(),
    limiter.NewModuleFromFlags(),
    redisconn.NewModuleFromFlags(),
    secrets.NewModuleFromFlags(),
  }
  server.Main(nil, modules, func(srv *server.Server) error {
    // Initialize global state, change root context (if necessary).
    if err := initializeGlobalStuff(srv.Context); err != nil {
      return err
    }
    srv.Context = injectGlobalStuff(srv.Context)

    // Install regular HTTP routes.
    srv.Routes.GET("/", router.MiddlewareChain{}, func(c *router.Context) {
      // ...
    })

    // Install pRPC services.
    servicepb.RegisterSomeServer(srv.PRPC, &SomeServer{})
    return nil
  })
}

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Main 

func Main(opts *Options, mods []module.Module, init func(srv *Server) error)

Main initializes the server and runs its serving loop until SIGTERM.

Registers all options in the default flag set and uses `flag.Parse` to parse them. If 'opts' is nil, the default options will be used.

Additionally recognizes GAE_APPLICATION env var as an indicator that the server is running on GAE. This slightly tweaks its behavior to match what GAE expects from servers.

On errors, logs them and aborts the process with non-zero exit code.

Types

type Options 

type Options struct {
	Prod     bool   // set when running in production (not on a dev workstation)
	GAE      bool   // set when running on GAE, implies Prod
	Hostname string // used for logging and metric fields, default is os.Hostname

	HTTPAddr  string // address to bind the main listening socket to
	AdminAddr string // address to bind the admin socket to, ignored on GAE

	ShutdownDelay time.Duration // how long to wait after SIGTERM before shutting down

	ClientAuth      clientauth.Options // base settings for client auth options
	TokenCacheDir   string             // where to cache auth tokens (optional)
	AuthDBPath      string             // if set, load AuthDB from a file
	AuthServiceHost string             // hostname of an Auth Service to use
	AuthDBDump      string             // Google Storage path to fetch AuthDB dumps from
	AuthDBSigner    string             // service account that signs AuthDB dumps

	CloudProject string // name of the hosting Google Cloud Project
	CloudRegion  string // name of the hosting Google Cloud region

	TraceSampling string // what portion of traces to upload to Stackdriver (ignored on GAE)

	TsMonAccount     string // service account to flush metrics as
	TsMonServiceName string // service name of tsmon target
	TsMonJobName     string // job name of tsmon target

	ProfilingDisable   bool   // set to true to explicitly disable Stackdriver Profiler
	ProfilingServiceID string // service name to associated with profiles in Stackdriver Profiler

	ContainerImageID string // ID of the container image with this binary, for logs (optional)

	EnableExperiments []string // names of go.chromium.org/luci/server/experiments to enable
	// contains filtered or unexported fields
}

Options are used to configure the server.

Most of them are exposed as command line flags (see Register implementation). Some (mostly GAE-specific) are only settable through code or are derived from the environment.

func (*Options) FromGAEEnv 

func (o *Options) FromGAEEnv()

FromGAEEnv uses the GAE_APPLICATION env var (and other GAE-specific env vars) to configure the server for the GAE environment.

Does nothing if GAE_APPLICATION is not set.

Equivalent to passing the following flags: 

-prod
-http-addr 0.0.0.0:${PORT}
-admin-addr -
-shutdown-delay 0s
-cloud-project ${GOOGLE_CLOUD_PROJECT}
-cloud-region <derived from the region code in GAE_APPLICATION>
-service-account-json :gce
-ts-mon-service-name ${GOOGLE_CLOUD_PROJECT}
-ts-mon-job-name ${GAE_SERVICE}

Additionally the hostname and -container-image-id (used in metric and trace fields) are derived from available GAE_* env vars to be semantically similar to what they represent in the GKE environment.

Note that a mapping between a region code in GAE_APPLICATION and the corresponding cloud region is not documented anywhere, so if you see warnings when your app starts up either update the code to recognize your region code or pass '-cloud-region' argument explicitly in app.yaml.

See https://cloud.google.com/appengine/docs/standard/go/runtime.

func (*Options) Register 

func (o *Options) Register(f *flag.FlagSet)

Register registers the command line flags.

type Port 

type Port struct {
	// Routes is a router for requests hitting this port.
	//
	// This router is used for all requests whose Host header does not match any
	// specially registered per-host routers (see VirtualHost). Normally, there
	// are no such per-host routers, so usually Routes is used for all requests.
	//
	// Should be populated before Server's ListenAndServe call.
	Routes *router.Router
	// contains filtered or unexported fields
}

Port is returned by Server's AddPort and used to setup the request routing.

func (*Port) VirtualHost 

func (p *Port) VirtualHost(host string) *router.Router

VirtualHost returns a router (registering it if necessary) used for requests that have the given Host header.

Note that requests that match some registered virtual host router won't reach the default router (port.Routes), even if the virtual host router doesn't have a route for them. Such requests finish with HTTP 404.

Should be called before Server's ListenAndServe (panics otherwise).

type PortOptions 

type PortOptions struct {
	Name           string // optional logical name of the port for logs
	ListenAddr     string // local address to bind to or "-" for a dummy port
	DisableMetrics bool   // do not collect HTTP metrics for requests on this port
}

PortOptions is a configuration of a single serving HTTP port.

See Server's AddPort.

type Server 

type Server struct {
	// Context is the root context used by all requests and background activities.
	//
	// Can be replaced (by a derived context) before ListenAndServe call, for
	// example to inject values accessible to all request handlers.
	Context context.Context

	// Routes is a router for requests hitting HTTPAddr port.
	//
	// This router is used for all requests whose Host header does not match any
	// specially registered per-host routers (see VirtualHost). Normally, there
	// are no such per-host routers, so usually Routes is used for all requests.
	//
	// This router is also accessible to the server modules and they can install
	// routes into it.
	//
	// Should be populated before ListenAndServe call.
	Routes *router.Router

	// PRPC is pRPC server with APIs exposed on HTTPAddr port via Routes router.
	//
	// Should be populated before ListenAndServe call.
	PRPC *prpc.Server

	// Options is a copy of options passed to New.
	Options Options
	// contains filtered or unexported fields
}

Server is responsible for initializing and launching the serving environment.

Generally assumed to be a singleton: do not launch multiple Server instances within the same process, use AddPort instead if you want to expose multiple ports.

Doesn't do TLS. Should be sitting behind a load balancer that terminates TLS.

func New 

func New(ctx context.Context, opts Options, mods []module.Module) (srv *Server, err error)

New constructs a new server instance.

It hosts one or more HTTP servers and starts and stops them in unison. It is also responsible for preparing contexts for incoming requests.

The given context will become the root context of the server and will be inherited by all handlers.

On errors returns partially initialized server (always non-nil). At least its logging will be configured and can be used to report the error. Trying to use such partially initialized server for anything else is undefined behavior.

func (*Server) AddPort 

func (s *Server) AddPort(opts PortOptions) *Port

AddPort prepares an additional serving HTTP port.

Can be used to open more listening HTTP ports (in addition to opts.HTTPAddr and opts.AdminAddr). The returned Port object can be used to populate the router that serves requests hitting the added port.

If opts.ListenAddr is '-', a dummy port will be added: it is a valid *Port object, but it is not actually exposed as a listening TCP socket. This is useful to disable listening ports without changing any code.

Should be called before ListenAndServe (panics otherwise).

func (*Server) Fatal 

func (s *Server) Fatal(err error)

Fatal logs the error and immediately shuts down the process with exit code 3.

No cleanup is performed. Deferred statements are not run. Not recoverable.

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe() error

ListenAndServe launches the serving loop.

Blocks forever or until the server is stopped via Shutdown (from another goroutine or from a SIGTERM handler). Returns nil if the server was shutdown correctly or an error if it failed to start or unexpectedly died. The error is logged inside.

Should be called only once. Panics otherwise.

func (*Server) RegisterCleanup 

func (s *Server) RegisterCleanup(cb func(context.Context))

RegisterCleanup registers a callback that is run in ListenAndServe after the server has exited the serving loop.

Registering a new cleanup callback from within a cleanup causes a deadlock, don't do that.

func (*Server) RegisterUnaryServerInterceptor 

func (s *Server) RegisterUnaryServerInterceptor(intr grpc.UnaryServerInterceptor)

RegisterUnaryServerInterceptor registers an grpc.UnaryServerInterceptor applied to all unary RPCs that hit the server.

Interceptors are chained in order they are registered, i.e. the first registered interceptor becomes the outermost. The initial chain already contains some base interceptors (e.g. for monitoring) and all interceptors registered by server modules. RegisterUnaryServerInterceptor extends this chain.

An interceptor set in server.PRPC.UnaryServerInterceptor (if any) is automatically registered as the last (innermost) one right before the server starts listening for requests in ListenAndServe.

Should be called before ListenAndServe (panics otherwise).

func (*Server) RunInBackground 

func (s *Server) RunInBackground(activity string, f func(context.Context))

RunInBackground launches the given callback in a separate goroutine right before starting the serving loop.

If the server is already running, launches it right away. If the server fails to start, the goroutines will never be launched.

Should be used for background asynchronous activities like reloading configs.

All logs lines emitted by the callback are annotated with "activity" field which can be arbitrary, but by convention has format "<namespace>.<name>", where "luci" namespace is reserved for internal activities.

The context passed to the callback is canceled when the server is shutting down. It is expected the goroutine will exit soon after the context is canceled.

func (*Server) Shutdown 

func (s *Server) Shutdown()

Shutdown gracefully stops the server if it was running.

Blocks until the server is stopped. Can be called multiple times.

func (*Server) VirtualHost 

func (s *Server) VirtualHost(host string) *router.Router

VirtualHost returns a router (registering it if necessary) used for requests that hit the main port (opts.HTTPAddr) and have the given Host header.

Should be used in rare cases when the server is exposed through multiple domain names and requests should be routed differently based on what domain was used. If your server is serving only one domain name, or you don't care what domain name is used to access it, do not use VirtualHost.

Note that requests that match some registered virtual host router won't reach the default router (server.Routes), even if the virtual host router doesn't have a route for them. Such requests finish with HTTP 404.

Also the router created by VirtualHost is initially completely empty: the server and its modules don't install anything into it (there's intentionally no mechanism to do this). For that reason VirtualHost should never by used to register a router for the "main" domain name: it will make the default server.Routes (and all handlers installed there by server modules) useless, probably breaking the server. Put routes for the main server functionality directly into server.Routes instead, using VirtualHost only for routes that critically depend on Host header.

Should be called before ListenAndServe (panics otherwise).

Source Files

View all Source files

Directories

Path Synopsis
Package analytics provides a standard way to store the Google Analytics tracking ID.
 Package analytics provides a standard way to store the Google Analytics tracking ID.
Package auth implements authentication and authorization framework for HTTP servers.
 Package auth implements authentication and authorization framework for HTTP servers.
authdb
Package authdb contains definition of Authentication Database (aka AuthDB).
 Package authdb contains definition of Authentication Database (aka AuthDB).
authdb/dump
Package dump implements loading AuthDB from dumps in Google Storage.
 Package dump implements loading AuthDB from dumps in Google Storage.
authdb/internal/certs
Package certs knows how to fetch certificate bundles of trusted services.
 Package certs knows how to fetch certificate bundles of trusted services.
authdb/internal/globset
Package globset preprocesses []identity.Glob for faster querying.
 Package globset preprocesses []identity.Glob for faster querying.
authdb/internal/graph
Package graph implements handling of the groups graph.
 Package graph implements handling of the groups graph.
authdb/internal/ipaddr
Package ipaddr implements IP whitelist check.
 Package ipaddr implements IP whitelist check.
authdb/internal/legacy
Package legacy contains older implementation of IsMember check.
 Package legacy contains older implementation of IsMember check.
authdb/internal/oauthid
Package oauthid implements OAuth client ID whitelist check.
 Package oauthid implements OAuth client ID whitelist check.
authdb/internal/realmset
Package realmset provides queryable representation of LUCI Realms DB.
 Package realmset provides queryable representation of LUCI Realms DB.
authdb/internal/seccfg
Package seccfg interprets SecurityConfig proto message.
 Package seccfg interprets SecurityConfig proto message.
authtest
Package authtest implements some interfaces used by auth package to simplify unit testing.
 Package authtest implements some interfaces used by auth package to simplify unit testing.
delegation
Package delegation contains low-level API for working with delegation tokens.
 Package delegation contains low-level API for working with delegation tokens.
delegation/messages
iap
Package iap implements auth.Method for GCP's Identity Aware Proxy.
 Package iap implements auth.Method for GCP's Identity Aware Proxy.
internal
openid
Package openid implements OpenID Connect Login protocol (client side).
 Package openid implements OpenID Connect Login protocol (client side).
realms
Package realms contains functionality related to LUCI Realms.
 Package realms contains functionality related to LUCI Realms.
service
Package service implements a wrapper around API exposed by auth_service: https://github.com/luci/luci-py/tree/master/appengine/auth_service The main focus is AuthDB replication protocol used to propagate changes to database of groups.
 Package service implements a wrapper around API exposed by auth_service: https://github.com/luci/luci-py/tree/master/appengine/auth_service The main focus is AuthDB replication protocol used to propagate changes to database of groups.
service/protocol
signing
Package signing provides interfaces to sign arbitrary small blobs with RSA-SHA256 signature (PKCS1v15) and verify such signatures.
 Package signing provides interfaces to sign arbitrary small blobs with RSA-SHA256 signature (PKCS1v15) and verify such signatures.
signing/signingtest
Package signingtest implements signing.Signer interface using small random keys.
 Package signingtest implements signing.Signer interface using small random keys.
xsrf
Package xsrf provides Cross Site Request Forgery prevention middleware.
 Package xsrf provides Cross Site Request Forgery prevention middleware.
Package caching implements common server object caches.
 Package caching implements common server object caches.
cachingtest
Package cachingtest contains helpers for testing code that uses caching package.
 Package cachingtest contains helpers for testing code that uses caching package.
layered
Package layered provides a two-layer cache for serializable objects.
 Package layered provides a two-layer cache for serializable objects.
cmd
statsd-to-tsmon
Executable statsd-to-tsmon implements a statsd sink that sends aggregated metrics to tsmon.
 Executable statsd-to-tsmon implements a statsd sink that sends aggregated metrics to tsmon.
statsd-to-tsmon/config
tq-sweeper-spanner
Executable tq-sweeper-spanner runs an inproc sweeping driver that scans Spanner database for TQ transactional tasks reminders.
 Executable tq-sweeper-spanner runs an inproc sweeping driver that scans Spanner database for TQ transactional tasks reminders.
Package experiments allow servers to use experimental code paths.
 Package experiments allow servers to use experimental code paths.
Package gaeemulation provides a server module that adds implementation of some https://godoc.org/go.chromium.org/luci/gae APIs to the global server context.
 Package gaeemulation provides a server module that adds implementation of some https://godoc.org/go.chromium.org/luci/gae APIs to the global server context.
Package internal is supporting code used by server.go.
 Package internal is supporting code used by server.go.
Package limiter implements load shedding for servers.
 Package limiter implements load shedding for servers.
Package middleware defines base type for context-aware HTTP request handler.
 Package middleware defines base type for context-aware HTTP request handler.
Package module defines a framework for extending server.Server with optional reusable bundles of functionality (called "modules", naturally).
 Package module defines a framework for extending server.Server with optional reusable bundles of functionality (called "modules", naturally).
Package portal implements HTTP routes for portal pages.
 Package portal implements HTTP routes for portal pages.
internal/assets
Package assets is generated by go.chromium.org/luci/tools/cmd/assets.
 Package assets is generated by go.chromium.org/luci/tools/cmd/assets.
Package pprof is similar to net/http/pprof, except it supports auth.
 Package pprof is similar to net/http/pprof, except it supports auth.
internal
Package redisconn implements integration with a Redis connection pool.
 Package redisconn implements integration with a Redis connection pool.
Package router provides an HTTP router with support for middleware and subrouters.
 Package router provides an HTTP router with support for middleware and subrouters.
Package secrets provides an interface for a simple secret store: you ask it for a secret (a byte blob, identifies by some key), and it returns it to you (current version, as well as a bunch of previous versions).
 Package secrets provides an interface for a simple secret store: you ask it for a secret (a byte blob, identifies by some key), and it returns it to you (current version, as well as a bunch of previous versions).
testsecrets
Package testsecrets provides a dumb in-memory secret store to use in unit tests.
 Package testsecrets provides a dumb in-memory secret store to use in unit tests.
Package settings implements storage for infrequently changing global settings.
 Package settings implements storage for infrequently changing global settings.
Package span implements a server module for communicating with Cloud Spanner.
 Package span implements a server module for communicating with Cloud Spanner.
static
rpcexplorer/test
Package templates implements wrapper around html/template to provide lazy loading of templates and better integration with HTTP middleware framework.
 Package templates implements wrapper around html/template to provide lazy loading of templates and better integration with HTTP middleware framework.
Package tokens provides means to generate and validate base64 encoded tokens compatible with luci-py's components.auth implementation.
 Package tokens provides means to generate and validate base64 encoded tokens compatible with luci-py's components.auth implementation.
tq
Package tq provides a task queue implementation on top of Cloud Tasks.
 Package tq provides a task queue implementation on top of Cloud Tasks.
internal
internal/db
Package db defines common database interface.
 Package db defines common database interface.
internal/lessor
Package lessor defines common lessor interface.
 Package lessor defines common lessor interface.
internal/metrics
internal/partition
Package partition encapsulates partitioning and querying large keyspace which can't be expressed even as uint64.
 Package partition encapsulates partitioning and querying large keyspace which can't be expressed even as uint64.
internal/reminder
Package reminder holds Reminder to avoid circular dependencies.
 Package reminder holds Reminder to avoid circular dependencies.
internal/sweep
internal/testutil
Package testutil provides fakes for testing TQ guts.
 Package testutil provides fakes for testing TQ guts.
internal/tqpb
internal/workset
Package workset contains a synchronized work queue implementation used by inproc sweeper.
 Package workset contains a synchronized work queue implementation used by inproc sweeper.
tqtesting
Package tqtesting contains helpers for running server/tq in tests and on localhost.
 Package tqtesting contains helpers for running server/tq in tests and on localhost.
txn/datastore
Datastore contains Transactional Enqueue support for Cloud Datastore.
 Datastore contains Transactional Enqueue support for Cloud Datastore.
txn/spanner
Spanner contains Transactional Enqueue support for Cloud Spanner.
 Spanner contains Transactional Enqueue support for Cloud Spanner.
Package tsmon adapts common/tsmon library to a server-side environment.
 Package tsmon adapts common/tsmon library to a server-side environment.
Package warmup allows to register hooks executed during the server warmup.
 Package warmup allows to register hooks executed during the server warmup.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.