pachyderm: github.com/pachyderm/pachyderm/src/server/pkg/serviceenv Index | Files

package serviceenv

import "github.com/pachyderm/pachyderm/src/server/pkg/serviceenv"

Index

Package Files

config.go service_env.go

type Configuration Uses

type Configuration struct {
    *GlobalConfiguration
    *PachdSpecificConfiguration
    *WorkerSpecificConfiguration
}

Configuration is the generic configuration structure used to access configuration fields.

func NewConfiguration Uses

func NewConfiguration(config interface{}) *Configuration

NewConfiguration creates a generic configuration from a specific type of configuration.

type FeatureFlags Uses

type FeatureFlags struct {
    NewHashTree bool `env:"NEW_HASH_TREE,default=false"`
}

FeatureFlags contains the configuration for feature flags.

type GlobalConfiguration Uses

type GlobalConfiguration struct {
    FeatureFlags
    EtcdHost      string `env:"ETCD_SERVICE_HOST,required"`
    EtcdPort      string `env:"ETCD_SERVICE_PORT,required"`
    PPSWorkerPort uint16 `env:"PPS_WORKER_GRPC_PORT,default=80"`
    Port          uint16 `env:"PORT,default=650"`
    PProfPort     uint16 `env:"PPROF_PORT,default=651"`
    HTTPPort      uint16 `env:"HTTP_PORT,default=652"`
    PeerPort      uint16 `env:"PEER_PORT,default=653"`
    PPSEtcdPrefix string `env:"PPS_ETCD_PREFIX,default=pachyderm_pps"`
    Namespace     string `env:"NAMESPACE,default=default"`
    StorageRoot   string `env:"PACH_ROOT,default=/pach"`
}

GlobalConfiguration contains the global configuration.

type PachdFullConfiguration Uses

type PachdFullConfiguration struct {
    GlobalConfiguration
    PachdSpecificConfiguration
}

PachdFullConfiguration contains the full pachd configuration.

type PachdSpecificConfiguration Uses

type PachdSpecificConfiguration struct {
    NumShards             uint64 `env:"NUM_SHARDS,default=32"`
    StorageBackend        string `env:"STORAGE_BACKEND,default="`
    StorageHostPath       string `env:"STORAGE_HOST_PATH,default="`
    EtcdPrefix            string `env:"ETCD_PREFIX,default="`
    PFSEtcdPrefix         string `env:"PFS_ETCD_PREFIX,default=pachyderm_pfs"`
    AuthEtcdPrefix        string `env:"PACHYDERM_AUTH_ETCD_PREFIX,default=pachyderm_auth"`
    EnterpriseEtcdPrefix  string `env:"PACHYDERM_ENTERPRISE_ETCD_PREFIX,default=pachyderm_enterprise"`
    KubeAddress           string `env:"KUBERNETES_PORT_443_TCP_ADDR,required"`
    Metrics               bool   `env:"METRICS,default=true"`
    Init                  bool   `env:"INIT,default=false"`
    BlockCacheBytes       string `env:"BLOCK_CACHE_BYTES,default=1G"`
    PFSCacheSize          string `env:"PFS_CACHE_SIZE,default=0"`
    WorkerImage           string `env:"WORKER_IMAGE,default="`
    WorkerSidecarImage    string `env:"WORKER_SIDECAR_IMAGE,default="`
    WorkerImagePullPolicy string `env:"WORKER_IMAGE_PULL_POLICY,default="`
    LogLevel              string `env:"LOG_LEVEL,default=info"`
    IAMRole               string `env:"IAM_ROLE,default="`
    ImagePullSecret       string `env:"IMAGE_PULL_SECRET,default="`
    NoExposeDockerSocket  bool   `env:"NO_EXPOSE_DOCKER_SOCKET,default=false"`
    ExposeObjectAPI       bool   `env:"EXPOSE_OBJECT_API,default=false"`
    MemoryRequest         string `env:"PACHD_MEMORY_REQUEST,default=1T"`
    WorkerUsesRoot        bool   `env:"WORKER_USES_ROOT,default=true"`
    S3GatewayPort         uint16 `env:"S3GATEWAY_PORT,default=600"`
}

PachdSpecificConfiguration contains the pachd specific configuration.

type ServiceEnv Uses

type ServiceEnv struct {
    *Configuration
    // contains filtered or unexported fields
}

ServiceEnv is a struct containing connections to other services in the cluster. In pachd, there is only one instance of this struct, but tests may create more, if they want to create multiple pachyderm "clusters" served in separate goroutines.

func InitPachOnlyEnv Uses

func InitPachOnlyEnv(config *Configuration) *ServiceEnv

InitPachOnlyEnv initializes this service environment. This dials a GRPC connection to pachd only (in a background goroutine), and creates the template pachClient used by future calls to GetPachClient.

This call returns immediately, but GetPachClient will block until the client is ready.

func InitServiceEnv Uses

func InitServiceEnv(config *Configuration) *ServiceEnv

InitServiceEnv initializes this service environment. This dials a GRPC connection to pachd and etcd (in a background goroutine), and creates the template pachClient used by future calls to GetPachClient.

This call returns immediately, but GetPachClient and GetEtcdClient block until their respective clients are ready.

func InitWithKube Uses

func InitWithKube(config *Configuration) *ServiceEnv

InitWithKube is like InitServiceEnv, but also assumes that it's run inside a kubernetes cluster and tries to connect to the kubernetes API server.

func (*ServiceEnv) GetEtcdClient Uses

func (env *ServiceEnv) GetEtcdClient() *etcd.Client

GetEtcdClient returns the already connected etcd client without modification.

func (*ServiceEnv) GetKubeClient Uses

func (env *ServiceEnv) GetKubeClient() *kube.Clientset

GetKubeClient returns the already connected Kubernetes API client without modification.

func (*ServiceEnv) GetPachClient Uses

func (env *ServiceEnv) GetPachClient(ctx context.Context) *client.APIClient

GetPachClient returns a pachd client with the same authentication credentials and cancellation as 'ctx' (ensuring that auth credentials are propagated through downstream RPCs).

Functions that receive RPCs should call this to convert their RPC context to a Pachyderm client, and internal Pachyderm calls should accept clients returned by this call.

(Warning) Do not call this function during server setup unless it is in a goroutine. A Pachyderm client is not available until the server has been setup.

type WorkerFullConfiguration Uses

type WorkerFullConfiguration struct {
    GlobalConfiguration
    WorkerSpecificConfiguration
}

WorkerFullConfiguration contains the full worker configuration.

type WorkerSpecificConfiguration Uses

type WorkerSpecificConfiguration struct {
    // Worker gets its own IP here, via the k8s downward API. It then writes that
    // IP back to etcd so that pachd can discover it
    PPSWorkerIP string `env:"PPS_WORKER_IP,required"`
    // The name of the pipeline that this worker belongs to
    PPSPipelineName string `env:"PPS_PIPELINE_NAME,required"`
    // The ID of the commit that contains the pipeline spec.
    PPSSpecCommitID string `env:"PPS_SPEC_COMMIT,required"`
    // The name of this pod
    PodName string `env:"PPS_POD_NAME,required"`
}

WorkerSpecificConfiguration contains the worker specific configuration.

Package serviceenv imports 15 packages (graph) and is imported by 13 packages. Updated 2019-08-13. Refresh now. Tools for package owners.