loader

package
v0.2.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 19, 2024 License: MIT Imports: 21 Imported by: 3

Documentation

Index

Constants

View Source 
const CmdTag = "cmd"

CmdTag is the tag used to specify a command line flag.

View Source 
const ConsulConfigPathPrefixEnv = "CONSUL_CONFIG_PATH_PREFIX"

ConsulConfigPathPrefixEnv holds the name of the env variable that is used to set custom path prefix.

View Source 
const DefaultTag = "default"

DefaultTag is a tag name for default value.

View Source 
const EnvConfigFile = "CONFIG_FILE"

EnvConfigFile sets a name for environmental variable that can hold path for configuration file.

View Source 
const EnvTag = "env"

EnvTag is a tag name for environment variable.

View Source 
const VaultAppRoleBasePathEnv = "VAULT_APPROLE_BASE_PATH"

VaultAppRoleBasePathEnv specifies the name of the environment variable that holds the path to login.

View Source 
const VaultRoleIDEnv = "VAULT_ROLE_ID"

VaultRoleIDEnv specifies the name of environment a variable that holds Vault role id to authenticate with.

View Source 
const VaultRoleSecretEnv = "VAULT_ROLE_SECRET" // nolint:gosec // false-positive

VaultRoleSecretEnv specifies the name of environment a variable that holds Vault role secret.

View Source 
const VaultSecretBasePathEnv = "VAULT_SECRET_BASE_PATH"

VaultSecretBasePathEnv specifies the name of the environment variable that holds the base path to secrets.

Variables

View Source 
var ConfFileSuffixes = []string{".toml", ".yml", ".yaml", ".json"}

ConfFileSuffixes is the ordered list of suffix for configuration file. It is not specific for type(.toml .yml, .yaml, .json) because it is possible to change which loader will be used.

View Source 
var ConsulConfigPathPrefix = "finops"

ConsulConfigPathPrefix stores the default base path for secrets.

View Source 
var ConsulTag = "cfg"

ConsulTag is a tag used to identify field name.

View Source 
var ErrNoClient = errors.New("no client available")

ErrNoClient is returned when no client is found, for Vault and Consul.

View Source 
var ErrNoConfFile = errors.New("config file not found")

ErrNoConfFile use to skip reader.

View Source 
var ErrNoDecoder = errors.New("decoder not found for this file type")

ErrNoDecoder is a serious error and not continue process.

View Source 
var ErrNoEnv = errors.New("CONFIG_FILE not declared")

ErrNoEnv use to skip reader.

View Source 
var FileDecoders = map[string]codec.Decoder{
	".toml": codec.TOML{},
	".yml":  codec.YAML{},
	".yaml": codec.YAML{},
	".json": codec.JSON{},
}

FileDecoders for file extensions

View Source 
var FileTag = "cfg"

FileTag is a tag name for file loader

View Source 
var VaultAppRoleBasePath = "auth/approle/login"

VaultAppRoleBasePath is the default base path for app role authentication.

View Source 
var VaultSecretAdditionalPaths = []AdditionalPath{
	{Map: "", Name: "generic"},
}

VaultSecretAdditionalPaths is usable for additional secrets to load. Such generic secrets might be re-used by any number of applications. They must not be application specific!

View Source 
var VaultSecretBasePath = "finops"

VaultSecretBasePath is the base path for secrets.

View Source 
var VaultSecretTag = "secret"

VaultSecretTag is the tag used to identify Vault secrets.

Functions

func FetchVaultAddrFromConsul 

func FetchVaultAddrFromConsul(ctx context.Context, client *api.Client, serviceFetcher LiveServiceFetcher) error

FetchVaultAddrFromConsul will try to find alive Vault instances in `serviceFetcher` and will set `client` address to random instance. If no instances were found or error happened - nothing will be changed.

If address will be changed - it will always be HTTPS.

func NewConsul 

func NewConsul(addr string) (*api.Client, error)

NewConsul creates a client from a client.

func NewConsulFromEnv 

func NewConsulFromEnv() (*api.Client, error)

NewConsulFromEnv creates a client from environmental variables.

This function uses api.DefatulConfig(), which means that variables should be named as Consul expects them. For example now CONSUL_ADDR should be set as CONSUL_HTTP_ADDR.

func NewConsulWithConfig 

func NewConsulWithConfig(config *api.Config) (*api.Client, error)

NewConsulWithConfig creates a client from a config.

func SimpleVaultLoad deprecated 

func SimpleVaultLoad(addr, token, name string, to interface{}) error

SimpleVaultLoad will load secret data based without need to configure everything.

Deprecated: Please try to use NewVaulterFromEnv or NewVaulterFromClient to create a client and call Load on it.

Types

type AdditionalPath 

type AdditionalPath struct {
	// Map to wrap values to a new map with this key.
	// It could be / seperated to create a nested map [e.g. "foo/bar"].
	Map string
	// Name to get that value.
	Name string
	// InnerPath to get that value [e.g. "foo/bar"].
	// If not found then it return empty map.
	InnerPath string
}

AdditionalPath is used to add additional path to the Vault path.

type AuthOption 

type AuthOption func(*api.Client) error

AuthOption options for authentication.

func SetAppRole 

func SetAppRole(role, secret string) AuthOption

SetAppRole sets the client token from the Approle auth Method It does authenticate to fetch the token, then sets it.

Vault can also be setup to authenticate with role_id only for this the secret id can be passed as blank.

func SetToken 

func SetToken(token string) AuthOption

SetToken sets the token auth method It does not do authentication here.

type Consul 

type Consul struct {
	Client *api.Client
	// Decoder specifies function that will decode the response from Consul.
	// By default it is YAML parser + Map decoder.
	//
	// Please prefer YAML to JSON or anything else if there is no strict requirement for it.
	//
	// Note: this function is not used in Watcher.
	Decoder codec.Decoder
	// Plan for dynamic changes
	Plan Planer
}

Consul is an instance of configuration loader from Consul.

Example usage: 

var config Config // some Config struct

cl, err := api.NewClient(&api.Config{Address: "http://consul:8500"})
if err != nil { ... }

consulLoader := Consul{Client: cl}
err = consulLoader.Load("adm0001s", &config)
if err != nil { ... }

// config is now populated from Consul.

func (Consul) DynamicValue 

func (l Consul) DynamicValue(ctx context.Context, key string) (<-chan []byte, error)

DynamicValue allows to get dynamically updated values at a runtime.

---

If specified key has new value - runner will be called.

If specified key deleted or consul server closed nothing happen and you can restart consul server safetly. Remove specified key not trigger function! This function will spin up Goroutine to track changes in background.

Developers will use a context. After that you can cancel that context to close listening and channel. Don't close channel manually. It return a channel to get new value as []byte.

Example: 

ch, err := loader.Consul{}.DynamicValue(ctx, "test/dynamic")
if err != nil {
	log.Logger.Debug().Err(err).Msg("uupps")

	return
}
for v := range ch {
	// use v here
}

func (*Consul) EnsureClient 

func (l *Consul) EnsureClient() error

EnsureClient creates and sets a Consul client if needed.

func (Consul) Load 

func (l Consul) Load(appName string, to interface{}) error

Load is just same as LoadWithContext without context.

func (Consul) LoadWithContext 

func (l Consul) LoadWithContext(ctx context.Context, appName string, to interface{}) error

LoadWithContext retrieves data from Consul and decode response into 'to' struct.

func (*Consul) SearchLiveServices 

func (l *Consul) SearchLiveServices(ctx context.Context, name string, tags []string) ([]*api.ServiceEntry, error)

SearchLiveServices is a wrapper for c.Client.Health().ServiceMultipleTags(name, tags, true, (&api.QueryOptions{}).WithContext(ctx))

This provides a bit nicer interface on fetching services and gives ability to have LiveServiceFetcher as an argument or a field instead of actual implementation.

type CustomFlagVar 

type CustomFlagVar struct {
	Setter internal.TypeSetter
	Val    reflect.Value
}

func (CustomFlagVar) Get 

func (c CustomFlagVar) Get() interface{}

Get is necessary to get reflect.Value as is and not as a string.

func (CustomFlagVar) Set 

func (c CustomFlagVar) Set(s string) error

func (CustomFlagVar) String 

func (c CustomFlagVar) String() string

type Default 

type Default struct{}

Default is a loader that loads config struct fields with their default value as defined in the tags.

func (Default) FieldNameFunc 

func (l Default) FieldNameFunc(outer string, field reflect.StructField) string

FieldNameFunc defined field retrieval function for Default loader.

This function will return string in format of <field_name>:<default_value> or just "-" if no default value is defined.

func (Default) IteratorFunc 

func (l Default) IteratorFunc(fieldName string, field reflect.Value) error

IteratorFunc returns a setter function for setting fields.

func (Default) Load 

func (l Default) Load(_ string, to interface{}) error

Load is just same as LoadWithContext without context.

func (Default) LoadWithContext 

func (l Default) LoadWithContext(_ context.Context, _ string, to interface{}) error

LoadWithContext loads the config struct fields with their default value as defined in the tags.

type DynamicValuer 

type DynamicValuer interface {
	// DynamicValue polls config field value once in dynamicConfig.RefreshInterval.
	// If after poll value was changed - dynamicConfig.Runner function will be called with new value.
	//
	// If requested service supports better(native) listening for changes - it will be implemented instead.
	//
	// Error handling should be done in runner function.
	DynamicValue(context.Context, string) (<-chan []byte, error)
}

DynamicValuer interface is used to get dynamic value from loader.

type Env 

type Env struct{}

Env allows to load values from environmental variables.

For consistency and best-practices of Linux environmental variables only upper-case values will be used. Even if in tag it is specified in lower-case.

Variable name will be upper-cased when doing lookup. No other cases are checked.

func (Env) FieldNameFunc 

func (l Env) FieldNameFunc(outer string, field reflect.StructField) string

FieldNameFunc returns a field function which will get name from `env` tag, concatenated with '_'(underscore) and uppercased.

func (Env) IteratorFunc 

func (l Env) IteratorFunc(fieldName string, field reflect.Value) error

IteratorFunc sets a field to a value from environment.

If field is not defined in environment - it is no-op.

func (Env) Load 

func (l Env) Load(_ string, to interface{}) error

Load is just same as LoadWithContext without context.

func (Env) LoadWithContext 

func (l Env) LoadWithContext(_ context.Context, _ string, to interface{}) error

LoadWithContext implementation for Env ignores variable name prefix. We want to load env vars without Vault like prefix.

type File 

type File struct {
	// EtcPath for setting new path for etc, default is '/etc'.
	EtcPath string
	// NoFolderCheck doesn't try to check working directory and 'EtcPath'
	// with this formation '<appname>.[yml|yaml|json]'.
	NoFolderCheck bool
}

File is intended to be a limited time option to read configuration from files. Set configuration path on CONFIG_FILE environment variable. '.yml|.yaml|.json' extensions supported.

Breaking changes from v1: config field name will be used as-is, without changing case.

func (File) Load 

func (l File) Load(appName string, to interface{}) error

Load is just same as LoadWithContext without context.

func (File) LoadEnv 

func (l File) LoadEnv(to interface{}) error

LoadEnv will load CONFIG_FILE environment variable.

func (*File) LoadEtc 

func (l *File) LoadEtc(appName string, to interface{}) error

LoadEtc will load configuration file from /etc directory. File name is appName, so resulting path will be /etc/<appName>.

func (File) LoadFile 

func (l File) LoadFile(fileName string, to interface{}) error

LoadFile loads config values from a fileName.

func (File) LoadFileSuffix 

func (l File) LoadFileSuffix(filePath string, to interface{}) error

LoadFileSuffix will load configuration from file path.

func (File) LoadWithContext 

func (l File) LoadWithContext(_ context.Context, appName string, to interface{}) error

LoadWithContext will try to load configuration file from two places: working directory(or file specified in env) and /etc. File in /etc will only be read if configuration file is missing in working directory.

See DefaultDecoder for understanding of which decoder will used in this loader.

Not existing configuration files are not treated as an error. If this behavior is required - use `Reader.Load*` methods directly.

func (File) LoadWorkDir 

func (l File) LoadWorkDir(appName string, to interface{}) error

LoadWorkDir will load configuration from current working directory.

If EnvConfigFile environment variable is specified - it will have priority over current working directory. EnvConfigFile value will be expanded with environment before use.

type Flags 

type Flags struct {
	// Args to give manually read from flags default os.Args[1:]
	Args []string
	// NoUsage may be used to silence usage on invalid flags.
	NoUsage bool
}

Flags will parse CMD args.

Flags defined in inner structs will be set with combination of outer flag name and inner flag name separated by '-'.

For example: 

type s struct{
	A struct{
		B int `cmd:"inner"`
	} `cmd:"outer"`
}

Field B could be set using flag '--outer-inner'. This works for any depth of field.

If flag name is not specified and "cfg" tag is not present - lowercase field name will be used as flag name.

Breaking change from v1: parsing will fail with usage if unknown flag will be found.

Also only first value from the tag will be used. This means that if tag is 'cmd:"tag,t"' - only "tag" will be used as command line. This might be changed in the future.

func (Flags) AddFlagsIterator 

func (l Flags) AddFlagsIterator(set *flag.FlagSet) internal.IteratorFunc

AddFlagsIterator is the function to add flags to a specified flag set.

func (Flags) FieldNameFunc 

func (l Flags) FieldNameFunc(outer string, field reflect.StructField) string

FieldNameFunc returns a field name retrieved from `cmd` tag, concatenated with '-'(minus sign) and lowercased.

func (Flags) Load 

func (l Flags) Load(_ string, to interface{}) error

Load loads config values from the command line.

func (Flags) LoadSlice 

func (l Flags) LoadSlice(to interface{}, args []string) error

LoadSlice loads config values from the command line.

func (Flags) LoadWithContext 

func (l Flags) LoadWithContext(_ context.Context, _ string, to interface{}) error

LoadWithContext loads config values from the command line with context. Context value not used in this loader.

func (Flags) ProcessFlagsIterator 

func (l Flags) ProcessFlagsIterator(set flag.FlagSet) internal.IteratorFunc

ProcessFlagsIterator is the function to set flag values based on already parsed flags.

type LiveServiceFetcher 

type LiveServiceFetcher func(ctx context.Context, name string, tags []string) ([]*api.ServiceEntry, error)

LiveServiceFetcher is a signature of the function that will fetch only live instances of the service.

If no services found - (nil, nil) will be returned.

type Loader 

type Loader interface {
	// Load will load all available data to at 'to' value.
	//
	// Even if particular loader type must implement ReflectLoader -
	// this interface still must be implemented as a proxy.
	Load(appName string, to interface{}) error

	// LoadWithContext same as Load but using predefined ctx in load process.
	// This is usable for logging.
	LoadWithContext(ctx context.Context, appName string, to interface{}) error
}

Loader is a interface for all loaders.

func NewVaulterer 

func NewVaulterer(addr string, opts ...AuthOption) (loader Loader, err error)

NewVaulterer returns the Vaulter interface. If role_id is given it will call SetTokenAppRole to set the token

Example usage: 

var config Config // some Config struct

vaultLoader, NewVaulterer(addr, SetAppRole(roleID, ""))
if err != nil { ... }

err = vaultLoader.Load("adm0001s", &config)
if err != nil { ... }

type Planer 

type Planer interface {
	SetHandler(func([]byte))
	Run(*api.Client) error
	Stop()
}

Planer for dynamically get changes interface.

type Vault 

type Vault struct {
	Client Vaulter
}

Vault loads secret values from Vault instance.

Generic secrets can also be loaded by using LoadGeneric method

Example usage: 

var config Config // some Config struct

cl, err := api.NewClient(&api.Config{Address: "http://vault:8200"})
if err != nil { ... }

cl.SetToken("some_token") // this could be also other means of authentication.

vaultLoader := Vault{Client: cl}
err = vaultLoader.Load("adm0001s", &config)
if err != nil { ... }

// config is now populated from Vault.

func (*Vault) EnsureClient 

func (l *Vault) EnsureClient(ctx context.Context) error

EnsureClient creates and sets a Vault client if needed.

func (*Vault) Load 

func (l *Vault) Load(appName string, to interface{}) error

Load is same as LoadWithContext without context.

func (*Vault) LoadFromReformat 

func (l *Vault) LoadFromReformat(ctx context.Context, paths []AdditionalPath, to interface{}) error

LoadFromReformat loads secrets from Vault and load to the input struct 'to'.

func (*Vault) LoadGeneric 

func (l *Vault) LoadGeneric(ctx context.Context, to interface{}) error

LoadGeneric loads generic(shared) secrets from Vault.

func (*Vault) LoadWithContext 

func (l *Vault) LoadWithContext(ctx context.Context, appName string, to interface{}) error

LoadWithContext will load data from Vault to input struct 'to'. 'name' is base secret path, or just name of application. Path will be constructed as "${VaultSecretTag}/${name}". By default VaultSecretTag value is "secrets/data", which allows to load secrets from root.

type Vaulter 

type Vaulter interface {
	Read(path string) (*api.Secret, error)
	List(path string) (*api.Secret, error)
}

Vaulter interface for Vault.

func NewVaulter 

func NewVaulter(addr, token string) (Vaulter, error)

NewVaulter creates a new Vault client.

func NewVaulterFromClient 

func NewVaulterFromClient(ctx context.Context, cl *api.Client) (Vaulter, error)

NewVaulterFromClient will create a Vaulter client based on input client.

This function will try to choose live Vault instance from the Consul.

func NewVaulterFromEnv 

func NewVaulterFromEnv(ctx context.Context) (Vaulter, error)

NewVaulterFromEnv creates default Vault client.

If VaultRoleIDEnv environment variable is set - also logs in based on role_id and role_secret.

See NewVaulterFromClient for more information.

type Watch 

type Watch struct {
	Plan *watch.Plan
}

Watch struct is adapter for consul watch api to Plan.

func (*Watch) Run 

func (w *Watch) Run(c *api.Client) error

Run start to get changes. If Plan value is nil, raise panic.

func (*Watch) SetHandler 

func (w *Watch) SetHandler(fn func([]byte))

SetHandler call function if new changes happen. If Plan value is nil, raise panic.

func (*Watch) Stop 

func (w *Watch) Stop()

Stop close run function. If Plan value is nil, raise panic.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.