cnfg

package module
v0.2.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 21, 2024 License: MIT Imports: 9 Imported by: 36

README

golift.io/cnfg

Go Report Card

Overview

Procedures for parsing config files and environment variables into data structures. Works much like json.Unmarshal and json.Marshal. Short explanation on how the env variable mapping works below. See GoDoc for several working examples and further explanation of how maps and slices can be accessed with shell env vars.

Supports all base types, including slices, maps, slices of maps, maps of slices, pointers of slices to maps of slices full of ints, strings, floats and the like!

Please open an issue if you run into a bug or an unsupported type.

Better documentation is needed. Most of it is in GoDoc. This package is full featured for environment variable parsing!

Bonus, it goes the other way too. You can convert a data structure into environment variables. This is useful when you wish to pass a lot of data into a command via exec.Command. Simply set the data into a struct, marshal it into environment variables and pass it in.

Examples

type Shelter struct {
	Title  string    `xml:"title"`
	Sym    float64   `xml:"sym"`
	People []*Person `xml:"people"`
	Dogs   []*Dog    `xml:"dogs"`
}

type Person struct {
	Name    string `xml:"name"`
	Present bool   `xml:"present"`
	Age     int    `xml:"age"`
	ID      int64  `xml:"id"`
}

type Dog struct {
	Name    string
	Elapsed config.Duration
	Owners  []string
}

type Config struct {
	*Shelter `xml:"shelter"`
}

The above struct can be configured with the following environment variables, assuming you set prefix := "APP" when you call UnmarshalENV(). Slices use env vars with numbers in them, starting at 0 and going to infinity, or the last env var provided + 1, whichever comes first. It just works. The ... and ++ indicate that those parameters belong to slices, and many items may be appended or overridden.

APP_SHELTER_TITLE
APP_SHELTER_SYM
APP_SHELTER_PEOPLE_0_NAME
APP_SHELTER_PEOPLE_0_PRESENT
APP_SHELTER_PEOPLE_0_AGE
APP_SHELTER_PEOPLE_0_ID
APP_SHELTER_PEOPLE_1_NAME
...
APP_SHELTER_PEOPLE_10_ID ++

APP_SHELTER_DOGS_0_NAME
APP_SHELTER_DOGS_0_ELAPSED
APP_SHELTER_DOGS_0_OWNERS_0
...
APP_SHELTER_DOGS_0_OWNERS_10 ++

APP_SHELTER_DOGS_1_NAME
APP_SHELTER_DOGS_1_ELAPSED
APP_SHELTER_DOGS_1_OWNERS_0
APP_SHELTER_DOGS_1_OWNERS_1 ++

If you passed in the Shelter struct instead of Config, all the of the SHELTER_ portions of the tags would be omitted. You can also set which struct tag to use by creating an &ENV{} pointer and setting Tag and/or Pfx . Tag defaults to "xml", but you could set it to "env" and make custom names for env variables. The env var prefix Pfx is optional, but recommended.

Documentation

Overview

Package cnfg provides procedures to parse a slew of environment variables into a struct pointer.

Use this package if your app uses a config file and you want to allow users to change or override the configurations in that file with environment variables. You can also use this app just to parse environment variables; handling a config file is entirely optional. Every member type is supported. If I missed one, please open an issue. If you need a non-base type supported (net.IP works already) please open an issue. New types are extremely easy to add. If this package interests you, pull requests and feature requests are welcomed!

I consider this package the pinnacle example of how to configure Go applications from a file. You can put your configuration into any file format: XML, YAML, JSON, TOML, and you can override any struct member using an environment variable. I created this package because I got tired of writing custom env parser code for every app I make. This simplifies all the heavy lifting and I don't even have to think about it now. I hope you enjoy using this simplification as much as I do!

Index

Examples

Constants

View Source
const ENVTag = "xml"

ENVTag is the tag to look for on struct members. You may choose to use a custom tag by creating an &ENV{} struct with a different Tag. "env" is popular, but I chose "xml" because the nouns are generally singular, and those look good as env variables. "xml" is also convenient because it's brief and doesn't add yet another struct tag. Those lines can get long quickly.

View Source
const LevelSeparator = "_"

LevelSeparator is used to separate the names from different struct levels. This is hard coded here and cannot be changed or modified.

Variables

View Source
var (
	ErrUnsupported      = fmt.Errorf("unsupported type, please report this if this type should be supported")
	ErrInvalidByte      = fmt.Errorf("invalid byte")
	ErrInvalidInterface = fmt.Errorf("can only unmarshal ENV into pointer to struct")
)

Custom errors this package may produce.

Functions

func UnmarshalENV added in v0.0.3

func UnmarshalENV(i interface{}, prefixes ...string) (bool, error)

UnmarshalENV copies environment variables into configuration values. This is useful for Docker users that find it easier to pass ENV variables than a specific configuration file. Uses reflection to find struct tags.

Example

Complete working example for UnmarshalENV(). Use this method when the "xml" struct tag suits your application.

package main

import (
	"fmt"
	"os"

	"golift.io/cnfg"
)

func main() {
	// Systems is used to show an example of how to access nested slices.
	type System struct {
		Name   string             `xml:"name"`
		Signal []byte             `xml:"signal"`
		Ion    *map[string]string `xml:"ion"`
	}

	// Config represents your application's environment variable based config inputs.
	// Works with or without pointers.
	type Config struct {
		Users []struct {
			Name   string    `xml:"name"`
			Levels []float64 `xml:"level"`
		} `xml:"user"`
		Systems []*System `xml:"system"`
	}

	// Make a pointer to your struct. It may be empty or contain defaults.
	// It may contain nested pointers, structs, maps, slices, etc. It all works.
	config := &Config{}

	// Okay set some ENV variables. Pretend you did this in bash.
	// Setting these will overwrite any existing data. If you set a slice that
	// does not exist, it has to be the _following_ index number. In other words,
	// if your slice is empty, setting APP_USER_1_NAME wont work, you have to start
	// with 0. If your slice len is 2, you can append by setting APP_USER_2_NAME
	os.Setenv("APP_USER_0_NAME", "Tim")
	os.Setenv("APP_USER_0_LEVEL_0", "1")
	os.Setenv("APP_USER_0_LEVEL_1", "13")
	os.Setenv("APP_USER_1_NAME", "Jon")
	os.Setenv("APP_USER_1_LEVEL_0", "1")

	// This adds (creates) systems and signals in sub-slices.
	os.Setenv("APP_SYSTEM_0_NAME", "SysWon")
	os.Setenv("APP_SYSTEM_1_NAME", "SysToo")
	// With []byte you can only pass a string, and it's converted.
	// You cannot access a byte member directly. Do you need to? Let me know!
	os.Setenv("APP_SYSTEM_0_SIGNAL", "123456")
	os.Setenv("APP_SYSTEM_1_SIGNAL", "654321")

	// Maps inside slices! You can nest all you want, but your variable names may get lengthy.
	fmt.Printf("BEFORE => Users: %v, Systems: %v\n", len(config.Users), len(config.Systems))
	os.Setenv("APP_SYSTEM_1_ION_reactor-1", "overload")
	os.Setenv("APP_SYSTEM_1_ION_reactor-2", "underload")

	// Run Unmarshal to parse the values into your config pointer.
	// We ignore "ok" here. You may choose to capture and it do something though.
	_, err := cnfg.UnmarshalENV(config, "APP")
	if err != nil {
		panic(err)
	}

	fmt.Printf("AFTER  => Users: %v\n", config.Users)

	for i, s := range config.Systems {
		fmt.Printf(" %v: System Name: %v, Signals: %v, Ion: %v\n", i, s.Name, s.Signal, s.Ion)
	}
}
Output:

BEFORE => Users: 0, Systems: 0
AFTER  => Users: [{Tim [1 13]} {Jon [1]}]
 0: System Name: SysWon, Signals: [49 50 51 52 53 54], Ion: <nil>
 1: System Name: SysToo, Signals: [54 53 52 51 50 49], Ion: &map[reactor-1:overload reactor-2:underload]

func UnmarshalMap added in v0.0.3

func UnmarshalMap(pairs map[string]string, i interface{}) (bool, error)

UnmarshalMap parses and processes a map of key/value pairs as though they were environment variables. Useful for testing, or unmarshaling values from places other than environment variables. This version of UnmarshalMap assumes default tag ("xml") and no prefix: "".

Example
package main

import (
	"fmt"

	"golift.io/cnfg"
)

func main() {
	type myConfig struct {
		Key    string `xml:"envkey"`
		Key2   string `xml:"envkey2"`
		Nested struct {
			SubSlice []string          `xml:"subslice"`
			SubMap   map[string]string `xml:"submap"`
		} `xml:"nested"`
	}

	// Create a pointer to unmarshal your map into.
	config := &myConfig{}

	// Generally you'd use MapEnvPairs() to create a map from a slice of []string.
	// You can also get your data from any other source, as long as it can be
	// formatted into a map.
	// The important part is formatting the map keys correctly. The struct tag names
	// are always upcased, but nested struct member maps are not. They can be any case.
	// Each nested struct is appended to the parent name(s) with an underscore _.
	// Slices (except byte slices) are accessed by their position, beginning with 0.
	pairs := make(cnfg.Pairs)
	pairs["ENVKEY"] = "some env value"
	pairs["ENVKEY2"] = "some other env value"
	pairs["NESTED_SUBSLICE_0"] = "first slice value"
	pairs["NESTED_SUBMAP_mapKey"] = "first map key value"

	worked, err := cnfg.UnmarshalMap(pairs, config)
	if err != nil {
		panic(err)
	}

	fmt.Printf("ok: %v, key: %v, key2: %v\n", worked, config.Key, config.Key2)
	fmt.Println("map:", config.Nested.SubMap)
	fmt.Println("slice:", config.Nested.SubSlice)
}
Output:

ok: true, key: some env value, key2: some other env value
map: map[mapKey:first map key value]
slice: [first slice value]

Types

type Duration

type Duration struct{ time.Duration }

Duration is useful if you need to load a time Duration from a config file into your application. Use the config.Duration type to support automatic unmarshal from all sources. If you do not use a config file, do not use this type because the environment unmarshaler supports time.Duration natively.

func (Duration) MarshalJSON added in v0.0.6

func (d Duration) MarshalJSON() ([]byte, error)

MarshalJSON returns the string representation of a Duration for JSON. ie. "1m32s".

func (Duration) MarshalText added in v0.0.6

func (d Duration) MarshalText() ([]byte, error)

MarshalText returns the string representation of a Duration. ie. 1m32s.

func (Duration) String added in v0.1.1

func (d Duration) String() string

String returns a Duration as string without trailing zero units.

func (*Duration) UnmarshalText

func (d *Duration) UnmarshalText(b []byte) error

UnmarshalText parses a duration type from a config file. This method works with the Duration type to allow unmarshaling of durations from files and env variables in the same struct. You won't generally call this directly.

type ENV added in v0.0.3

type ENV struct {
	Tag string // Struct tag name.
	Pfx string // ENV var prefix.
	Low bool   // Set this false to avoid capitalizing variables.
}

ENV allows you to parse environment variables using an object instead of global state. This package allows using the default ENVTag from global state, or you can pass in your own using this struct. See the UnmarshalENV function (it's 1 line) for an example of how to use this.

func (*ENV) Marshal added in v0.1.0

func (e *ENV) Marshal(i interface{}) (Pairs, error)

Marshal deconstructs a data structure into environment variable pairs.

func (*ENV) Unmarshal added in v0.0.3

func (e *ENV) Unmarshal(i interface{}) (bool, error)

Unmarshal parses and processes environment variables into the provided interface. Uses the Prefix and Tag name from the &ENV{} struct values.

Example (Simple)

Complete working example for ENV.Unmarshal().

package main

import (
	"fmt"
	"os"
	"time"

	"golift.io/cnfg"
)

func main() {
	// Systems is used to show an example of how to access nested slices.
	type System struct {
		Name   string `env:"name"`
		Signal *[]int `env:"signal"`
	}

	// Config represents your application's environment variable based config inputs.
	// Works with or without pointers.
	type Config struct {
		Debug    bool           `env:"debug"`
		Users    []string       `env:"user"`
		Interval *time.Duration `env:"interval"`
		Systems  []*System      `env:"system"`
	}

	// Make a pointer to your struct with some default data.
	// Maybe this data came from a config file? Using ParseFile()!
	config := &Config{
		Debug:    true,
		Users:    []string{"me", "you", "them"},
		Interval: nil,
		Systems:  nil,
	}

	// Okay set some ENV variables. Pretend you did this in bash.
	os.Setenv("APP_DEBUG", "false")   // turn off debug
	os.Setenv("APP_USER_1", "dad")    // replace "you" with "dad"
	os.Setenv("APP_USER_3", "mom")    // add "mom"
	os.Setenv("APP_INTERVAL", "7m1s") // don't forget the interval!!

	// This adds (creates) systems and signals in sub-slices.
	os.Setenv("APP_SYSTEM_0_NAME", "SysWon")
	os.Setenv("APP_SYSTEM_1_NAME", "SysToo")
	os.Setenv("APP_SYSTEM_1_SIGNAL_0", "12")
	// You can add as many as you like, as long as they are in numerical order.
	os.Setenv("APP_SYSTEM_1_SIGNAL_1", "77")

	fmt.Printf("BEFORE => Debug: %v, Interval: %v, Users: %v, Systems: %v\n",
		config.Debug, config.Interval, config.Users, config.Systems)

	// Make a ENV Decoder with special tag and prefix.
	env := &cnfg.ENV{Tag: "env", Pfx: "APP"}

	// Run Unmarshal to parse the values into your config pointer:
	found, err := env.Unmarshal(config)
	if err != nil {
		panic(err)
	}

	// And optionally, do something with the "found" return value.
	// If you wanted to overwrite ALL configs if ANY env variables are present
	// you could use ok to make and if statement that does that.
	if found {
		fmt.Println("~ Environment variables were parsed into the config!")
	}

	// If you don't set an env variable for it, it will stay nil.
	// Same for structs and slices.
	if config.Interval == nil {
		fmt.Printf("You forgot to set an interval!")

		return
	}

	fmt.Printf("AFTER => Debug: %v, Interval: %v, Users: %v\n", config.Debug, *config.Interval, config.Users)
	// We added some systems, check them!
	for i, s := range config.Systems {
		fmt.Printf(" %v: System Name: %v, Signals: %v\n", i, s.Name, s.Signal)
	}
}
Output:

BEFORE => Debug: true, Interval: <nil>, Users: [me you them], Systems: []
~ Environment variables were parsed into the config!
AFTER => Debug: false, Interval: 7m1s, Users: [me dad them mom]
 0: System Name: SysWon, Signals: <nil>
 1: System Name: SysToo, Signals: &[12 77]

func (*ENV) UnmarshalMap added in v0.0.3

func (e *ENV) UnmarshalMap(pairs map[string]string, i interface{}) (bool, error)

UnmarshalMap parses and processes a map of key/value pairs as though they were environment variables. Useful for testing, or unmarshaling values from places other than environment variables. Use this version of UnmarshalMap if you need to change the tag or prefix.

type ENVMarshaler added in v0.1.0

type ENVMarshaler interface {
	MarshalENV(tag string) (map[string]string, error)
}

ENVMarshaler allows marshaling custom types into env variables.

type ENVUnmarshaler

type ENVUnmarshaler interface {
	UnmarshalENV(tag, envval string) error
}

ENVUnmarshaler allows custom unmarshaling on a custom type. If your type implements this, it will be called and the logic stops there.

Example

This simple example shows how you may use the ENVUnmarshaler interface. This shows how to use two environment variables to set one custom value.

package main

import (
	"fmt"
	"os"
	"strconv"
	"time"

	"golift.io/cnfg"
)

// TimeX uses two environment variables to multiply a duration.
type TimeX struct {
	time.Duration
}

type AppConfig struct {
	Name    string `xml:"name"`
	Special TimeX  `xml:"in"`
}

func (t *TimeX) UnmarshalENV(tag, val string) error {
	xTag := tag + "_X"

	xString, ok := os.LookupEnv(xTag)
	if !ok {
		xString = "1"
	}

	multiplier, err := strconv.Atoi(xString)
	if err != nil {
		return fmt.Errorf("multiplier invalid %s: %w", xTag, err)
	}

	t.Duration, err = time.ParseDuration(val)
	if err != nil {
		return fmt.Errorf("duration invalid %s: %w", tag, err)
	}

	t.Duration *= time.Duration(multiplier)

	return nil
}

func main() {
	config := &AppConfig{}

	os.Setenv("APP_IN", "5m")
	os.Setenv("APP_IN_X", "10")
	os.Setenv("APP_NAME", "myApp")

	_, err := cnfg.UnmarshalENV(config, "APP")
	if err != nil {
		panic(err)
	}

	fmt.Printf("%s starts in %v", config.Name, config.Special)
}
Output:

myApp starts in 50m0s

type Pairs added in v0.0.3

type Pairs map[string]string

Pairs represents pairs of environment variables. These can be used directly or converted to other usable formats.

func MapEnvPairs added in v0.0.3

func MapEnvPairs(prefix string, pairs []string) Pairs

MapEnvPairs turns the pairs returned by os.Environ() into a map[string]string. Providing a prefix returns only variables with that prefix.

Example

MapEnvPairs can be used when you want to inspect or modify the environment variable values before unmarshaling them.

package main

import (
	"fmt"
	"os"

	"golift.io/cnfg"
)

func main() {
	type myConfig struct {
		Key  string `env:"envkey"`
		Key2 string `env:"envkey2"`
		Key3 string `env:"envkey3"`
	}

	os.Setenv("TESTAPP_ENVKEY", "some env value")
	os.Setenv("TESTAPP_ENVKEY2", "some other env value")

	// Create pairs from the current environment.
	// Only consider environment variables that begin with "TESTAPP"
	pairs := cnfg.MapEnvPairs("TESTAPP", os.Environ())
	for k, v := range pairs {
		fmt.Println(k, v)
	}

	// This is the magic offered by this method.
	pairs["TESTAPP_ENVKEY3"] = "add (or overwrite) a third value in code"
	config := &myConfig{}

	// We have to use &ENV{} to set a custom prefix, and change the struct tag.
	ok, err := (&cnfg.ENV{Pfx: "TESTAPP", Tag: "env"}).UnmarshalMap(pairs, config)
	if err != nil {
		panic(err)
	}

	fmt.Printf("ok: %v, key: %v, key2: %v, key3: %v\n", ok, config.Key, config.Key2, config.Key3)
}
Output:

TESTAPP_ENVKEY some env value
TESTAPP_ENVKEY2 some other env value
ok: true, key: some env value, key2: some other env value, key3: add (or overwrite) a third value in code

func MarshalENV added in v0.1.0

func MarshalENV(i interface{}, prefix string) (Pairs, error)

MarshalENV turns a data structure into an environment variable. The resulting slice can be copied into exec.Command.Env. Prefix is optional, and will prefix returned variables.

func (Pairs) Env added in v0.1.0

func (p Pairs) Env() []string

Env turns the Pairs map into an envionrment variable slice. This slice can be set to exec.Command().Env.

func (*Pairs) Get added in v0.0.3

func (p *Pairs) Get(prefix string) Pairs

Get allows getting only specific env variables by prefix. The prefix is trimmed before returning.

func (Pairs) Merge added in v0.1.0

func (p Pairs) Merge(pairs Pairs)

Merge merges two Pairs maps.

func (Pairs) Quoted added in v0.1.0

func (p Pairs) Quoted() []string

Quoted turns the Pairs map into an envionrment variable slice that can be used by bash or other shells.

func (Pairs) Set added in v0.1.0

func (p Pairs) Set(k, v string)

Set simply sets a value in a map.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL