kvmapstruct: github.com/uthng/kvmapstruct Index | Examples | Files

package kvmapstruct

import "github.com/uthng/kvmapstruct"

Package kvmapstruct exposes various utility functions to do conversions between: Consul KV pairs and native Go Struct or map[string]interface{}.

It also provides several utilities to convert directly: nested map to flatten/kv map or Consul kv pairs, flatten/kv map to Go struct, Kv map to nested map, etc.

There are some notions that are used in this package. Nested map: classic map[string]interface{}. Flatten map: map[string]interface{} represents key/value and value can be a normal type including slice or map KV map: map[string]interface{} represents key/value but value can not be slice or map. A slice will be represented by keys suffixed by 0, 1, 2 etc.

Only the following value types are supported: int, bool, string, []int, []bool, []string and map[string]interface{}

Index

Examples

Package Files

kvmapstruct.go

func FlattenMapToStruct Uses

func FlattenMapToStruct(in map[string]interface{}, out interface{}) error

FlattenMapToStruct converts a flatten map to a Go struct. Out argument must be a initialiezed pointer to a Go struct. Its substructs can be a pointer to a struct, embedded struct or struct. If it is a pointer, it must be initialized.

Code:

type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    Key43 *ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    Key4 *ExSTChildLevel1
}

input := map[string]interface{}{
    "Key1": "val1",
    "Key2": 2,
    "Key3": []int{1, 2, 3},
    "Key4": map[string]interface{}{
        "Key41": "val41",
        "Key42": map[string]interface{}{
            "Key421": "val421",
            "Key422": []string{"one", "two", "three"},
        },
        "Key43": map[string]interface{}{
            "Key431": map[string]interface{}{
                "Key4311": "val4311",
            },
        },
    },
}

st := &ExST{
    Key4: &ExSTChildLevel1{
        Key43: &ExSTChildLevel2{},
    },
}

err := FlattenMapToStruct(input, st)
if err != nil {
    return
}

fmt.Println(st)

func KVMapToMap Uses

func KVMapToMap(in map[string]interface{}, prefix string) (map[string]interface{}, error)

KVMapToMap converts a KV map to nested map.

Code:

input := map[string]interface{}{
    "test/key1":                "val1",
    "test/key2":                2,
    "test/key3/0":              1,
    "test/key3/1":              2,
    "test/key3/2":              3,
    "test/key4/key41":          "val41",
    "test/key4/key42/key421":   "val421",
    "test/key4/key42/key422/0": "one",
    "test/key4/key42/key422/1": "two",
    "test/key4/key42/key422/2": "three",
}

o, err := KVMapToMap(input, "test")
if err != nil {
    return
}

fmt.Println(o)

func KVMapToStruct Uses

func KVMapToStruct(in map[string]interface{}, prefix string, out interface{}) error

KVMapToStruct converts a KV map to a Go struct. Out argument must be a initialiezed pointer to a Go struct. Its substructs can be a pointer to a struct, embedded struct or struct. If it is a pointer, it must be initialized.

Code:

type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    Key43 *ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    Key4 *ExSTChildLevel1
}

input := map[string]interface{}{
    "Key1":                      "val1",
    "Key2":                      2,
    "Key3/0":                    1,
    "Key3/1":                    2,
    "Key3/2":                    3,
    "Key4/Key41":                "val41",
    "Key4/Key42/Key421":         "val421",
    "Key4/Key42/Key422/0":       "one",
    "Key4/Key42/Key422/1":       "two",
    "Key4/Key42/Key422/2":       "three",
    "Key4/Key43/Key431/Key4311": "val4311",
}

st := &ExST{
    Key4: &ExSTChildLevel1{
        Key43: &ExSTChildLevel2{},
    },
}

err := KVMapToStruct(input, "", st)
if err != nil {
    return
}

fmt.Println(st)

Code:

type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    ExSTChildLevel1
}

input := map[string]interface{}{
    "Key1":                           "val1",
    "Key2":                           2,
    "Key3/0":                         1,
    "Key3/1":                         2,
    "Key3/2":                         3,
    "ExSTChildLevel1/Key41":          "val41",
    "ExSTChildLevel1/Key42/Key421":   "val421",
    "ExSTChildLevel1/Key42/Key422/0": "one",
    "ExSTChildLevel1/Key42/Key422/1": "two",
    "ExSTChildLevel1/Key42/Key422/2": "three",
    "ExSTChildLevel1/ExSTChildLevel2/Key431/Key4311": "val4311",
}

st := &ExST{}

err := KVMapToStruct(input, "", st)
if err != nil {
    return
}

fmt.Printf("%++v\n", st)

Output:

&{Key1:val1 Key2:2 Key3:[1 2 3] ExSTChildLevel1:{Key41:val41 Key42:map[Key421:val421 Key422:[one two three]] ExSTChildLevel2:{Key431:map[Key4311:val4311]}}}

func MapToFlattenMap Uses

func MapToFlattenMap(in map[string]interface{}, prefix string) map[string]interface{}

MapToFlattenMap converts a nested map to a flatten map.

Code:

input := map[string]interface{}{
    "key1": "val1",
    "key2": 2,
    "key3": []int{1, 2, 3},
    "key4": map[string]interface{}{
        "key41": "val41",
        "key42": map[string]interface{}{
            "key421": "val421",
            "key422": []string{"one", "two", "three"},
        },
        "key43": map[string]interface{}{
            "key431": map[string]interface{}{
                "key4311": "val4311",
            },
        },
    },
}

o := MapToFlattenMap(input, "test")

keys := []string{}

for k := range o {
    keys = append(keys, k)
}

sort.Strings(keys)

for _, key := range keys {
    fmt.Println(key, ":", o[key])
}

Output:

test/key1 : val1
test/key2 : 2
test/key3 : [1 2 3]
test/key4/key41 : val41
test/key4/key42/key421 : val421
test/key4/key42/key422 : [one two three]
test/key4/key43/key431/key4311 : val4311

func MapToKVMap Uses

func MapToKVMap(in map[string]interface{}, prefix string) map[string]interface{}

MapToKVMap convert a nested map to a KV map.

Code:

input := map[string]interface{}{
    "key1": "val1",
    "key2": 2,
    "key3": []int{1, 2, 3},
    "key4": map[string]interface{}{
        "key41": "val41",
        "key42": map[string]interface{}{
            "key421": "val421",
            "key422": []string{"one", "two", "three"},
        },
    },
}

output := map[string]interface{}{
    "test/key1":                "val1",
    "test/key2":                2,
    "test/key3/0":              1,
    "test/key3/1":              2,
    "test/key3/2":              3,
    "test/key4/key41":          "val41",
    "test/key4/key42/key421":   "val421",
    "test/key4/key42/key422/0": "one",
    "test/key4/key42/key422/1": "two",
    "test/key4/key42/key422/2": "three",
}

o := MapToKVMap(input, "test")

// Compare result with expected output
fmt.Println(reflect.DeepEqual(o, output))

Output:

true

type KVMapStruct Uses

type KVMapStruct struct {
    // Path is consul key parent to store struct's fields
    Path string
    // Client is consul client
    Client *consul.Client
}

KVMapStruct contains consul informations.

func NewKVMapStruct Uses

func NewKVMapStruct(url, token, path string) (*KVMapStruct, error)

NewKVMapStruct creates a new *KVMapStruct. URL format is ip:port.

func (*KVMapStruct) ConsulKVToMap Uses

func (kms *KVMapStruct) ConsulKVToMap() (map[string]interface{}, error)

ConsulKVToMap gets list of all consul keys from kvmapstruct path and match them to a map[string]interface{}.

Code:

input := map[string]interface{}{
    "test/Key1":                      "val1",
    "test/Key2":                      "2",
    "test/Key3/0":                    "1",
    "test/Key3/1":                    "2",
    "test/Key3/2":                    "3",
    "test/Key4/Key41":                "val41",
    "test/Key4/Key42/Key421":         "val421",
    "test/Key4/Key42/Key422/0":       "one",
    "test/Key4/Key42/Key422/1":       "two",
    "test/Key4/Key42/Key422/2":       "three",
    "test/Key4/Key43/Key431/Key4311": "val4311",
}

kms, err := NewKVMapStruct("localhost:8500", "adf4238a-882b-9ddc-4a9d-5b6758e4159e", "test")
if err != nil {
    return
}

kms.Path = "test"

// Insert data in to consul
for k, v := range input {
    kv := &consul.KVPair{
        Key:   k,
        Value: []byte(v.(string)),
    }

    _, err := kms.Client.KV().Put(kv, nil)
    if err != nil {
        return
    }
}

out, err := kms.ConsulKVToMap()
if err != nil {
    return
}

fmt.Println(out)

func (*KVMapStruct) ConsulKVToStruct Uses

func (kms *KVMapStruct) ConsulKVToStruct(out interface{}) error

ConsulKVToStruct gets list of all consul keys from kvmapstruct path and match them to the given struct in argument. Out argument must be a initialiezed pointer to a Go struct. Its substructs can be a pointer to a struct, embedded struct or struct. If it is a pointer, it must be initialized.

Code:

type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    ExSTChildLevel1
}

input := map[string]interface{}{
    "test/Key1":                           "val1",
    "test/Key2":                           "2",
    "test/Key3/0":                         "1",
    "test/Key3/1":                         "2",
    "test/Key3/2":                         "3",
    "test/ExSTChildLevel1/Key41":          "val41",
    "test/ExSTChildLevel1/Key42/Key421":   "val421",
    "test/ExSTChildLevel1/Key42/Key422/0": "one",
    "test/ExSTChildLevel1/Key42/Key422/1": "two",
    "test/ExSTChildLevel1/Key42/Key422/2": "three",
    "test/ExSTChildLevel1/ExSTChildLevel2/Key431/Key4311": "val4311",
}

kms, err := NewKVMapStruct("localhost:8500", "adf4238a-882b-9ddc-4a9d-5b6758e4159e", "test")
if err != nil {
    return
}

st := &ExST{}

kms.Path = "test"

// Insert data in to consul
for k, v := range input {
    kv := &consul.KVPair{
        Key:   k,
        Value: []byte(v.(string)),
    }

    _, err := kms.Client.KV().Put(kv, nil)
    if err != nil {
        return
    }
}

err = kms.ConsulKVToStruct(st)
if err != nil {
    return
}

kms.Client.KV().DeleteTree(kms.Path, nil)

fmt.Printf("%++v\n", st)

Output:

&{Key1:val1 Key2:2 Key3:[1 2 3] ExSTChildLevel1:{Key41:val41 Key42:map[Key421:val421 Key422:[one two three]] ExSTChildLevel2:{Key431:map[Key4311:val4311]}}}

Code:

type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    Key43 ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    Key4 ExSTChildLevel1
}

input := map[string]interface{}{
    "test/Key1":                      "val1",
    "test/Key2":                      "2",
    "test/Key3/0":                    "1",
    "test/Key3/1":                    "2",
    "test/Key3/2":                    "3",
    "test/Key4/Key41":                "val41",
    "test/Key4/Key42/Key421":         "val421",
    "test/Key4/Key42/Key422/0":       "one",
    "test/Key4/Key42/Key422/1":       "two",
    "test/Key4/Key42/Key422/2":       "three",
    "test/Key4/Key43/Key431/Key4311": "val4311",
}

kms, err := NewKVMapStruct("localhost:8500", "adf4238a-882b-9ddc-4a9d-5b6758e4159e", "test")
if err != nil {
    return
}

st := &ExST{}

kms.Path = "test"

// Insert data in to consul
for k, v := range input {
    kv := &consul.KVPair{
        Key:   k,
        Value: []byte(v.(string)),
    }

    _, err := kms.Client.KV().Put(kv, nil)
    if err != nil {
        return
    }
}

err = kms.ConsulKVToStruct(st)
if err != nil {
    return
}

kms.Client.KV().DeleteTree(kms.Path, nil)

fmt.Printf("%++v\n", st)

Output:

&{Key1:val1 Key2:2 Key3:[1 2 3] Key4:{Key41:val41 Key42:map[Key421:val421 Key422:[one two three]] Key43:{Key431:map[Key4311:val4311]}}}

Code:


type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    Key43 *ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    Key4 *ExSTChildLevel1
}

input := map[string]interface{}{
    "test/Key1":                      "val1",
    "test/Key2":                      "2",
    "test/Key3/0":                    "1",
    "test/Key3/1":                    "2",
    "test/Key3/2":                    "3",
    "test/Key4/Key41":                "val41",
    "test/Key4/Key42/Key421":         "val421",
    "test/Key4/Key42/Key422/0":       "one",
    "test/Key4/Key42/Key422/1":       "two",
    "test/Key4/Key42/Key422/2":       "three",
    "test/Key4/Key43/Key431/Key4311": "val4311",
}

kms, err := NewKVMapStruct("localhost:8500", "adf4238a-882b-9ddc-4a9d-5b6758e4159e", "test")
if err != nil {
    return
}

st := &ExST{
    Key4: &ExSTChildLevel1{
        Key43: &ExSTChildLevel2{},
    },
}

kms.Path = "test"

// Insert data in to consul
for k, v := range input {
    kv := &consul.KVPair{
        Key:   k,
        Value: []byte(v.(string)),
    }

    _, err := kms.Client.KV().Put(kv, nil)
    if err != nil {
        return
    }
}

err = kms.ConsulKVToStruct(st)
if err != nil {
    return
}

fmt.Println(st)

func (*KVMapStruct) MapToConsulKV Uses

func (kms *KVMapStruct) MapToConsulKV(input interface{}) error

MapToConsulKV converts and saves the map to Consul KV store. input argument must be a map[string]interface{}.

Code:

input := map[string]interface{}{
    "key1": "val1",
    "key2": 2,
    "key3": []int{1, 2, 3},
    "key4": map[string]interface{}{
        "key41": "val41",
        "key42": map[string]interface{}{
            "key421": "val421",
            "key422": []string{"one", "two", "three"},
        },
    },
}

kms, err := NewKVMapStruct("localhost:8500", "adf4238a-882b-9ddc-4a9d-5b6758e4159e", "test")
if err != nil {
    return
}

out := make(map[string]interface{})
keys := []string{}

kms.Path = "nestedmap"
err = kms.MapToConsulKV(input)
if err != nil {
    return
}

pairs, _, err := kms.Client.KV().List(kms.Path, nil)
if err != nil {
    return
}

for _, kv := range pairs {
    out[kv.Key] = string(kv.Value)
    keys = append(keys, kv.Key)
}

sort.Strings(keys)

// Compare result with expected output
for _, key := range keys {
    fmt.Println(key, ":", out[key])
}

Output:

nestedmap/key1 : val1
nestedmap/key2 : 2
nestedmap/key3/0 : 1
nestedmap/key3/1 : 2
nestedmap/key3/2 : 3
nestedmap/key4/key41 : val41
nestedmap/key4/key42/key421 : val421
nestedmap/key4/key42/key422/0 : one
nestedmap/key4/key42/key422/1 : two
nestedmap/key4/key42/key422/2 : three

func (*KVMapStruct) MapToKVPairs Uses

func (kms *KVMapStruct) MapToKVPairs(in map[string]interface{}, prefix string) (consul.KVPairs, error)

MapToKVPairs convert a nested map to an array of Consul KV pairs

Code:

input := map[string]interface{}{
    "key1": "val1",
    "key2": 2,
    "key3": []int{1, 2, 3},
    "key4": map[string]interface{}{
        "key41": "val41",
        "key42": map[string]interface{}{
            "key421": "val421",
            "key422": []string{"one", "two", "three"},
        },
    },
}

kms, err := NewKVMapStruct("", "", "test")
if err != nil {
    fmt.Println(err)
    return
}

o, err := kms.MapToKVPairs(input, kms.Path)
if err != nil {
    fmt.Println(err)
    return
}

keys := []string{}
out := make(map[string]interface{})

for _, kv := range o {
    out[kv.Key] = string(kv.Value)
    keys = append(keys, kv.Key)
}

sort.Strings(keys)

// Compare result with expected output
for _, key := range keys {
    fmt.Println(key, ":", out[key])
}

Output:

test/key1 : val1
test/key2 : 2
test/key3/0 : 1
test/key3/1 : 2
test/key3/2 : 3
test/key4/key41 : val41
test/key4/key42/key421 : val421
test/key4/key42/key422/0 : one
test/key4/key42/key422/1 : two
test/key4/key42/key422/2 : three

func (*KVMapStruct) StructToConsulKV Uses

func (kms *KVMapStruct) StructToConsulKV(input interface{}) error

StructToConsulKV converts and saves the struct to Consul KV store input argument must be a Go struct.

Code:

type ExSTChildLevel2 struct {
    Key431 map[string]interface{}
}

type ExSTChildLevel1 struct {
    Key41 string
    Key42 map[string]interface{}
    Key43 *ExSTChildLevel2
}

type ExST struct {
    Key1 string
    Key2 int
    Key3 []int
    Key4 *ExSTChildLevel1
}

input := ExST{
    Key1: "val1",
    Key2: 2,
    Key3: []int{1, 2, 3},
    Key4: &ExSTChildLevel1{
        Key41: "val41",
        Key42: map[string]interface{}{
            "Key421": "val421",
            "Key422": []string{"one", "two", "three"},
        },
        Key43: &ExSTChildLevel2{
            Key431: map[string]interface{}{
                "Key4311": "val4311",
            },
        },
    },
}

kms, err := NewKVMapStruct("localhost:8500", "adf4238a-882b-9ddc-4a9d-5b6758e4159e", "test")
if err != nil {
    return
}

out := make(map[string]interface{})
keys := []string{}

kms.Path = "nestedstructmap"

err = kms.StructToConsulKV(input)
if err != nil {
    fmt.Println(err)
    return
}

pairs, _, err := kms.Client.KV().List(kms.Path, nil)
if err != nil {
    fmt.Println(err)
    return
}

for _, kv := range pairs {
    out[kv.Key] = string(kv.Value)
    keys = append(keys, kv.Key)
}

sort.Strings(keys)

for _, key := range keys {
    fmt.Println(key, ":", out[key])
}

Output:

nestedstructmap/Key1 : val1
nestedstructmap/Key2 : 2
nestedstructmap/Key3/0 : 1
nestedstructmap/Key3/1 : 2
nestedstructmap/Key3/2 : 3
nestedstructmap/Key4/Key41 : val41
nestedstructmap/Key4/Key42/Key421 : val421
nestedstructmap/Key4/Key42/Key422/0 : one
nestedstructmap/Key4/Key42/Key422/1 : two
nestedstructmap/Key4/Key42/Key422/2 : three
nestedstructmap/Key4/Key43/Key431/Key4311 : val4311

Package kvmapstruct imports 9 packages (graph). Updated 2018-09-02. Refresh now. Tools for package owners.