README
¶
Starlark to Go struct converter
The starstruct package implements conversion of Go structs to Starlark StringDict and from Starlark StringDict to a Go struct.
It uses the starlark struct tag key to indicate the name of the corresponding Starlark StringDict key for a field and optional conversion options, as is common in Go marshalers (such as JSON and XML).
Since Starlark is a useful language for configuration, and configuration can often be represented by a well-defined Go struct, the idea for this package is to make it easy to provide the default configuration (as a Go struct) to a Starlark program, and read back the final configuration after the Starlark program's execution, so that the Go code can use the Go configuration struct from there on.
The code documentation is the canonical source for documentation.
Installation
Note that starstruct requires Go 1.20+.
$ go get github.com/mna/starstruct
Example
From example_test.go:
func Example() {
const script = `
def set_server(srv):
srv["ports"].append(2020)
def set_admin(adm):
adm["name"] = "root"
adm["is_admin"] = True
roles = adm["roles"]
adm["roles"] = roles.union(["editor", "admin"])
set_server(server)
set_admin(user)
`
type Server struct {
Addr string `starlark:"addr"`
Ports []int `starlark:"ports"`
}
type User struct {
Name string `starlark:"name"`
IsAdmin bool `starlark:"is_admin"`
Ignored int `starlark:"-"`
Roles []string `starlark:"roles,asset"`
}
type S struct {
Server Server `starlark:"server"`
User User `starlark:"user"`
}
// initialize with default values for the starlark script
s := S{
Server: Server{Addr: "localhost", Ports: []int{80, 443}},
User: User{Name: "Martin", Roles: []string{"viewer"}, Ignored: 42},
}
initialVars := make(starlark.StringDict)
if err := starstruct.ToStarlark(s, initialVars); err != nil {
log.Fatal(err)
}
// execute the starlark script (it doesn't create any new variables, but if
// it did, we would capture them in outputVars and merge all global vars
// together before calling FromStarlark).
var th starlark.Thread
outputVars, err := starlark.ExecFile(&th, "example", script, initialVars)
if err != nil {
log.Fatal(err)
}
allVars := mergeStringDicts(nil, initialVars, outputVars)
if err := starstruct.FromStarlark(allVars, &s); err != nil {
log.Fatal(err)
}
fmt.Printf("%v", s)
// Output:
// {{localhost [80 443 2020]} {root true 42 [viewer editor admin]}}
}
License
The BSD 3-Clause license.
Documentation
¶
Overview ¶
Package starstruct implements conversion of Go structs to Starlark string dictionary (starlark.StringDict) and vice versa. By default, the exported struct field's name is used as corresponding Starlark dictionary key. The 'starlark' struct tag can be used to alter that behavior and to specify other options, as is common in Go marshalers (such as in Go's JSON and XML standard library packages).
See the documentation of ToStarlark and FromStarlark for more information about the encoding and decoding processing.
The Starlark dictionary can hold any hashable value as a key, but only a subset is supported by starstruct. It can only convert to and from dictionaries where the key is a valid Go identifier or representable as a valid struct tag name, excluding any names and characters with special meaning.
Those are:
- "-" : the name used to ignore a field
- "," : the character used to separate the name and the conversion options
Example ¶
const script = `
def set_server(srv):
srv["ports"].append(2020)
def set_admin(adm):
adm["name"] = "root"
adm["is_admin"] = True
roles = adm["roles"]
adm["roles"] = roles.union(["editor", "admin"])
set_server(server)
set_admin(user)
`
type Server struct {
Addr string `starlark:"addr"`
Ports []int `starlark:"ports"`
}
type User struct {
Name string `starlark:"name"`
IsAdmin bool `starlark:"is_admin"`
Ignored int `starlark:"-"`
Roles []string `starlark:"roles,asset"`
}
type S struct {
Server Server `starlark:"server"`
User User `starlark:"user"`
}
// initialize with default values for the starlark script
s := S{
Server: Server{Addr: "localhost", Ports: []int{80, 443}},
User: User{Name: "Martin", Roles: []string{"viewer"}, Ignored: 42},
}
initialVars := make(starlark.StringDict)
if err := starstruct.ToStarlark(s, initialVars); err != nil {
log.Fatal(err)
}
// execute the starlark script (it doesn't create any new variables, but if
// it did, we would capture them in outputVars and merge all global vars
// together before calling FromStarlark).
var th starlark.Thread
outputVars, err := starlark.ExecFile(&th, "example", script, initialVars)
if err != nil {
log.Fatal(err)
}
allVars := mergeStringDicts(nil, initialVars, outputVars)
if err := starstruct.FromStarlark(allVars, &s); err != nil {
log.Fatal(err)
}
fmt.Printf("%v", s)
Output: {{localhost [80 443 2020]} {root true 42 [viewer editor admin]}}
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func FromStarlark ¶
func FromStarlark(vals starlark.StringDict, dst any, opts ...FromOption) error
FromStarlark loads the starlark values from vals into a destination Go struct. It supports the following data types from Starlark to Go, and all Go types can also be a pointer to that type:
- NoneType => nil (Go field must be a pointer, slice or map)
- Bool => bool
- Bytes => []byte or string
- String => []byte or string
- Float => float32 or float64
- Int => int, uint, and any sized (u)int if it fits
- Dict => struct
- List => slice of any supported Go type
- Tuple => slice of any supported Go type
- Set => map[T]bool or []T where T is any supported Go type
In addition to those conversions, if the Go type is starlark.Value (or a pointer to that type), then the starlark value is assigned as-is.
Additional conversions can be supported via a custom converter (see CustomFromConverter).
It panics if dst is not a non-nil pointer to an addressable and settable struct. If a target Go field does have a matching key in the starlark dictionary, it is unmodified.
Decoding into a slice follows the same behavior as JSON umarshaling: it resets the slice length to zero and then appends each element to the slice. As a special case, to decode an empty starlark List, Tuple or Set into a slice, it replaces the slice with a new empty slice.
Decoding a Set into a map also follows the same behavior as JSON unmarshaling: if the map is nil, it allocates a new map. Otherwise it reuses the existing map, keeping existing entries. It then stores each Set key with a true value into the map.
Embedded fields in structs are supported as follows:
- The field must be exported
- The type of the field must be a struct or a pointer to a struct
- If the embedded field has no starlark name specified in its struct tag, the starlark values are decoded into the fields of the embedded struct as if they were part of the parent struct.
- If the embedded field has a starlark name specified in its struct tag (and that name is not "-"), the starlark dictionary corresponding to that name is decoded to that embedded struct.
func ToStarlark ¶
func ToStarlark(vals any, dst starlark.StringDict, opts ...ToOption) error
ToStarlark converts the values from the Go struct to corresponding Starlark values stored into a destination Starlark string dictionary. Existing values in dst, if any, are left untouched unless the Go struct conversion overwrites them.
It supports the following data types from Go to Starlark, and all Go types can also be a pointer to that type:
- nil (pointer, slice or map) => NoneType
- bool => Bool
- []byte => Bytes
- string => String
- float32 or float64 => Float
- int, uint, and any sized (u)int => Int
- struct => Dict
- slice of any supported Go type => List
- map[T]bool => Set
In addition to those conversions, if the Go type is starlark.Value (or a pointer to that type), then the starlark value is transferred as-is.
Additional conversions can be supported via a custom converter (see CustomToConverter).
Conversion can be further controlled by using struct tags. Besides the naming of the starlark variable, a comma-separated argument can be provided to control the target encoding. The following arguments are supported:
- For string fields, `starlark:"name,asbytes"` to convert to Bytes
- For []byte fields, `starlark:"name,asstring"` to convert to String
- For []byte ([]uint8) fields, `starlark:"name,aslist"` to convert to List (of Int)
- For slices (including []byte), `starlark:"name,astuple"` to convert to Tuple
- For slices (including []byte), `starlark:"name,asset"` to convert to Set
Any level of conversion arguments can be provided, to support for nested conversions, e.g. this would convert to a Set of Tuples of Bytes:
- [][]string `starlark:"name,asset,astuple,asbytes"`
Embedded fields in structs are supported as follows:
- The field must be exported
- The type of the field must be a struct or a pointer to a struct
- If the embedded field has no starlark name specified in its struct tag, the fields of the embedded struct are encoded as if they were part of the parent struct.
- If the embedded field has a starlark name specified in its struct tag (and that name is not "-"), the embedded struct is encoded as a starlark dictionary under that name.
ToStarlark panics if vals is not a struct or a pointer to a struct. If dst is nil, it proceeds with the conversion but the results of it will not be visible to the caller (it can be used to validate the Go to Starlark conversion).
Types ¶
type ConvOp ¶
type ConvOp string
ConvOp is the type indicating the conversion operation, either ToStarlark or FromStarlark.
type CustomConvError ¶
type CustomConvError struct {
// Op indicates if this is in a FromStarlark or ToStarlark call.
Op ConvOp
// Path indicates the Go struct path to the field in error.
Path string
// StarVal is the starlark value in a From conversion, nil otherwise.
StarVal starlark.Value
// GoVal is the Go value associated with the error.
GoVal reflect.Value
// Err is the error as returned by the custom converter.
Err error
}
CustomConvError wraps an error that occurred in a custom converter with additional information about the values and struct path involved.
func (*CustomConvError) Error ¶
func (e *CustomConvError) Error() string
Error returns the error message for the custom conversion error.
func (*CustomConvError) Unwrap ¶
func (e *CustomConvError) Unwrap() error
Unwrap returns the underlying custom converter error.
type FromOption ¶
type FromOption func(*decoder)
FromOption is the type of the decoding options that can be provided to the FromStarlark function.
func CustomFromConverter ¶
func CustomFromConverter(fn func(path string, starVal starlark.Value, goVal reflect.Value) (didConvert bool, err error)) FromOption
CustomFromConverter allows specifying a custom converter from a starlark value to a Go type. The function receives the Go struct path, the starlark value to convert and the target Go value where to store the result of the conversion as arguments. It returns whether it did convert the value and an optional error.
If an error is returned, it will be wrapped in a CustomConvError and will be returned as part of the errors collected in the call to FromStarlark, and the conversion of that starlark value is skipped.
If it returns true for the didConvert boolean return value, the starlark value is considered converted and is skipped. Otherwise, if it returns false and a nil error, the standard conversion is applied to the starlark value.
func MaxFromErrors ¶
func MaxFromErrors(max int) FromOption
MaxFromErrors sets the maximum numbers of errors to return. If too many errors are reached, the error returned by FromStarlark will wrap max + 1 errors, the last one being an error indicating that the maximum was reached. If max <= 0, all errors will be returned.
type NumberError ¶
type NumberError struct {
// Reason indicates the cause of the number conversion failure.
Reason NumberFailReason
// Path indicates the Go struct path to the field in error.
Path string
// StarNum is the Starlark integer or float value associated with the error.
StarNum starlark.Value
// GoVal is the target Go value where the number was attempted to be stored.
GoVal reflect.Value
}
NumberError represents a numeric conversion error from a starlark Int or Float to a Go number type. The Reason field indicates why the conversion failed: whether it's because the number could not be exactly represented in the target Go number type, or because it was out of range.
The distinction is because the source value may be in the range but unrepresentable, for example the float 1.234 is in the range of values for an int8 Go type, but it cannot be exactly represented - the fraction would be lost. Another example is for a float value that requires 64 bits to be represented, if stored in a Go float32, it might be inside the range of the float32, but not exactly representable (the value would not be the same).
To determine if a value is the same after conversion from float64 to float32, starstruct checks if the absolute difference is smaller than epsilon.
func (*NumberError) Error ¶
func (e *NumberError) Error() string
Error returns the error message for the number conversion failure.
type NumberFailReason ¶
type NumberFailReason byte
NumberFailReason defines the failure reasons for a number conversion.
const ( NumCannotExactlyRepresent NumberFailReason = iota NumOutOfRange )
List of number failure reasons.
type StarlarkContainerError ¶
type StarlarkContainerError struct {
// Path indicates the Go struct path to the field in error.
Path string
// Container is the Starlark Set, Dict or StringDict that failed to insert
// the value.
Container starlark.Value
// Key is the key (always a string) at which the value was being inserted,
// nil if the container is not a Dict or StringDict.
Key starlark.Value
// Value is the value that failed to be inserted in the container.
Value starlark.Value
// GoVal is the Go value that converted to the Value being inserted.
GoVal reflect.Value
// Err is the error returned by Starlark when inserting into the container.
// The StarlarkContainerError wraps this error.
Err error
}
StarlarkContainerError indicates an error that occurred when inserting a value into a Starlark container such as a dictionary or a set. It wraps the actual error returned by Starlark and provides additional information about where in the Go struct encoding the error occurred.
func (*StarlarkContainerError) Error ¶
func (e *StarlarkContainerError) Error() string
Error returns the error message for the Starlark container insertion.
func (*StarlarkContainerError) Unwrap ¶
func (e *StarlarkContainerError) Unwrap() error
Unwrap returns the underlying Starlark error.
type ToOption ¶
type ToOption func(*encoder)
ToOption is the type of the encoding options that can be provided to the ToStarlark function.
func CustomToConverter ¶
func CustomToConverter(fn func(path string, goVal reflect.Value, tagOpts []string) (starlark.Value, error)) ToOption
CustomToConverter allows specifying a custom converter from a Go value to a starlark value. The function receives the Go struct path, the Go value as a reflect.Value and the struct tag options applied to that Go value (if any) as arguments. It returns the resulting starlark value and an optional error.
If an error is returned, it will be wrapped in a CustomConvError and will be returned as part of the errors collected in the call to ToStarlark, and the conversion of that Go value is skipped.
If it returns a non-nil starlark value, the Go value is considered converted and is skipped. Otherwise, if it returns a nil value and a nil error, the standard conversion is applied to the Go value.
func MaxToErrors ¶
MaxToErrors sets the maximum numbers of errors to return. If too many errors are reached, the error returned by ToStarlark will wrap max + 1 errors, the last one being an error indicating that the maximum was reached. If max <= 0, all errors will be returned.
type TypeError ¶
type TypeError struct {
// Op indicates if this is in a FromStarlark or ToStarlark call.
Op ConvOp
// Path indicates the Go struct path to the field in error.
Path string
// StarVal is the starlark value in a From conversion, nil otherwise.
StarVal starlark.Value
// GoVal is the Go value associated with the error.
GoVal reflect.Value
// Embedded is true if the Go value is an embedded struct field.
Embedded bool
}
TypeError represents a starstruct conversion error. Errors returned from ToStarlark and FromStarlark may wrap errors of this type - that is, the returned error is created by using the Go standard library errors.Join function, and the errors wrapped in that call may contain starstruct.TypeError errors.
Source Files