README
¶
go-jsonnet
This an implementation of Jsonnet in pure Go. It is feature complete but is not as heavily exercised as the Jsonnet C++ implementation. Please try it out and give feedback.
This code is known to work on Go 1.8 and above. We recommend always using the newest stable release of Go.
Installation instructions
go get github.com/sh0rez/go-jsonnet/cmd/jsonnet
Build instructions (go 1.11+)
git clone git@github.com:google/go-jsonnet.git
cd go-jsonnet
go build ./cmd/jsonnet
go build ./cmd/jsonnetfmt
To build with Bazel instead:
git clone git@github.com:google/go-jsonnet.git
cd go-jsonnet
git submodule init
git submodule update
bazel build //cmd/jsonnet
bazel build //cmd/jsonnetfmt
The resulting jsonnet program will then be available at a platform-specific path, such as bazel-bin/cmd/jsonnet/darwin_amd64_stripped/jsonnet for macOS.
Bazel also accommodates cross-compiling the program. To build the jsonnet program for various popular platforms, run the following commands:
| Target platform | Build command |
|---|---|
| Current host | bazel build //cmd/jsonnet |
| Linux | bazel build --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 //cmd/jsonnet |
| macOS | bazel build --platforms=@io_bazel_rules_go//go/toolchain:darwin_amd64 //cmd/jsonnet |
| Windows | bazel build --platforms=@io_bazel_rules_go//go/toolchain:windows_amd64 //cmd/jsonnet |
For additional target platform names, see the per-Go release definitions here in the rules_go Bazel package.
Additionally if any files were moved around, see the section Keeping the Bazel files up to date.
Running tests
./tests.sh # Also runs `go test ./...`
Running Benchmarks
Setup
go get golang.org/x/tools/cmd/benchcmp
- Make sure you build a jsonnet binary prior to making changes.
go build ./cmd/jsonnet -o jsonnet-old
- Make changes (iterate as needed), and rebuild new binary
go build ./cmd/jsonnet
- Run benchmark:
# e.g. ./benchmark.sh Builtin
./benchmark.sh <TestNameFilter>
Implementation Notes
We are generating some helper classes on types by using http://clipperhouse.github.io/gen/. Do the following to regenerate these if necessary:
go get github.com/clipperhouse/gen
go get github.com/clipperhouse/set
export PATH=$PATH:$GOPATH/bin # If you haven't already
go generate
Updating and modifying the standard library
Standard library source code is kept in cpp-jsonnet submodule, because it is shared with Jsonnet C++
implementation.
For performance reasons we perform preprocessing on the standard library, so for the changes to be visible, regeneration is necessary:
git submodule init
git submodule update
go run cmd/dumpstdlibast/dumpstdlibast.go cpp-jsonnet/stdlib/std.jsonnet > astgen/stdast.go
The above command creates the astgen/stdast.go file which puts the desugared standard library into the right data structures, which lets us avoid the parsing overhead during execution. Note that this step is not necessary to perform manually when building with Bazel; the Bazel target regenerates the astgen/stdast.go (writing it into Bazel's build sandbox directory tree) file when necessary.
Keeping the Bazel files up to date
Note that we maintain the Go-related Bazel targets with the Gazelle tool. The Go module (go.mod in the root directory) remains the primary source of truth. Gazelle analyzes both that file and the rest of the Go files in the repository to create and adjust appropriate Bazel targets for building Go packages and executable programs.
After changing any dependencies within the files covered by this Go module, it is helpful to run go mod tidy to ensure that the module declarations match the state of the Go source code. In order to synchronize the Bazel rules with material changes to the Go module, run the following command to invoke Gazelle's update-repos command:
bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro=bazel/deps.bzl%jsonnet_go_dependencies
Similarly, after adding or removing Go source files, it may be necessary to synchronize the Bazel rules by running the following command:
bazel run //:gazelle
Documentation
¶
Overview ¶
Package jsonnet implements a parser and evaluator for jsonnet.
Jsonnet is a domain specific configuration language that helps you define JSON data. Jsonnet lets you compute fragments of JSON within the structure, bringing the same benefit to structured data that templating languages bring to plain text.
See http://jsonnet.org/ for a full language description and tutorial.
Index ¶
- func SnippetToAST(filename string, snippet string) (ast.Node, error)
- func Version() string
- type ColorFormatter
- type Contents
- type ErrorFormatter
- type FileImporter
- type Importer
- type MemoryImporter
- type NativeFunction
- type RuntimeError
- type VM
- func (vm *VM) Evaluate(node ast.Node) (val string, err error)
- func (vm *VM) EvaluateMulti(node ast.Node) (output interface{}, err error)
- func (vm *VM) EvaluateSnippet(filename string, snippet string) (json string, formattedErr error)
- func (vm *VM) EvaluateSnippetMulti(filename string, snippet string) (files map[string]string, formattedErr error)
- func (vm *VM) EvaluateSnippetStream(filename string, snippet string) (docs []string, formattedErr error)
- func (vm *VM) EvaluateStream(node ast.Node) (output interface{}, err error)
- func (vm *VM) ExtCode(key string, val string)
- func (vm *VM) ExtVar(key string, val string)
- func (vm *VM) ImportAST(importedFrom, importedPath string) (contents ast.Node, foundAt string, err error)
- func (vm *VM) ImportData(importedFrom, importedPath string) (contents string, foundAt string, err error)
- func (vm *VM) Importer(i Importer)
- func (vm *VM) NativeFunction(f *NativeFunction)
- func (vm *VM) ResolveImport(importedFrom, importedPath string) (foundAt string, err error)
- func (vm *VM) TLACode(key string, val string)
- func (vm *VM) TLAVar(key string, val string)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SnippetToAST ¶
SnippetToAST parses a snippet and returns the resulting AST.
Types ¶
type ColorFormatter ¶
ColorFormatter represents a function that writes to the terminal using color.
type Contents ¶ added in v0.11.2
type Contents struct {
// contains filtered or unexported fields
}
Contents is a representation of imported data. It is a simple string wrapper, which makes it easier to enforce the caching policy.
func MakeContents ¶ added in v0.11.2
MakeContents creates Contents from a string.
type ErrorFormatter ¶
type ErrorFormatter interface {
// Format static, runtime, and unexpected errors prior to printing them.
Format(err error) string
// Set the the maximum length of stack trace before cropping.
SetMaxStackTraceSize(size int)
// Set the color formatter for the location color.
SetColorFormatter(color ColorFormatter)
}
An ErrorFormatter formats errors with stacktraces and color.
type FileImporter ¶
type FileImporter struct {
JPaths []string
// contains filtered or unexported fields
}
FileImporter imports data from the filesystem.
type Importer ¶
type Importer interface {
// Import fetches data from a given path. It may be relative
// to the file where we do the import. What "relative path"
// means depends on the importer.
//
// It is required that:
// a) for given (importedFrom, importedPath) the same
// (contents, foundAt) are returned on subsequent calls.
// b) for given foundAt, the contents are always the same
//
// It is recommended that if there are multiple locations that
// need to be probed (e.g. relative + multiple library paths)
// then all results of all attempts will be cached separately,
// both nonexistence and contents of existing ones.
// FileImporter may serve as an example.
//
// Importing the same file multiple times must be a cheap operation
// and shouldn't involve copying the whole file - the same buffer
// should be returned.
Import(importedFrom, importedPath string) (contents Contents, foundAt string, err error)
}
An Importer imports data from a path. TODO(sbarzowski) caching of errors (may require breaking changes)
type MemoryImporter ¶
MemoryImporter "imports" data from an in-memory map.
type NativeFunction ¶
type NativeFunction struct {
Func func([]interface{}) (interface{}, error)
Params ast.Identifiers
Name string
}
NativeFunction represents a function implemented in Go.
type RuntimeError ¶
type RuntimeError struct {
StackTrace []traceFrame
Msg string
}
RuntimeError is an error discovered during evaluation of the program
func (RuntimeError) Error ¶
func (err RuntimeError) Error() string
type VM ¶
type VM struct {
MaxStack int
ErrorFormatter ErrorFormatter
StringOutput bool
// contains filtered or unexported fields
}
VM is the core interpreter and is the touchpoint used to parse and execute Jsonnet.
func (*VM) Evaluate ¶ added in v0.14.0
Evaluate evaluates a Jsonnet program given by an Abstract Syntax Tree and returns serialized JSON as string. TODO(sbarzowski) perhaps is should return JSON in standard Go representation
func (*VM) EvaluateMulti ¶ added in v0.14.0
EvaluateMulti evaluates a Jsonnet program given by an Abstract Syntax Tree and returns key-value pairs. The keys are strings and the values are JSON strigns (serialized JSON).
func (*VM) EvaluateSnippet ¶
EvaluateSnippet evaluates a string containing Jsonnet code, return a JSON string.
The filename parameter is only used for error messages.
func (*VM) EvaluateSnippetMulti ¶
func (vm *VM) EvaluateSnippetMulti(filename string, snippet string) (files map[string]string, formattedErr error)
EvaluateSnippetMulti evaluates a string containing Jsonnet code to key-value pairs. The keys are field name strings and the values are JSON strings.
The filename parameter is only used for error messages.
func (*VM) EvaluateSnippetStream ¶
func (vm *VM) EvaluateSnippetStream(filename string, snippet string) (docs []string, formattedErr error)
EvaluateSnippetStream evaluates a string containing Jsonnet code to an array. The array is returned as an array of JSON strings.
The filename parameter is only used for error messages.
func (*VM) EvaluateStream ¶ added in v0.14.0
EvaluateStream evaluates a Jsonnet program given by an Abstract Syntax Tree and returns an array of JSON strings.
func (*VM) ImportAST ¶ added in v0.15.0
func (vm *VM) ImportAST(importedFrom, importedPath string) (contents ast.Node, foundAt string, err error)
ImportAST fetches the Jsonnet AST just as if it was imported from a Jsonnet file located at `importedFrom`. It shares the cache with the actual evaluation.
func (*VM) ImportData ¶ added in v0.15.0
func (vm *VM) ImportData(importedFrom, importedPath string) (contents string, foundAt string, err error)
ImportData fetches the data just as if it was imported from a Jsonnet file located at `importedFrom`. It shares the cache with the actual evaluation.
func (*VM) NativeFunction ¶
func (vm *VM) NativeFunction(f *NativeFunction)
NativeFunction registers a native function.
func (*VM) ResolveImport ¶ added in v0.15.0
ResolveImport finds the actual path where the imported file can be found. It will cache the contents of the file immediately as well, to avoid the possibility of the file disappearing after being checked.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package ast provides AST nodes and ancillary structures and algorithms.
|
Package ast provides AST nodes and ancillary structures and algorithms. |
|
cmd
|
|
|
internal
|
|
|
dump
Package dump can dump a Go data structure to Go source file, so that it can be statically embedded into other code.
|
Package dump can dump a Go data structure to Go source file, so that it can be statically embedded into other code. |
|
parser
Package parser reads Jsonnet files and parses them into AST nodes.
|
Package parser reads Jsonnet files and parses them into AST nodes. |
|
Package linter analyses Jsonnet code for code "smells".
|
Package linter analyses Jsonnet code for code "smells". |
|
Package toolutils includes several utilities handy for use in code analysis tools
|
Package toolutils includes several utilities handy for use in code analysis tools |