jsonnet

go-jsonnet

This an implementation of Jsonnet in pure Go. It is a feature complete, production-ready implementation. It is compatible with the original Jsonnet C++ implementation. Bindings to C and Python are available (but not battle-tested yet).

This code is known to work on Go 1.12 and above. We recommend always using the newest stable release of Go.

Installation instructions

# go >= 1.17
# Using `go get` to install binaries is deprecated.
# The version suffix is mandatory.
go install github.com/google/go-jsonnet/cmd/jsonnet@latest

# Or other tools in the 'cmd' directory
go install github.com/google/go-jsonnet/cmd/jsonnet-lint@latest

# go < 1.17
go get github.com/google/go-jsonnet/cmd/jsonnet

It's also available on Homebrew:

brew install go-jsonnet

jsonnetfmt and jsonnet-lint are also available as pre-commit hooks. Example .pre-commit-config.yaml:

- repo: https://github.com/google/go-jsonnet
  rev: # ref you want to point at, e.g. v0.17.0
  hooks:
    - id: jsonnet-format
    - id: jsonnet-lint

It can also be embedded in your own Go programs as a library:

package main

import (
	"fmt"
	"log"

	"github.com/google/go-jsonnet"
)

func main() {
	vm := jsonnet.MakeVM()

	snippet := `{
		person1: {
		    name: "Alice",
		    welcome: "Hello " + self.name + "!",
		},
		person2: self.person1 { name: "Bob" },
	}`

	jsonStr, err := vm.EvaluateAnonymousSnippet("example1.jsonnet", snippet)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(jsonStr)
	/*
	   {
	     "person1": {
	         "name": "Alice",
	         "welcome": "Hello Alice!"
	     },
	     "person2": {
	         "name": "Bob",
	         "welcome": "Hello Bob!"
	     }
	   }
	*/
}

Build instructions (go 1.12+)

git clone git@github.com:google/go-jsonnet.git
cd go-jsonnet
go build ./cmd/jsonnet
go build ./cmd/jsonnetfmt
go build ./cmd/jsonnet-deps

To build with Bazel instead:

git clone git@github.com:google/go-jsonnet.git
cd go-jsonnet
git submodule init
git submodule update
bazel build //cmd/jsonnet
bazel build //cmd/jsonnetfmt
bazel build //cmd/jsonnet-deps

The resulting jsonnet program will then be available at a platform-specific path, such as bazel-bin/cmd/jsonnet/darwin_amd64_stripped/jsonnet for macOS.

Bazel also accommodates cross-compiling the program. To build the jsonnet program for various popular platforms, run the following commands:

For additional target platform names, see the per-Go release definitions here in the rules_go Bazel package.

Additionally if any files were moved around, see the section Keeping the Bazel files up to date.

Building libjsonnet.wasm

GOOS=js GOARCH=wasm go build -o libjsonnet.wasm ./cmd/wasm 

Or if using bazel:

bazel build //cmd/wasm:libjsonnet.wasm

Running tests

./tests.sh  # Also runs `go test ./...`

Running Benchmarks

Method 1

go get golang.org/x/tools/cmd/benchcmp

1. Make sure you build a jsonnet binary prior to making changes.

go build -o jsonnet-old ./cmd/jsonnet

1. Make changes (iterate as needed), and rebuild new binary

go build ./cmd/jsonnet

1. Run benchmark:

# e.g. ./benchmark.sh Builtin
./benchmark.sh <TestNameFilter>

Method 2

1. get benchcmp

go get golang.org/x/tools/cmd/benchcmp

1. Make sure you build a jsonnet binary prior to making changes.

make build-old

1. iterate with (which will also automatically rebuild the new binary ./jsonnet)

replace the FILTER with the name of the test you are working on

FILTER=Builtin_manifestJsonEx make benchmark

Implementation Notes

We are generating some helper classes on types by using http://clipperhouse.github.io/gen/. Do the following to regenerate these if necessary:

go get github.com/clipperhouse/gen
go get github.com/clipperhouse/set
export PATH=$PATH:$GOPATH/bin  # If you haven't already
go generate

Update cpp-jsonnet sub-repo

This repo depends on the original Jsonnet repo. Shared parts include the standard library, headers files for C API and some tests.

You can update the submodule and regenerate dependent files with one command:

./update_cpp_jsonnet.sh

Note: It needs to be run from repo root.

Updating and modifying the standard library

Standard library source code is kept in cpp-jsonnet submodule, because it is shared with Jsonnet C++ implementation.

For performance reasons we perform preprocessing on the standard library, so for the changes to be visible, regeneration is necessary:

go run cmd/dumpstdlibast/dumpstdlibast.go cpp-jsonnet/stdlib/std.jsonnet > astgen/stdast.go

**The

The above command creates the astgen/stdast.go file which puts the desugared standard library into the right data structures, which lets us avoid the parsing overhead during execution. Note that this step is not necessary to perform manually when building with Bazel; the Bazel target regenerates the astgen/stdast.go (writing it into Bazel's build sandbox directory tree) file when necessary.

Keeping the Bazel files up to date

Note that we maintain the Go-related Bazel targets with the Gazelle tool. The Go module (go.mod in the root directory) remains the primary source of truth. Gazelle analyzes both that file and the rest of the Go files in the repository to create and adjust appropriate Bazel targets for building Go packages and executable programs.

After changing any dependencies within the files covered by this Go module, it is helpful to run go mod tidy to ensure that the module declarations match the state of the Go source code. In order to synchronize the Bazel rules with material changes to the Go module, run the following command to invoke Gazelle's update-repos command:

bazel run //:gazelle -- update-repos -from_file=go.mod -to_macro=bazel/deps.bzl%jsonnet_go_dependencies

Similarly, after adding or removing Go source files, it may be necessary to synchronize the Bazel rules by running the following command:

bazel run //:gazelle