README
¶
HIL
HIL (HashiCorp Interpolation Language) is a lightweight embedded language used primarily for configuration interpolation. The goal of HIL is to make a simple language for interpolations in the various configurations of HashiCorp tools.
HIL is built to interpolate any string, but is in use by HashiCorp primarily with HCL. HCL is not required in any way for use with HIL.
HIL isn't meant to be a general purpose language. It was built for basic configuration interpolations. Therefore, you can't currently write functions, have conditionals, set intermediary variables, etc. within HIL itself. It is possible some of these may be added later but the right use case must exist.
Why?
Many of our tools have support for something similar to templates, but within the configuration itself. The most prominent requirement was in Terraform where we wanted the configuration to be able to reference values from elsewhere in the configuration. Example:
foo = "hi ${var.world}"
We originally used a full templating language for this, but found it was too heavy weight. Additionally, many full languages required bindings to C (and thus the usage of cgo) which we try to avoid to make cross-compilation easier. We then moved to very basic regular expression based string replacement, but found the need for basic arithmetic and function calls resulting in overly complex regular expressions.
Ultimately, we wrote our own mini-language within Terraform itself. As we built other projects such as Nomad and Otto, the need for basic interpolations arose again.
Thus HIL was born. It is extracted from Terraform, cleaned up, and better tested for general purpose use.
Syntax
For a complete grammar, please see the parser itself. A high-level overview of the syntax and grammar is listed here.
Code begins within ${ and }. Outside of this, text is treated
literally. For example, foo is a valid HIL program that is just the
string "foo", but foo ${bar} is an HIL program that is the string "foo "
concatened with the value of bar. For the remainder of the syntax
docs, we'll assume you're within ${}.
-
Identifiers are any text in the format of
[a-zA-Z0-9-.]. Example identifiers:foo,var.foo,foo-bar. -
Strings are double quoted and can contain any UTF-8 characters. Example:
"Hello, World" -
Numbers are assumed to be base 10. If you prefix a number with 0x, it is treated as a hexadecimal. If it is prefixed with 0, it is treated as an octal. Numbers can be in scientific notation: "1e10".
-
Unary
-can be used for negative numbers. Example:-10or-0.2 -
Boolean values:
true,false -
The following arithmetic operations are allowed: +, -, *, /, %.
-
Function calls are in the form of
name(arg1, arg2, ...). Example:add(1, 5). Arguments can be any valid HIL expression, example:add(1, var.foo)or even nested function calls:add(1, get("some value")). -
Within strings, further interpolations can be opened with
${}. Example:"Hello ${nested}". A full example including the original${}(remember this list assumes were inside of one already) could be:foo ${func("hello ${var.foo}")}.
Language Changes
We've used this mini-language in Terraform for years. For backwards compatibility reasons, we're unlikely to make an incompatible change to the language but we're not currently making that promise, either.
The internal API of this project may very well change as we evolve it to work with more of our projects. We recommend using some sort of dependency management solution with this package.
Future Changes
The following changes are already planned to be made at some point:
-
Richer types: lists, maps, etc.
-
Convert to a more standard Go parser structure similar to HCL. This will improve our error messaging as well as allow us to have automatic formatting.
-
Allow interpolations to result in more types than just a string. While within the interpolation basic types are honored, the result is always a string.
Documentation
¶
Overview ¶
Example (Basic) ¶
package main
import (
"fmt"
"log"
"github.com/hashicorp/hil"
)
func main() {
input := "${6 + 2}"
tree, err := hil.Parse(input)
if err != nil {
log.Fatal(err)
}
result, err := hil.Eval(tree, &hil.EvalConfig{})
if err != nil {
log.Fatal(err)
}
fmt.Printf("Type: %s\n", result.Type)
fmt.Printf("Value: %s\n", result.Value)
}
Output: Type: TypeString Value: 8
Example (Functions) ¶
package main
import (
"fmt"
"log"
"strings"
"github.com/hashicorp/hil"
"github.com/hashicorp/hil/ast"
)
func main() {
input := "${lower(var.test)} - ${6 + 2}"
tree, err := hil.Parse(input)
if err != nil {
log.Fatal(err)
}
lowerCase := ast.Function{
ArgTypes: []ast.Type{ast.TypeString},
ReturnType: ast.TypeString,
Variadic: false,
Callback: func(inputs []interface{}) (interface{}, error) {
input := inputs[0].(string)
return strings.ToLower(input), nil
},
}
config := &hil.EvalConfig{
GlobalScope: &ast.BasicScope{
VarMap: map[string]ast.Variable{
"var.test": ast.Variable{
Type: ast.TypeString,
Value: "TEST STRING",
},
},
FuncMap: map[string]ast.Function{
"lower": lowerCase,
},
},
}
result, err := hil.Eval(tree, config)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Type: %s\n", result.Type)
fmt.Printf("Value: %s\n", result.Value)
}
Output: Type: TypeString Value: test string - 8
Example (Variables) ¶
package main
import (
"fmt"
"log"
"github.com/hashicorp/hil"
"github.com/hashicorp/hil/ast"
)
func main() {
input := "${var.test} - ${6 + 2}"
tree, err := hil.Parse(input)
if err != nil {
log.Fatal(err)
}
config := &hil.EvalConfig{
GlobalScope: &ast.BasicScope{
VarMap: map[string]ast.Variable{
"var.test": ast.Variable{
Type: ast.TypeString,
Value: "TEST STRING",
},
},
},
}
result, err := hil.Eval(tree, config)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Type: %s\n", result.Type)
fmt.Printf("Value: %s\n", result.Value)
}
Output: Type: TypeString Value: TEST STRING - 8
Index ¶
- Constants
- Variables
- func FixedValueTransform(root ast.Node, Value *ast.LiteralNode) ast.Node
- func InterfaceToVariable(input interface{}) (ast.Variable, error)
- func Parse(v string) (ast.Node, error)
- func ParseWithPosition(v string, pos ast.Pos) (ast.Node, error)
- func VariableToInterface(input ast.Variable) (interface{}, error)
- func Walk(v interface{}, cb WalkFn) error
- type EvalConfig
- type EvalNode
- type EvalType
- type EvaluationResult
- type IdentifierCheck
- type SemanticChecker
- type TypeCheck
- type TypeCheckNode
- type WalkData
- type WalkFn
Examples ¶
Constants ¶
const UnknownValue = "74D93920-ED26-11E3-AC10-0800200C9A66"
UnknownValue is a sentinel value that can be used to denote that a value of a variable (or map element, list element, etc.) is unknown. This will always have the type ast.TypeUnknown.
Variables ¶
var InvalidResult = EvaluationResult{Type: TypeInvalid, Value: nil}
InvalidResult is a structure representing the result of a HIL interpolation which has invalid syntax, missing variables, or some other type of error. The error is described out of band in the accompanying error return value.
Functions ¶
func FixedValueTransform ¶
FixedValueTransform transforms an AST to return a fixed value for all interpolations. i.e. you can make "hi ${anything}" always turn into "hi foo".
The primary use case for this is for config validations where you can verify that interpolations result in a certain type of string.
func InterfaceToVariable ¶
func Parse ¶
Parse parses the given program and returns an executable AST tree.
Syntax errors are returned with error having the dynamic type *parser.ParseError, which gives the caller access to the source position where the error was found, which allows (for example) combining it with a known source filename to add context to the error message.
func ParseWithPosition ¶
ParseWithPosition is like Parse except that it overrides the source row and column position of the first character in the string, which should be 1-based.
This can be used when HIL is embedded in another language and the outer parser knows the row and column where the HIL expression started within the overall source file.
func VariableToInterface ¶
func Walk ¶
Walk will walk an arbitrary Go structure and parse any string as an HIL program and call the callback cb to determine what to replace it with.
This function is very useful for arbitrary HIL program interpolation across a complex configuration structure. Due to the heavy use of reflection in this function, it is recommend to write many unit tests with your typical configuration structures to hilp mitigate the risk of panics.
Types ¶
type EvalConfig ¶
type EvalConfig struct {
// GlobalScope is the global scope of execution for evaluation.
GlobalScope *ast.BasicScope
// SemanticChecks is a list of additional semantic checks that will be run
// on the tree prior to evaluating it. The type checker, identifier checker,
// etc. will be run before these automatically.
SemanticChecks []SemanticChecker
}
EvalConfig is the configuration for evaluating.
type EvalNode ¶
EvalNode is the interface that must be implemented by any ast.Node to support evaluation. This will be called in visitor pattern order. The result of each call to Eval is automatically pushed onto the stack as a LiteralNode. Pop elements off the stack to get child values.
type EvalType ¶
type EvalType uint32
EvalType represents the type of the output returned from a HIL evaluation.
type EvaluationResult ¶
type EvaluationResult struct {
Type EvalType
Value interface{}
}
EvaluationResult is a struct returned from the hil.Eval function, representing the result of an interpolation. Results are returned in their "natural" Go structure rather than in terms of the HIL AST. For the types currently implemented, this means that the Value field can be interpreted as the following Go types:
TypeInvalid: undefined
TypeString: string
TypeList: []interface{}
TypeMap: map[string]interface{}
TypBool: bool
func Eval ¶
func Eval(root ast.Node, config *EvalConfig) (EvaluationResult, error)
type IdentifierCheck ¶
IdentifierCheck is a SemanticCheck that checks that all identifiers resolve properly and that the right number of arguments are passed to functions.
type SemanticChecker ¶
SemanticChecker is the type that must be implemented to do a semantic check on an AST tree. This will be called with the root node.
type TypeCheck ¶
type TypeCheck struct {
Scope ast.Scope
// Implicit is a map of implicit type conversions that we can do,
// and that shouldn't error. The key of the first map is the from type,
// the key of the second map is the to type, and the final string
// value is the function to call (which must be registered in the Scope).
Implicit map[ast.Type]map[ast.Type]string
// Stack of types. This shouldn't be used directly except by implementations
// of TypeCheckNode.
Stack []ast.Type
// contains filtered or unexported fields
}
TypeCheck implements ast.Visitor for type checking an AST tree. It requires some configuration to look up the type of nodes.
It also optionally will not type error and will insert an implicit type conversions for specific types if specified by the Implicit field. Note that this is kind of organizationally weird to put into this structure but we'd rather do that than duplicate the type checking logic multiple times.
func (*TypeCheck) ImplicitConversion ¶
type TypeCheckNode ¶
TypeCheckNode is the interface that must be implemented by any ast.Node that wants to support type-checking. If the type checker encounters a node that doesn't implement this, it will error.
type WalkData ¶
type WalkData struct {
// Root is the parsed root of this HIL program
Root ast.Node
// Location is the location within the structure where this
// value was found. This can be used to modify behavior within
// slices and so on.
Location reflectwalk.Location
// The below two values must be set by the callback to have any effect.
//
// Replace, if true, will replace the value in the structure with
// ReplaceValue. It is up to the caller to make sure this is a string.
Replace bool
ReplaceValue string
}
WalkData is the structure passed to the callback of the Walk function.
This structure contains data passed in as well as fields that are expected to be written by the caller as a result. Please see the documentation for each field for more information.
Source Files
Directories