Documentation
¶
Overview ¶
Package io is used for all of the input/output logic for rq.
Index ¶
- func ListInputHandlers() []string
- func ListOutputHandlers() []string
- func LoadInput(d *DataSpec) (interface{}, error)
- func LoadInputFromReader(d *DataSpec, r io.Reader) (interface{}, error)
- func WriteOutput(data interface{}, d *DataSpec) error
- func WriteOutputToWriter(data interface{}, d *DataSpec, w io.Writer) error
- type Base64InputHandler
- type CSVInputHandler
- type CSVOutputHandler
- type DataSpec
- type DhallInputHandler
- type DotEnvInputHandler
- type HCLInputHandler
- type HJSONInputHandler
- type INIInputHandler
- type InputHandler
- type JSONInputHandler
- type JSONOutputHandler
- type MarkdownTableOutputHandler
- type NDJSONInputHandler
- type NDJSONOutputHandler
- type NullOutputHandler
- type OutputHandler
- type RawInputHandler
- type RawOutputHandler
- type ShOutputHandler
- type TOMLInputHandler
- type TOMLOutputHandler
- type TemplateOutputHandler
- type XMLInputHandler
- type XMLOutputHandler
- type YAMLInputHandler
- type YAMLOutputHandler
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ListInputHandlers ¶
func ListInputHandlers() []string
ListInputHandlers lists all known handler names.
func ListOutputHandlers ¶
func ListOutputHandlers() []string
ListOutputHandlers lists all known handler names.
func LoadInput ¶
LoadInput loads the input file specified by the DataSpec to an interface.
If the DataSpec does not specify a format (format is the empty string), the format defaults to the file extension if it is a known format, and else "json".
If the FilePath is the empty string, then an empty buffer is used as the input file to Parse(). When this happens, the option `generic.empty_file_path` will be set to true.
func LoadInputFromReader ¶
LoadInputFromReader behaves like LoadInput, except that it ignores the FilePath in the DataSpec, and instead reads input from the provided reader.
Like with LoadInput, if the FilePath is omitted, then generic.empty_file_path is set to true.
func WriteOutput ¶
WriteOutput uses the given DataSpec to select an appropriate output handler and write the given data out using it to the file it specifies.
If the DataSpec's FilePath is empty, then the options generic.empty_file_path is set to true.
func WriteOutputToWriter ¶
WriteOutputToWriter works similarly to WriteOutput, but ignores the DataSpec's FilePath, instead writing all data to the given writer.
If the DataSpec's FilePath is empty, then the options generic.empty_file_path is set to true.
Types ¶
type Base64InputHandler ¶
type Base64InputHandler struct {
// contains filtered or unexported fields
}
Base64InputHandler reads the "file data" from the 'base64.data' option, entirely ignoring the FilePath. This can be useful for testing.
Note that because base64 data can contain '=' symbols for zero padding, it is recommended to use the JSON syntax for specifying the DataSpec.
The base64 handler supports these input options:
- base64.data: the data to be returned when Parse() is called, in JSON format.
func (*Base64InputHandler) Name ¶
func (b *Base64InputHandler) Name() string
Name implements InputHandler.Name().
func (*Base64InputHandler) Parse ¶
func (b *Base64InputHandler) Parse(reader io.Reader) (interface{}, error)
Parse implements InputHandler.Parse().
Unlike other input handlers, the base64 handler does not read from the given reader, and instead parses the data contained in the base64.data option as JSON and returns that.
type CSVInputHandler ¶
type CSVInputHandler struct {
// contains filtered or unexported fields
}
CSVInputHandler implements a CSV input InputHandler.
The following options are supported:
csv.comma (rune) the comma-character to use (default: ,)
csv.comment (rune) the comment leader character to use (default: none)
csv.skip_lines (int) number of leading lines to skip, ignored if negative (default: 0)
csv.headers (bool) if true, the first row is treated as headers and used to generate object keys for the remaining rows (default: false)
csv.infer (bool) if true, attempt to convert boolean, integer, or floating point values in the CSV to those types before serialization, otherwise leave all values as strings (default: true)
func (*CSVInputHandler) Name ¶
func (c *CSVInputHandler) Name() string
Name implements InputHandler.Name().
type CSVOutputHandler ¶
type CSVOutputHandler struct {
// contains filtered or unexported fields
}
CSVOutputHandler handles serializing CSVs.
The following options are supported:
csv:comma (rune) the comma-character to use (default: ,)
csv:headers (bool) if true, a header row will be generated in situations where it is possible to do so (default: true)
func (*CSVOutputHandler) Format ¶
func (c *CSVOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*CSVOutputHandler) Name ¶
func (c *CSVOutputHandler) Name() string
Name implements OutputHandler.Name().
type DataSpec ¶
DataSpec describes a single DataSpec after it has been parsed.
func ParseDataSpec ¶
ParseDataSpec parses a data specification string into a DataSpec object.
The DataSpec format is used by rq for --data arguments, as well as for certain builtins that allow loading data. It intends to solve the following problems:
- Allow specifying what format some data should loaded or stored in(e.g. CSV, JSON, YAML, etc.)
- Allow specifying options pertinent to that format, such as the delimiter character for CSV.
- Allow specifying where data should be loaded into Rego.
- Be concise for users to specify when writing one-liners.
The FilePath portion of the spec may or may not be an actual local file path, depending on the format used, or may be omitted entirely (e.g. when using a DataSpec to specify options for how something should be parsed). The RegoPath also may not be meaningful in all contexts, for example when specifying where data should be written to
The pseudo-BNF for the DataSpec format follows:
DATASPEC -> FILEPATH |
FORMAT ":" FILEPATH |
OPTIONS ":" FILEPATH |
FORMAT ":" OPTIONS ":" FILEPATH
OPTIONS -> KEY "=" VALUE |
KEY "=" VALUE ";" OPTIONS
KEY -> SEGMENT
VALUE -> SEGMENT
FORMAT -> SEGMENT
FILEPATH -> SEGMENT
SEGMENT -> /[^;:=]+/
Note that this BNF does not ascribe any special value to any of the terminals like FORMAT or REGOPATH. A valid DataSpec may end up invalid when the data is actually used - validation is not the job of this parser.
Leading and trailing whitespace on terminals is not considered significant and will be trimmed.
Examples of valid DataSpecs:
- /foo/bar - json:/foo/bar - csv:csv.headers=true;csv.separator=,;rego.path=foo:/foo/bar - csv.headers=true;csv.separator=,;rego.path=foo:/foo/bar - {"format": "csv", "file_path": "/foo/bar", "options": {"bool": true, "int": 7}}
NOTE: if the first character of the DataPath is '{', and the last character of the DataPath is '}' (ignoring whitespace), then this function instead attempts to parse the DataPath as JSON and Unmarshal it into the DataSpec struct. This allows for arbitrary path and option strings, for situations where disallowed characters, leading or trailing whitespace, etc. are needed.
func (*DataSpec) ResolveDefaults ¶
ResolveDefaults is used to apply default behaviors in a sensible and consistent way.
If the FilePath is omitted, then the FilePath from the default is used instead.
If the Format is omitted (empty string) from the DataSpec, then the default's Format will be used. If it is also the empty string, then the FilePath's extension will be used. If the FilePath is empty or does not contain a '.', then the Format will be set to "json".
Any options which are specified in the default but not in the DataSpec will be applied as well.
func (*DataSpec) UnmarshalJSON ¶
UnmarshalJSON implements json.Unmarshaler
This is needed because we want to map bool/float values back to strings.
type DhallInputHandler ¶ added in v0.0.6
type DhallInputHandler struct{}
DhallInputHandler handles parsing Dhall data.
func (*DhallInputHandler) Name ¶ added in v0.0.6
func (d *DhallInputHandler) Name() string
Name implements InputHandler.Name().
type DotEnvInputHandler ¶ added in v0.0.5
type DotEnvInputHandler struct{}
DotEnvInputHandler handles parsing DotEnv data.
func (*DotEnvInputHandler) Name ¶ added in v0.0.5
func (j *DotEnvInputHandler) Name() string
Name implements InputHandler.Name().
type HCLInputHandler ¶ added in v0.0.5
type HCLInputHandler struct{}
HCLInputHandler handles parsing HCL data.
func (*HCLInputHandler) Name ¶ added in v0.0.5
func (h *HCLInputHandler) Name() string
Name implements InputHandler.Name().
type HJSONInputHandler ¶ added in v0.0.6
type HJSONInputHandler struct{}
HJSONInputHandler handles parsing HJSON data.
func (*HJSONInputHandler) Name ¶ added in v0.0.6
func (h *HJSONInputHandler) Name() string
Name implements InputHandler.Name().
type INIInputHandler ¶ added in v0.0.5
type INIInputHandler struct{}
INIInputHandler handles parsing INI data.
func (*INIInputHandler) Name ¶ added in v0.0.5
func (i *INIInputHandler) Name() string
Name implements InputHandler.Name().
type InputHandler ¶
type InputHandler interface {
// Parse consumes input from a reader and unmarshals it into an
// interface. This will later be passed to ast.InterfaceToValue()
// for consumption by Rego.
Parse(reader io.Reader) (interface{}, error)
// SetOption specifies an option to control how the handler should
// behave. Setting options that the handler does not implement should
// cause SetOption to return nil, this way the CLI handling logic does
// not have to determine which handler is being used before deciding
// which options to try to set.
//
// The canonical method for parsing non-string values is to use
// util.StringToValue().
SetOption(name string, value string) error
// Name should return the name of this handler, e.g. "json" or "csv".
Name() string
}
InputHandler represents a handler for one type of input format, such as JSON or CSV.
func SelectInputHandler ¶
func SelectInputHandler(name string) (InputHandler, error)
SelectInputHandler chooses a handler based on the name provided.
If name is the empty string, then it default to "json".
type JSONInputHandler ¶
type JSONInputHandler struct{}
JSONInputHandler handles parsing JSON data.
func (*JSONInputHandler) Name ¶
func (j *JSONInputHandler) Name() string
Name implements InputHandler.Name().
type JSONOutputHandler ¶
type JSONOutputHandler struct {
// contains filtered or unexported fields
}
JSONOutputHandler handles serializing JSON data.
func (*JSONOutputHandler) Format ¶
func (j *JSONOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*JSONOutputHandler) Name ¶
func (j *JSONOutputHandler) Name() string
Name implements OutputHandler.Name().
type MarkdownTableOutputHandler ¶ added in v0.0.5
type MarkdownTableOutputHandler struct {
// contains filtered or unexported fields
}
MarkdownTableOutputHandler handles serializing JSON data.
func (*MarkdownTableOutputHandler) Format ¶ added in v0.0.5
func (t *MarkdownTableOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*MarkdownTableOutputHandler) Name ¶ added in v0.0.5
func (t *MarkdownTableOutputHandler) Name() string
Name implements OutputHandler.Name().
type NDJSONInputHandler ¶ added in v0.0.6
type NDJSONInputHandler struct {
// contains filtered or unexported fields
}
NDJSONInputHandler handles parsing NDJSON data.
The following options are supported:
ndjson.allowempty (bool) if asserted, empty lines are ignored, otherwise they cause errors (default: true)
func (*NDJSONInputHandler) Name ¶ added in v0.0.6
func (j *NDJSONInputHandler) Name() string
Name implements InputHandler.Name().
type NDJSONOutputHandler ¶ added in v0.0.6
type NDJSONOutputHandler struct {
// contains filtered or unexported fields
}
NDJSONOutputHandler handles serializing NDJSON data.
func (*NDJSONOutputHandler) Format ¶ added in v0.0.6
func (j *NDJSONOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*NDJSONOutputHandler) Name ¶ added in v0.0.6
func (j *NDJSONOutputHandler) Name() string
Name implements OutputHandler.Name().
type NullOutputHandler ¶ added in v0.0.7
type NullOutputHandler struct {
}
NullOutputHandler is a "dummy" output handler which simply ignores the data given to it, without writing anything. This can be useful in situations where you are going to drop the output data anyway, as using the null handler means you won't spend any overhead on formatting output you plan to discard.
func (*NullOutputHandler) Format ¶ added in v0.0.7
func (n *NullOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*NullOutputHandler) Name ¶ added in v0.0.7
func (n *NullOutputHandler) Name() string
Name implements OutputHandler.Name().
type OutputHandler ¶
type OutputHandler interface {
// Format marshals the given data to the relevant format and then
// writes it to the specified writer.
Format(writer io.Writer, data interface{}) error
// SetOption specifies an option to control how the handler should
// behave. Setting options that the handler does not implement should
// cause SetOption to return nil, this way the CLI handling logic does
// not have to determine which handler is being used before deciding
// which options to try to set.
//
// The following options should be supported by all output formatters
// to the greatest extent possible:
//
// output.colorize (bool) if true, syntax highlight the output if
// possible, otherwise do not do so (defaults to true)
//
// output.pretty (bool) if true, pretty-print the output if possible,
// otherwise minify the output or otherwise leave in the default format
// (defaults to true)
//
// output.indent (string) for formats such as JSON and YAML where
// indenting is used, specify the indent character (defaults to '\t')
//
// output.style (string) specifies the Chroma format to use when
// colorizing output (default: "native"); see:
// https://xyproto.github.io/splash/docs/
SetOption(name string, value string) error
// Name should return the name of this handler, e.g. "json" or "csv".
Name() string
}
OutputHandler represents a handler for one type of output format, such as JSON or CSV.
func SelectOutputHandler ¶
func SelectOutputHandler(name string) (OutputHandler, error)
SelectOutputHandler chooses a handler based on the name provided.
If name is the empty string, then it default to "json".
type RawInputHandler ¶ added in v0.0.6
type RawInputHandler struct {
// contains filtered or unexported fields
}
RawInputHandler simply converts the input directly to a string. It supports optional AWK-like field and record splitting.
The data shape of this handler depends on how it is configured. If fs and rs are both unset, then the result is a string. If only rs is set, the result is an array, and if both fs and rs are set, the result is an array of arrays. In the latter case the data shape can instead be a list of objects if raw.header is true.
If raw.infer is true, then the type of the innermost scalar values will be inferred automatically, otherwise they will be left as strings. Type inference happens after the cutset(s) are applied.
The following options are supported:
raw.fs (regex) split records into fields with the given regex as the delimiter. If this value is the empty string, then records will remain as strings and not be split into fields. (default: "")
raw.rs (regex) split records into records with the given regex as the delimiter. If this value is the empty string, then the input will remain as a string and not be split into records. (default: "")
raw.lcutset (string) apply strings.TrimLeft with this cutset to each element after splitting is complete, if the cutset is non-empty (default: "").
raw.rcutset (string) apply strings.TrimLeft with this cutset to each element after splitting is complete, if the cutset is non-empty (default: "").
raw.cutset (string) apply strings.Trim with this cutset to each element after splitting is complete, if the cutset is non-empty (default: "").
raw.coalesce (int) if a nonzero value n, only the first n elements are split and any remaining elements are coalesced into column n (default: 0).
raw.infer (bool) if true, attempt to convert boolean, integer, or floating point elements into those Rego types, rather than leaving them as strings (default: false).
raw.headers (bool) if true, and rs and fs are both set, then treat the first record as a header row (default: false).
func (*RawInputHandler) Name ¶ added in v0.0.6
func (r *RawInputHandler) Name() string
Name implements InputHandler.Name().
type RawOutputHandler ¶
type RawOutputHandler struct {
// contains filtered or unexported fields
}
RawOutputHandler handles serializing JSON data.
func (*RawOutputHandler) Format ¶
func (r *RawOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*RawOutputHandler) Name ¶
func (r *RawOutputHandler) Name() string
Name implements OutputHandler.Name().
type ShOutputHandler ¶ added in v0.0.5
type ShOutputHandler struct {
}
ShOutputHandler handles serializing Sh data.
func (*ShOutputHandler) Format ¶ added in v0.0.5
func (s *ShOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*ShOutputHandler) Name ¶ added in v0.0.5
func (s *ShOutputHandler) Name() string
Name implements OutputHandler.Name().
type TOMLInputHandler ¶
type TOMLInputHandler struct{}
TOMLInputHandler handles parsing TOML data.
func (*TOMLInputHandler) Name ¶
func (t *TOMLInputHandler) Name() string
Name implements InputHandler.Name().
type TOMLOutputHandler ¶ added in v0.0.6
type TOMLOutputHandler struct {
// contains filtered or unexported fields
}
TOMLOutputHandler handles serializing TOML data.
func (*TOMLOutputHandler) Format ¶ added in v0.0.6
func (t *TOMLOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*TOMLOutputHandler) Name ¶ added in v0.0.6
func (t *TOMLOutputHandler) Name() string
Name implements OutputHandler.Name().
type TemplateOutputHandler ¶ added in v0.0.6
type TemplateOutputHandler struct {
// contains filtered or unexported fields
}
TemplateOutputHandler handles serializing Template data.
func (*TemplateOutputHandler) Format ¶ added in v0.0.6
func (t *TemplateOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*TemplateOutputHandler) Name ¶ added in v0.0.6
func (t *TemplateOutputHandler) Name() string
Name implements OutputHandler.Name().
type XMLInputHandler ¶ added in v0.0.5
type XMLInputHandler struct{}
XMLInputHandler handles parsing XML data.
func (*XMLInputHandler) Name ¶ added in v0.0.5
func (x *XMLInputHandler) Name() string
Name implements InputHandler.Name().
type XMLOutputHandler ¶ added in v0.0.5
type XMLOutputHandler struct {
// contains filtered or unexported fields
}
XMLOutputHandler handles serializing XML data.
func (*XMLOutputHandler) Format ¶ added in v0.0.5
func (x *XMLOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*XMLOutputHandler) Name ¶ added in v0.0.5
func (x *XMLOutputHandler) Name() string
Name implements OutputHandler.Name().
type YAMLInputHandler ¶
type YAMLInputHandler struct{}
YAMLInputHandler handles parsing YAML data.
func (*YAMLInputHandler) Name ¶
func (y *YAMLInputHandler) Name() string
Name implements InputHandler.Name().
type YAMLOutputHandler ¶
type YAMLOutputHandler struct {
// contains filtered or unexported fields
}
YAMLOutputHandler handles parsing YAML data.
YAMLOutputHandler does not support output:pretty or output:indent, because go-yaml does not support modifying the indent behavior.
func (*YAMLOutputHandler) Format ¶
func (y *YAMLOutputHandler) Format(writer io.Writer, data interface{}) error
Format implements OutputHandler.Format()
func (*YAMLOutputHandler) Name ¶
func (y *YAMLOutputHandler) Name() string
Name implements OutputHandler.Name().
Source Files
¶
- dataspec.go
- input.go
- input_base64.go
- input_csv.go
- input_dhall.go
- input_dotenv.go
- input_hcl.go
- input_hjson.go
- input_ini.go
- input_json.go
- input_ndjson.go
- input_raw.go
- input_toml.go
- input_xml.go
- input_yaml.go
- io.go
- output.go
- output_csv.go
- output_json.go
- output_markdown_table.go
- output_ndjson.go
- output_null.go
- output_raw.go
- output_sh.go
- output_template.go
- output_toml.go
- output_xml.go
- output_yaml.go