README
¶
ltsv
High performance LTSV (Labeled Tab Separeted Value) parser for Go.
About LTSV: http://ltsv.org/
Labeled Tab-separated Values (LTSV) format is a variant of
Tab-separated Values (TSV). Each record in a LTSV file is represented
as a single line. Each field is separated by TAB and has a label and
a value. The label and the value have been separated by ':'. With
the LTSV format, you can parse each line by spliting with TAB (like
original TSV format) easily, and extend any fields with unique labels
in no particular order.
Installation
go get github.com/Wing924/ltsv
Examples
package main
import (
"fmt"
"github.com/Wing924/ltsv"
)
func main() {
line := []byte("foo:123\tbar:456")
record, err := ltsv.ParseLineAsMap(line, nil)
if err != nil {
panic(err)
}
fmt.Printf("%#v", record) // map[string]string{"foo":"123", "bar":"456"}
}
Benchmarks
Benchmark against
- Songmu/go-ltsv: 635% faster
- ymotongpoo/goltsv: 365% faster
- najeira/ltsv: 782% faster
Source code: bench/line_test.go.
Result
$ go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/Wing924/ltsv/bench
Benchmark_line_Wing924_ltsv-4 2000000 626 ns/op 224 B/op 17 allocs/op
Benchmark_line_Wing924_ltsv_strict-4 2000000 788 ns/op 224 B/op 17 allocs/op
Benchmark_line_Songmu_goltsv-4 300000 3975 ns/op 1841 B/op 32 allocs/op
Benchmark_line_ymotongpoo_goltsv-4 500000 2286 ns/op 5793 B/op 17 allocs/op
Benchmark_line_najeira_ltsv-4 300000 4896 ns/op 5529 B/op 26 allocs/op
PASS
ok github.com/Wing924/ltsv/bench 8.245s
Documentation
¶
Index ¶
- Variables
- func ParseField(field []byte) (label []byte, value []byte, err error)
- func ParseLine(line []byte, callback func(label []byte, value []byte) error) error
- func ParseLineAsMap(line []byte, record map[string]string) (map[string]string, error)
- type Field
- type Parser
- func (p Parser) ParseField(field []byte) (label []byte, value []byte, err error)
- func (p Parser) ParseLine(line []byte, callback func(label []byte, value []byte) error) error
- func (p Parser) ParseLineAsMap(line []byte, record map[string]string) (map[string]string, error)
- func (p Parser) ParseLineAsSlice(line []byte, record []Field) ([]Field, error)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrMissingLabel is an error to describe label is missing (ex. 'my_value') ErrMissingLabel = xerrors.New("missing label") // ErrEmptyLabel is an error to describe label is empty (ex. ':my_value') ErrEmptyLabel = xerrors.New("empty label") // ErrInvalidLabel is an error to describe label contains invalid char (ex. 'my\tlabel:my_value') ErrInvalidLabel = xerrors.New("invalid label") // ErrInvalidValue is an error to describe value contains invalid char (ex. 'my_label:my_value\n') ErrInvalidValue = xerrors.New("invalid value") // Break is an error for break loop Break = xerrors.New("break") )
var DefaultParser = Parser{ FieldDelimiter: '\t', ValueDelimiter: ':', StrictMode: true, }
DefaultParser is the default parser
Functions ¶
func ParseField ¶ added in v0.2.0
ParseField parse LTSV-encoded field and return the label and value. The result share same memory with inputted field
func ParseLine ¶
ParseLine parse one line of LTSV-encoded data and call callback. The callback function will be called for each field.
func ParseLineAsMap ¶ added in v0.2.0
ParseLineAsMap parse one line of LTSV-encoded data and return the map[string]string. For reducing memory allocation, you can pass a map to record to reuse the given map.
Example ¶
line := []byte("foo:123\tbar:456")
record, err := ParseLineAsMap(line, nil)
if err != nil {
panic(err)
}
fmt.Printf("%#v", record) // map[string]string{"foo":"123", "bar":"456"}
Output:
Types ¶
type Field ¶ added in v0.2.0
Field is a struct to hold label-value pair.
func ParseLineAsSlice ¶ added in v0.2.0
ParseLineAsSlice parse one line of LTSV-encoded data and return the []Field. For reducing memory allocation, you can pass a slice to record to reuse the given slice.
Example ¶
line := []byte("foo:123\tbar:456")
record, err := ParseLineAsSlice(line, nil)
if err != nil {
panic(err)
}
fmt.Printf("%+v", record) // [{Label:foo Value:123} {Label:bar Value:456}]
Output:
type Parser ¶ added in v0.3.0
type Parser struct {
// FieldDelimiter is the delimiter of fields. It defaults to '\t'.
FieldDelimiter byte
// ValueDelimiter is the delimiter of label-value pairs. It defaults to ':'.
ValueDelimiter byte
// StrictMode is a flag to check if labels and values are valid.
// If strictMode is false,
// the parser just split fields with `FieldDelimiter`
// and split label and value with `ValueDelimiter` without checking if they are valid.
// The valid label is `/[0-9A-Za-z_.-]+/`.
// The valid value is `/[^\b\t\r\n]*/`.
StrictMode bool
}
Parser is for parsing LTSV-encoded format.
func (Parser) ParseField ¶ added in v0.3.0
ParseField parse LTSV-encoded field and return the label and value. The result share same memory with inputted field.
func (Parser) ParseLine ¶ added in v0.3.0
ParseLine parse one line of LTSV-encoded data and call callback. The callback function will be called for each field.
func (Parser) ParseLineAsMap ¶ added in v0.3.0
ParseLineAsMap parse one line of LTSV-encoded data and return the map[string]string. For reducing memory allocation, you can pass a map to record to reuse the given map.
Source Files