README
¶
go-simple-serializer
Description
go-simple-serializer (aka GSS) is a simple library to easily convert data between formats that aims to decrease the burden on developers to support multiple serialization formats in their applications. GSS supports a variety of operating systems, architectures, and use cases. A CLI is released for Microsoft Windows, Linux distributions, and Darwin platforms.
Using cross compilers, this library can also be called by other languages, including C, C++, Python, and JavaScript. This library is cross compiled into a Shared Object file (*.so), which can be called by C, C++, and Python on Linux machines. This library is also compiled to pure JavaScript using GopherJS, which can be called by Node.js and loaded in the browser. See the examples folder for patterns that you can use.
Formats
GSS supports many common formats, including CSV, JSON, and YAML. Pull requests to support other formats are welcome! See the Formats.md document for a full list of supported formats.
Packages
The main public api for GSS is in the gss package. However, this library does ship with internal packages under /pkg/... that can be imported and used directly.
Usage
CLI
The command line tool, gss, can be used to easily convert data between formats. We currently support the following platforms.
| GOOS | GOARCH |
|---|---|
| darwin | amd64 |
| linux | amd64 |
| windows | amd64 |
| linux | arm64 |
Pull requests to support other platforms are welcome! See the CLI.md document for detailed usage and examples.
Go
You can install the go-simple-serializer packages with.
go get -u -d github.com/spatialcurrent/go-simple-serializer/...
You can then import the main public API with import "github.com/spatialcurrent/go-simple-serializer/pkg/gss" or one of the underlying packages, e.g., import "github.com/spatialcurrent/go-simple-serializer/pkg/tags".
See go-simple-serializer in GoDoc for API documentation and examples.
Node
GSS is built as a module. In modern JavaScript, the module can be imported using destructuring assignment.
const { serialize, deserialize, convert, formats } = require('./dist/gss.mod.min.js');
In legacy JavaScript, you can use the gss.global.js file that simply adds gss to the global scope.
Android
The go-simple-serializer code is available for use in Android applications under com.spatialcurrent.gss. For example,
import com.spatialcurrent.gss.Gss;
...
String output_format = Gss.convert(input_string, input_format, input_header, input_comment, output_format, verbose);
...
C
A variant of the Convert function is exported in a Shared Object file (*.so), which can be called by C, C++, and Python programs on Linux machines. For complete patterns for C, C++, and Python, see the examples folder in this repo.
Releases
go-simple-serializer is currently in alpha. See releases at https://github.com/spatialcurrent/go-simple-serializer/releases. See the Building section below to build from scratch.
JavaScript
gss.global.js,gss.global.js.map- JavaScript global build with source mapgss.global.min.js,gss.global.min.js.map- Minified JavaScript global build with source mapgss.mod.js,gss.mod.js.map- JavaScript module build with source mapgss.mod.min.js,gss.mod.min.js.map- Minified JavaScript module with source map
Darwin
gss_darwin_amd64- CLI for Darwin on amd64 (includesmacOSandiOSplatforms)
Linux
gss_linux_amd64- CLI for Linux on amd64gss_linux_amd64- CLI for Linux on arm64gss_linux_amd64.h,gss_linuxamd64.so- Shared Object for Linux on amd64gss_linux_armv7.h,gss_linux_armv7.so- Shared Object for Linux on ARMv7gss_linux_armv8.h,gss_linux_armv8.so- Shared Object for Linux on ARMv8
Windows
gss_windows_amd64.exe- CLI for Windows on amd64
Examples
CLI
.gitignore file to jsonl
cat .gitignore | gss -i csv --input-header path -o json
Get language from CircleCI config.
cat .circleci/config.yml | gss -i yaml -o json -c '#' | jq -r .version
Convert list of files to JSON Lines
find . -name '*.go' | gss -i csv --input-header path -o jsonl
Go
See the examples in GoDoc.
C
See the examples/c/main.c file. You can run the example with make run_example_c.
C++
See the examples/cpp/main.cpp file. You can run the example with make run_example_cpp.
Python
See the examples/python/test.py file. You can run the example with make run_example_python.
JavaScript
See the examples/js/index.js file. You can run the example with make run_example_javascript.
Building
Use make help to see help information for each target.
CLI
The make build_cli script is used to build executables for Linux and Windows.
JavaScript
You can compile GSS to pure JavaScript with the make build_javascript script.
Android
The make build_android script is used to build an Android Archive (AAR) file and associated Javadocs.
Shared Object
The make build_so script is used to build a Shared Object (*.go), which can be called by C, C++, and Python on Linux machines.
Changing Destination
The default destination for build artifacts is go-simple-serializer/bin, but you can change the destination with an environment variable. For building on a Chromebook consider saving the artifacts in /usr/local/go/bin, e.g., DEST=/usr/local/go/bin make build_cli
Testing
CLI
To run CLI testes use make test_cli, which uses shUnit2. If you recive a shunit2:FATAL Please declare TMPDIR with path on partition with exec permission. error, you can modify the TMPDIR environment variable in line or with export TMPDIR=<YOUR TEMP DIRECTORY HERE>. For example:
TMPDIR="/usr/local/tmp" make test_cli
Go
To run Go tests use make test_go (or bash scripts/test.sh), which runs unit tests, go vet, go vet with shadow, errcheck, ineffassign, staticcheck, and misspell.
JavaScript
To run JavaScript tests, first install Jest using make deps_javascript, use Yarn, or another method. Then, build the JavaScript module with make build_javascript. To run tests, use make test_javascript. You can also use the scripts in the package.json.
Contributing
Spatial Current, Inc. is currently accepting pull requests for this repository. We'd love to have your contributions! Please see Contributing.md for how to get started.
License
This work is distributed under the MIT License. See LICENSE file.
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
gss
gss is the command line program for go-simple-serializer (GSS).
|
gss is the command line program for go-simple-serializer (GSS). |
|
gss.global.js
gss.global.js is the package for go-simple-serializer (GSS) that adds GSS functions to the global scope under the "gss" variable.
|
gss.global.js is the package for go-simple-serializer (GSS) that adds GSS functions to the global scope under the "gss" variable. |
|
gss.mod.js
gss.mod.js is the package for go-simple-serializer (GSS) that is built as a JavaScript module.
|
gss.mod.js is the package for go-simple-serializer (GSS) that is built as a JavaScript module. |
|
pkg
|
|
|
bson
Package bson provides an API for BSON serialization.
|
Package bson provides an API for BSON serialization. |
|
cli
Package cli manages loading and testing configuration for serializing and deserializing objects from the command line.
|
Package cli manages loading and testing configuration for serializing and deserializing objects from the command line. |
|
cli/formats
Package formats provides the command for printing a list of supported formats.
|
Package formats provides the command for printing a list of supported formats. |
|
cli/input
Package output contains the code for processing the user provided configuration for the input.
|
Package output contains the code for processing the user provided configuration for the input. |
|
cli/output
Package output contains the code for processing the user provided configuration for the output.
|
Package output contains the code for processing the user provided configuration for the output. |
|
cli/version
Package version contains the command for printing version information.
|
Package version contains the command for printing version information. |
|
escaper
Package escaper provides an easy API to escapae and unescape a string using a custom escape prefix.
|
Package escaper provides an easy API to escapae and unescape a string using a custom escape prefix. |
|
fit
Package fit includes functions for fitting loose types to their underlying values.
|
Package fit includes functions for fitting loose types to their underlying values. |
|
fmt
Package contains a writer for writing objects as formatted lines by wrapping the "fmt" package in the standard library.
|
Package contains a writer for writing objects as formatted lines by wrapping the "fmt" package in the standard library. |
|
gob
Package gob provides support for reading and writing a stream of gob-encoded obejcts.
|
Package gob provides support for reading and writing a stream of gob-encoded obejcts. |
|
gss
Package gss provides simple functions for serializing/deserializing objects into common formats.
|
Package gss provides simple functions for serializing/deserializing objects into common formats. |
|
gssjs
Package gssjs includes functions for the JavaScript build of GSS.
|
Package gssjs includes functions for the JavaScript build of GSS. |
|
inspector
Package inspector is a package of utility functions for inspecting field names and keys from objects.
|
Package inspector is a package of utility functions for inspecting field names and keys from objects. |
|
iterator
Package iterator provides an easy API to create an iterator to read objects from a file.
|
Package iterator provides an easy API to create an iterator to read objects from a file. |
|
json
Package json provides an API for JSON serialization that automatically infers types.
|
Package json provides an API for JSON serialization that automatically infers types. |
|
jsonl
Package jsonl provides a simple API for reading and writing to JSON Lines (aka jsonl).
|
Package jsonl provides a simple API for reading and writing to JSON Lines (aka jsonl). |
|
properties
Package properties includes functions for reading and writing from properties files.
|
Package properties includes functions for reading and writing from properties files. |
|
scanner
Package scanner provides an interface{} for scanning through a stream of bytes.
|
Package scanner provides an interface{} for scanning through a stream of bytes. |
|
serializer
Package serializer provides a middle layer between the gss package and the lower-level packages.
|
Package serializer provides a middle layer between the gss package and the lower-level packages. |
|
splitter
Package splitter provides a wrapper for creating a custom bufio.SplitFunc.
|
Package splitter provides a wrapper for creating a custom bufio.SplitFunc. |
|
sv
Package sv includes iterators for reading from separated-values sources and writing using separated-values formats, such as CSV and TSV.
|
Package sv includes iterators for reading from separated-values sources and writing using separated-values formats, such as CSV and TSV. |
|
tags
Package tags provides a simple API for reading and writing to lines of tags.
|
Package tags provides a simple API for reading and writing to lines of tags. |
|
toml
Package toml provides an API for TOML serialization.
|
Package toml provides an API for TOML serialization. |
|
writer
Package iterator provides an easy API to create an iterator to read objects from a file.
|
Package iterator provides an easy API to create an iterator to read objects from a file. |
|
yaml
Package yaml provides an API for YAML serialization that can automatically infers types.
|
Package yaml provides an API for YAML serialization that can automatically infers types. |
|
plugins
|
|
|
gss
gss.so creates a shared library of Go that can be called by C, C++, or Python - https://godoc.org/cmd/cgo - https://blog.golang.org/c-go-cgo
|
gss.so creates a shared library of Go that can be called by C, C++, or Python - https://godoc.org/cmd/cgo - https://blog.golang.org/c-go-cgo |