README
¶
Google HCLS Data Harmonization Mapping Engine
Summary
This is an engine that converts data of one structure to another, based on a configuration file which describes how.
The configuration file can be written in any protobuf format. This can be rather verbose, so we recommend using the Data Harmonization Mapping Language which is transpiled to protobuf configs for you.
The engine accepts data in JSON format and outputs it in JSON format. For information on the mapping configuration, look at the protobuf files in the proto directory.
Building and Testing
TL;DR
Make sure you have installed and added to PATH
- Golang (>= 1.13)
- Protobuf Compiler
protoc(>= 3.11.4)
Then run build.sh
Details
This project consists of 5 go modules:
- mapping_engine
- mapping_engine/main
- mapping_engine/proto
- mapping_engine/transform
- mapping_engine/util
They can be built and tested independently with go build and go test.
The .pb.go files in the proto directory are generated by
protoc-gen-go.
If you modify any .proto files make sure to re-run generate_protos.sh or
build.sh.
If you wish to modify the language and engine simultaneously in a local
filesystem, you may run go_mod_edits.sh to add the appropriate replace
clauses to the go.mod files, redirecting dependencies from github to your
local filesystem. Be aware that the script assumes the engine is in a directory
called mapping_engine under the same parent as mapping_language (the directory
this README is in).
License
Apache License, Version 2.0
Directories
¶
| Path | Synopsis |
|---|---|
|
Package auth provides a client to call the service backends based on the type of server specified in the config.
|
Package auth provides a client to call the service backends based on the type of server specified in the config. |
|
Package builtins contains function definitions and implementation for built-in mapping functions.
|
Package builtins contains function definitions and implementation for built-in mapping functions. |
|
Package cloudfunction contains methods for creating and calling projectors containing Google cloud functions.
|
Package cloudfunction contains methods for creating and calling projectors containing Google cloud functions. |
|
Package errors contains utilties for wrapping and pretty printing mapping errors.
|
Package errors contains utilties for wrapping and pretty printing mapping errors. |
|
Package fetch handles operations related to prefetching resources from an existing store.
|
Package fetch handles operations related to prefetching resources from an existing store. |
|
harmonization
|
|
|
harmonizecode
Package harmonizecode handles harmonization of codes.
|
Package harmonizecode handles harmonization of codes. |
|
harmonizeunit
Package harmonizeunit harmonizes units.
|
Package harmonizeunit harmonizes units. |
|
Package mapping contains methods and mechanisms for executing a mapping config.
|
Package mapping contains methods and mechanisms for executing a mapping config. |
|
Package postprocess handles post processing during the mapping process.
|
Package postprocess handles post processing during the mapping process. |
|
Package projector contains methods and mechanisms for creating and calling projectors.
|
Package projector contains methods and mechanisms for creating and calling projectors. |
|
testing
|
|
|
util
Package util contains helper functions for mapping unit tests.
|
Package util contains helper functions for mapping unit tests. |
|
transform
module
|
|
|
Package types contains shared type definitions.
|
Package types contains shared type definitions. |
|
register_all
Package registerall contains registerAll function that registers builtin function into projectors
|
Package registerall contains registerAll function that registers builtin function into projectors |