goten-sdk

Goten SDK

Overview

Goten is a development framework for resource-oriented gRPC service. In a sense, it is an additional language built on top of gRPC and Google Protocol Buffers. It provides more higher-level components like services, references, resources. It standardizes service description in proto files and provides dedicated protoc compilers generating basic Golang client/server code.

Goten-based service is basically made of set of logically bound APIs and resources with additional REST transcoding. Each resource, unless explicitly configured by service developer, has CRUD functionality automatically generated, with many extra features like watching (updates streaming), text based searching (optional, for selected resources), CAS (compare and swap). More about conventions adopted in Goten: Google API design, basic CRUD

We can define following components:

• Goten-based Service -> This is the highest level object, which basically describes full service. It contains logically bound set of resources and RPC APIs, forming together coherent and possibly minimalistic service, focusing on specific business job.
• Resource -> This is basically protobuf message with resource annotation and name. It conforms to the standards defined in Google API conventions. If you look at proto files in any Goten service, you should see that each resource has basically two files: <resource_name>.proto and <resource_name>_change.proto. First one describes resource schema itself, second describes "change" of a resource and is used in Watch requests (subscription for updates). Each resource also has its own dedicated RPC API (of the same name) with CRUD functionality.
• RPC API -> Set of RPC methods. Most of them are dedicated for single resources and have CRUD functionality, but it may not always be the case. Each API has <api_name>_service.proto file and, if it contains custom developer-defined methods, <api_name>_custom.proto.

As said before, Goten provides dedicated protoc compilers producing service code in Go language for both client and server side. While auto-generated code forms the biggest part of a service (usually), it also references some common, non-generated parts. This repository contains common, client-side components referenced by that auto generated code. Therefore, Goten SDK is not meant to be used directly by any final application. However, it is used by any public client-side library generated by Goten.

Goten SDK contains also some common protobuf files used by services developed on it.

Components

Goten SDK contains common functions/interfaces used by auto-generated Golang libraries + some common protobuf files.

Access (runtime/access)

Access is a module in Goten-based service containing higher level objects for client side code. As higher level, I mean it does not contain raw basic clients for making API calls, it is for components built on top of that. Currently, Access is used primarily for Watcher component. Watcher utilizes Watch request (like LIST, but with real time updates) and handles boilerplate parts like reconnections, automatic recovery etc.

Api (runtime/api)

This is not exactly Api, because Goten SDK is not that. This component contains just common protobuf files used by Goten based services. You will need them if you want to generate library in your preferable programming language.

Common Goten message interfaces (runtime/goten)

This is primarily home for GotenMessage interface, which is implemented by every generated (by Goten compiler) protobuf message in Golang.

Object interfaces (runtime/object)

While GotenMessage is a minimal interface used by any generated struct (for protobuf message), Goten also comes with additional protoc compilers capable of enriching these structs with much more functionality. For each message, Goten can provide dedicated, type-safe FieldMask and FieldPath objects. This is non-standard functionality that is typically not available using widely-known protoc compilers for Golang. This module contains all definitions that can be provided by those extra compilers.

Resource interfaces (runtime/resource)

This module contains many interfaces associated with Goten resource - Resource itself, name object, reference, filter, descriptor and so on.

Validate interfaces (runtime/validate)

Optional interfaces implemented by some goten-generated messages.

How to use

As it was said, Goten SDK contains only public, client-side components referenced by Goten based services and its not meant for development on its own. However, there are common steps to follow if you are developing application talking to a Goten service, and you want to use its SDK.

If you are happy with developing in Golang, we recommend to simply use our Go libraries generated by dedicated Goten compilers. We include Go library in each SDK. All you need to do, apart from having access to an API endpoint, is to install Go (platform specific - see https://golang.org/doc/install). Then, you need to include SDK for interested service in your list of dependencies (Each of our SDKs is a Go module too).

However, we also provide proto files, which can be used to generate client libraries in practically any programming language. Then, you have a couple of more things to do:

1. You need to install protoc protoc. You need to install it in version at least 3.6.1. You need to verify you have the most basic files like WellKnownTypes. If you have installed in /usr/local (for example), then check /usr/local/include/google/protobuf directory.
2. You need also some extra protobuf files from google. We use certain files from googleapis.
3. When using protoc, you need to also include directory runtime/api (from this repository) as proto include path.
4. You need to install protoc plugin for your language.
5. With all proto files and protoc installed, you should be able to generate library in your language.

Very simple example for Golang:

PROTOINCLUDE=~/workspace:/usr/local/include
GOGENPATH=$GOPATH/src

protoc \
    -I "${PROTOINCLUDE}" \
    "--go_out=:${GOGENPATH}" \
    "${SERVICEPATH}"/proto/v1/*.proto

Assumptions:

• I put all google proto files in /usr/local/include - both well knowns and from googleapis.
• SERVICEPATH points to location of goten based service. It is located in ~/workspace in fact. Therefore, PROTOINCLUDE will work.

Part "go_out" is for golang, however they are other plugins for different languages. As a tip, generated files are put according to a parameters taken from proto files. For example, for Golang, output directory is dictated by parameter provided to go_out plugin combined with go_package option. Open any proto file and you should see something like:

syntax = "proto3";

// ....
// some lines here....
// ....

option go_package = "github.com/cloudwan/goten-sdk/runtime/api/view";

In result, for this little example, output file will be located in $GOPATH/src/github.com/cloudwan/goten-sdk/runtime/api/view. In some cases, you may want to create a symlink to your repository, if you want to commit generated files!

Of course each service may impose additional requirements, but above are the common steps.