goworld

package module
v0.0.0-...-48890ff Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 19, 2018 License: MIT Imports: 17 Imported by: 0

README

Table of Contents

goworld

Aplication which allows to access Smallworld data as JSON via HTTP

[Visit project page] (http://kpawlik.github.io/goworld)

About

This is an application to get data from Smallworld via HTTP in JSON format. Goworld is composed of set of concurrent workers and one HTTP server. Worker communicates with Smallworld session via ACP protocol. HTTP server and workers communicates via RPC protocol.

Features
  • Zero installation
  • One executable file, one Magik file
  • Simple, light, fast and scalable
  • Linux/Windows support
  • One simple config file
  • Scalable - multiple ACP workers -> one concurrency HTTP server
  • You can run multiple workers on single Smallworld session
  • You can run multiple workers on multiple Smallworld sessions
  • HTTP Server can be run on Windows or Linux

Configuration

Configuration file

Config file is a simple JSON file.

{
    "server": {
        "port": Number,
		"protocols": 
			[
			...
			]
    },
    "workers": [
	    {
        	"port": Number,
	        "host": String,
	        "name": String
    	},
	...

	]
}

server - HTTP server configuration

server.port - HTTP server port number

server.protocols - list of protocols definitions. See Protocols

workers - list of workers definitions

worker.port - port number to communicate with HTTP server

worker.host - name of the host where worker is started

worker.name - unique name of the worker. This name will be also used as a Magik ACP process name

List protocol

This is predefined protocol described in quick start. To disable this protocol set false for attribute enabled, or just remove JSON object from configuration file.

```
protocols:
[
	{
	  		"name": "list",
	  		"enabled": bool
		},
...
]
```
### Custom protocol To define custom protocol you need to add protocol definition in configuration file. Configuration:
```
 {
  "name": string,
  "enabled": bool,
  "params": [
    {
      "name": string,
      "type": string
    }
	...
  ],
  "results": [
    {
      "name": string,
      "type": string
    }
	...
  ]
},
```

Definiton of custom protocol contains:

  • name - uniqe protocol name. This will be first part of the HTTP Request
  • enabled - bool attribute, which allows to disable/enable protocol
  • params - list of Parameter objects (name and type). Parameters values must be pass in request URL after protocol name and should be separated by '/' char. Parameters will be converted to appropriate type and send to ACP.
eg. 
http://localhost:4000/protocol_name/param1/param2/param3
  • results - list of fields names and types which will be received from ACP.

Supported types:

  • boolean
  • unsigned_byte
  • signed_byte
  • unsigned_short
  • signed_short
  • unsigned_int
  • signed_int
  • unsigned_long
  • signed_long
  • short_float
  • float
  • chars

Download binaries and source

Binaries download

Source download

Build from sources

Prerequisites

Install Go SDK (or extract zip archive) and setup GOPATH

Go download

GOPATH

Instalation

go get github.com/kpawlik/goworld

Compilation

go to GOPATH/github.com/kpawlik/goworld/goworldc run:

go build main.go -o c:\tmp\goworld.exe

Start HTTP Server and Worker

Start HTTP Server

To start HTTP sever:

  1. Create configuration file with Server.port, at least list protocol enbled and at least one worker definition.
  2. Open command line terminal and type:
goworld.exe -t http -c [PATH TO CONFIG FILE]

HTTP server will start on defined port number, running workers from definition will be connected. Workers can be start before or after you start HTTP.

Start Worker
  1. To start worker you can use Magik procedure start_goworld_worker from file goworld.magik
start_goworld_worker([NAME], [PATH TO goworld.exe], [PATH TO CONFIG FILE], [PATH TO LOG FILE])

NAME - uniqe worker name. Need to be the same as in configuration file

PATH TO goworld.exe - path to goworld executable file

PATH TO CONFIG FILE - path to JSON configuration file

PATH TO LOG FILE - path where to store log file for this worker

  1. This procedure will start ACP process. In background it will call:
goworld.exe -n [NAME] -t worker -c [PATH TO CONFIG FILE] -l [PATH TO LOG FILE]

Tutorial

Quick start

This example is for Windows, but this works the same way for Linux. This quick start example shows how to run goworld with example list protocol, which just lists fields from Smallworld objects.

  • Download appropriate executable file to C:\tmp\

  • Create JSON config file C:\tmp\goworld.json

    {
    	"server": {
    		"port": 4000,
    		"protocols": 
    			[
    				{
    					"name": "list",
    					"enabled": true
    				}
    			]
    	},
    	"workers": [{
    		"port": 4002,
    		"host": "localhost",
    		"name": "w1"
    	}, {
    		"port": 4001,
    		"host": "localhost",
    		"name": "w2"
    	}]
    }
    
  • load Magik source file goworld.magik from magik folder into the Smallworld session

  • in the Smallworld console type:

    start_goworld_worker("w1", "c:\tmp\goworld.exe", "c:\tmp\goworld.json", "c:\tmp\w1.log")
    $
    start_goworld_worker("w2", "c:\tmp\goworld.exe", "c:\tmp\goworld.json", "c:\tmp\w2.log")
    $
    

    this, will start two concurrent workers w1 and w2, which will communicate with HTTP server via RPC protocol on ports 4001 n 4002. Procedure start_goworld_worker takes 4 parameters:

    • worker_name - unique name which need to be same as in configuration file
    • path to goworld.exe
    • path to config file
    • path where worker log file will be written
  • open Windows command line and type:

    C:\tmp\goworld.exe -t http -c C:\tmp\goworld.json
    

    this, will start the HTTP server on port 4000

  • start internet browser and type in address bar

    http://localhost:4000/list/[DATASET NAME]/[COLLECTION NAME]/[LIMIT, 0 = ALL]/[LIST OF FIELDS SEPARATED BY "/"]
    eg.
    http://localhost:4000/list/gis/hotel/100/id/name/address1/address2
    

    application will display list 100 (or less if size of collection is less then 100) of JSON object eg.

    [
    	{
    		"address1": "154 Palad Road",
    		"address2": "Wyoming",
    		"id": "condition(does_not_understand: Object hotel133:(AAA Guest House) does not understand message id)",
    		"name": "AAA Guest House"
    	},
    	{
    		"address1": "219 Pinguin Road",
    		"address2": "Miami",
    		"id": "condition(does_not_understand: Object hotel133:(All 5 Seasons) does not understand message id)",
    		"name": "All 5 Seasons"
    	},
    	...
    	...
    	...
    ]
    
Define custom protocol
  1. Create config file same as is described in quick start. Add new protocol definition with name find_hotel
...
"protocols": [
	{
      	"name": "find_hotel",
        "enabled": true,
        "params": [
            {
                "name": "name",
                "type": "chars"
            }
        ],
        "results": [
            {
                "name": "address_1",
                "type": "chars"
            },
            {
                "name": "address_2",
                "type": "chars"
            },
            {
                "name": "x",
                "type": "float"
            },
			{
                "name": "y",
                "type": "float"
            }
     	 	]
  	}
],
...
  1. Create new Magik method in class goworld, this method will handle new protocol
_method goworld.find_hotel_protocol()
## 
## 
	!print_float_precision! << 12
	# This will get name from "params"
    	_local name << _self.get_chars()
	# send status
	_self.send_success_status()
	_self.flush()
	_local ds << gis_program_manager.databases[:gis]
	_local coll << ds.collections[:hotel]
	_local select << coll.select(predicate.eq(:name, name))
	# send no of recs
	_self.put_unsigned_int(select.size)
	_self.flush()       
	# send results fields in the same order as in config file 
	_for rec _over select.fast_elements()
	_loop 
	    _self.put_chars(write_string(rec.address1))
	    _self.flush()
	    _self.put_chars(write_string(rec.address2))
	    _self.flush()
	    _self.put_float(rec.location.x)
	    _self.flush()
	    _self.put_float(rec.location.y)
	    _self.flush()
	 _endloop 
_endmethod
$
  1. Register method with protocol name. Name must be the same as protocol name in config file:
goworld.register_protocol("find_hotel", :|find_hotel_protocol()|)
$
  1. Start goworld worker:
start_goworld_worker("w1", "c:\tmp\goworld.exe", "c:\tmp\goworld.json", "c:\tmp\w1.log")
$
  1. Start HTTP server
c:\tmp\goworld.exe -t http -c c:\tmp\goworld.json
  1. In browser type:
http://localhost:4000/find_hotel/[HOTEL_NAME]

Protocols

Protocol describes how to Magik ACP and goworld worker communicate.

List protocol

List protocol starts with list prefix eg.

http://localhost:4000/list/gis/hotel/100/id/name/address1

Request structure: http://[HOST]:[PORT]/list/[DATASET]/[COLLECTION]/[LIMIT, 0 = ALL]/[LIST OF FIELDS SEPARATED BY "/"]

Protocol returns a list of JSON objects, each JSON object contain LIST OF FIELDS and VALUES from COLLECTION. Number of objects is limited by LIMIT or size of COLLECTION. If DATASET or COLLECTION does not exists, error message will be returned. If field with requested name does not exists, then error will be returned as a field value.

Communication:

  1. goworld send to Magik ACP a vector of chars, this is a part of HTTP address which occurs after protocol name. http://localhost:4000/list/gis/hotel/100/id/name/address1/address2 it will be gis/hotel/100/id/name/address1/address2. Magik code:
_local path << _self.get_chars()
  1. Magik ACP send to goworld status code as unsigned byte

    • 0 means no error
    _self.put_unsigned_byte(0)
    
    • > 0 means error (no such dataset, no access etc.). In this case as a next Magik ACP needs to send string with error message
    _self.put_unsigned_byte(1)
    _self.chars(write_string("No such dataset with name", dataset_name))
    _continue
    
  2. Magik ACP send to goworld number of record which will be send as unsigned int

_self.put_unsigned_int(records_to_get)
  1. Magik ACP send to goworld number of fields which will be send as unsigned int
_self.put_unsigned_int(no_of_fields)
  1. in the loop magik ACP sends field names and field values
_self.put_chars(field_name)
_self.flush()
_self.put_chars(field_value)
_self.flush()
Custom protocol

To define custom protocol you need to:

  1. Define protocol in config file.
  2. Protocol name
  3. Enabled flag
  4. List of entry parameters
  5. List of result fields
  6. Create magik method which will handle protocol on Smallworld side. This method need to:
    1. Receive all parameters defined in config file
    2. Send sucess code, or error code and error message
    3. Send number of records to send
    4. In loop, send fields which are defined in config file
  7. Register magik method with protocol name

See example in tutotrial.

Stop

To stop just open system Task Manager and kill all goworld processes.


Documentation

Index

Constants

View Source
const (
	// SucessStatus value which should be returned from ACP if no error ocure
	SucessStatus = 0
)

Variables

This section is empty.

Functions

func ParseStringParam

func ParseStringParam(value, dataType string) (result interface{}, err error)

func StartServer

func StartServer(config *Config, mode WorkMode)

StartServer initialize workers and starts HTTP server

func StartWorker

func StartWorker(config *Config, name string, mode WorkMode)

StartWorker start register structs and start RPC server

Types

type Acp

type Acp struct {
	Name string
	// contains filtered or unexported fields
}

Acp holds I/O buffer to communicate with Magik ACP

func NewAcp

func NewAcp(name string) *Acp

NewAcp creates and init new Acp with name

func (*Acp) Connect

func (a *Acp) Connect(processName string, protocolMin, protocolMax int) (err error)

Connect verify connection and protocol to Acp

func (*Acp) EstablishProtocol

func (a *Acp) EstablishProtocol(minProtocol, maxProtocol int) bool

EstablishProtocol checks Acp protocol

func (*Acp) Flush

func (a *Acp) Flush()

Flush send buffer data

func (*Acp) Get

func (a *Acp) Get(dataType string) (value interface{}, err *AcpErr)

Get method reads dataType value from ACP

func (*Acp) GetBool

func (a *Acp) GetBool() bool

GetBool reads boolean value from Acp input

func (*Acp) GetByte

func (a *Acp) GetByte() int

GetByte reads byte from Acp input

func (*Acp) GetFloat

func (a *Acp) GetFloat() float64

GetFloat read float64 from Acp input

func (*Acp) GetInt

func (a *Acp) GetInt() int

GetInt reads unsigned int from Acp input

func (*Acp) GetLong

func (a *Acp) GetLong() int64

GetLong reads long from Acp input

func (*Acp) GetShort

func (a *Acp) GetShort() int

GetShort reads short from Acp input

func (*Acp) GetShortFloat

func (a *Acp) GetShortFloat() float32

GetShortFloat read float32 from Acp input

func (*Acp) GetString

func (a *Acp) GetString() string

GetString reads string from Acp input

func (*Acp) GetUbyte

func (a *Acp) GetUbyte() int

GetUbyte reads unsigned byte from Acp input

func (*Acp) GetUint

func (a *Acp) GetUint() int

GetUint reads unsigned int from Acp input

func (*Acp) GetUlong

func (a *Acp) GetUlong() uint64

GetUlong reads unsigned long from Acp input

func (*Acp) GetUshort

func (a *Acp) GetUshort() int

GetUshort reads unsigned short from Acp input

func (*Acp) Put

func (a *Acp) Put(dataType string, value interface{}) (err error)

Put convert value to dataType and send this value to ACP

func (*Acp) PutBool

func (a *Acp) PutBool(b bool)

PutBool sends boolean value to Acp output

func (*Acp) PutByte

func (a *Acp) PutByte(value int8)

PutByte sends byte value to Acp output

func (*Acp) PutFloat

func (a *Acp) PutFloat(value float64)

PutFloat sends float value to Acp output

func (*Acp) PutInt

func (a *Acp) PutInt(value int32)

PutInt sends int value to Acp output

func (*Acp) PutLong

func (a *Acp) PutLong(value int64)

PutLong sends long value to Acp output

func (*Acp) PutShort

func (a *Acp) PutShort(value int16)

PutShort sends short value to Acp output

func (*Acp) PutShortFloat

func (a *Acp) PutShortFloat(value float32)

PutShortFloat sends short float value to Acp output

func (*Acp) PutString

func (a *Acp) PutString(s string)

PutString sends string value to Acp output

func (*Acp) PutUbyte

func (a *Acp) PutUbyte(value uint8)

PutUbyte sends unsigned byte value to Acp output

func (*Acp) PutUint

func (a *Acp) PutUint(value uint32)

PutUint sends int value to Acp output

func (*Acp) PutUlong

func (a *Acp) PutUlong(value uint64)

PutUlong sends unsigned long value to Acp output

func (*Acp) PutUshort

func (a *Acp) PutUshort(value uint16)

PutUshort sends unsigned short value to Acp output

func (*Acp) ReadNumber

func (a *Acp) ReadNumber(data interface{})

ReadNumber reads number from Acp input

func (*Acp) VerifyConnection

func (a *Acp) VerifyConnection(name string) bool

VerifyConnection verify Acp process name

func (*Acp) Write

func (a *Acp) Write(buf []byte)

Write writes buffer to Acp output

type AcpErr

type AcpErr struct {
	Err string
}

func NewAcpErr

func NewAcpErr(msg string) *AcpErr

func (*AcpErr) Error

func (err *AcpErr) Error() string

type Body

type Body []BodyElement

Body list of BodyElemnts. JSON response object.

type BodyElement

type BodyElement map[string]interface{}

BodyElement is a type which is a part of JSON reposnse

type Config

type Config struct {
	Server  ServerConf
	Workers []*WorkerConf
}

Config application configuration structure

func ReadConf

func ReadConf(filePath string) (conf *Config, err error)

ReadConf reads and decodes JSON from file

func (Config) GetProtocolDef

func (c Config) GetProtocolDef(name string) *ProtocolConf

GetProtocolDef returns Protocol definition of nil if not found

func (Config) GetWorkerDef

func (c Config) GetWorkerDef(name string) *WorkerConf

GetWorkerDef returns worker connection definition of nil if not found

type ParameterConf

type ParameterConf struct {
	Name string
	Type string
}

ParameterConf is parameter name and type definition. Type could take values string, unsigned_int, signed_int, etc

type ProtocolConf

type ProtocolConf struct {
	Name    string
	Enabled bool
	Params  []*ParameterConf
	Results []*ParameterConf
}

ProtocolConf is a definition of protocol. Contains name, list of entry parameters and list of results fields

type ReqHandler

type ReqHandler struct {
	Online   workerChan
	Offline  workerChan
	Config   *Config
	WorkMode WorkMode
}

ReqHandler struct to implement ServeHTTP method

func (*ReqHandler) ServeHTTP

func (r *ReqHandler) ServeHTTP(w http.ResponseWriter, req *http.Request)

ServeHTTP is http request handler.

type Request

type Request struct {
	Path     string
	Protocol *ProtocolConf
}

Request struct

type Response

type Response struct {
	Body  Body
	Error error
}

Response struct Body - result map (field, value) to json

type ServerConf

type ServerConf struct {
	Port      int
	Protocols []*ProtocolConf
}

ServerConf server configuration

type WorkMode

type WorkMode int

WorkMode - enumerator with serwer mode types

const (
	// Version number
	Version = "0.9"
	// UnknownMode is unrecognized mode
	UnknownMode WorkMode = iota
	// NormalMode is production mode
	NormalMode
	// TestMode  to test communication between Acp and worker
	TestMode
)

func WorkModeFromString

func WorkModeFromString(mode string) WorkMode

WorkModeFromString converts string value to enumerator if not found then UnknownMode returns

func (WorkMode) String

func (w WorkMode) String() string

type Worker

type Worker struct {
	Port       int
	WorkerName string
	Protocol   *ProtocolConf
}

Worker type to wrap RPC communication

func (*Worker) Custom

func (w *Worker) Custom(request *Request, resp *Response) (err error)

Custom handles communication defined by custom protocol in config file

func (*Worker) GetTestResponse

func (t *Worker) GetTestResponse(request *Request, resp *Response) error

GetTestResponse returns response object from worker

func (*Worker) ListObjectsFields

func (w *Worker) ListObjectsFields(request *Request, resp *Response) error

ListObjectsFields returns response object from worker Demo protocol method. Returns list of fields from objects. All data are converted to strings

type WorkerConf

type WorkerConf struct {
	Host string
	Name string
	Port int
}

WorkerConf wrkers configuration

type WorkerConnection

type WorkerConnection struct {
	Name string
	Host string
	Port int
	Conn *rpc.Client
}

WorkerConnection type to store worker conenction and data

Directories

Path Synopsis
gosworld project main.go
gosworld project main.go

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL