pulsar

go-pulsar

go-pulsar is a pulsar client library.

• Pulsar Documentation

Development Status

Alpha.

go-pulsar is still under heavy development. Some functionality are known to be broken, missing or incomplete. The interface may also change.

TODO

• partitioned topics functions
• payload compression
• unimplemented commands
• error handling

How to build

pulsar-client

pulsar-client is a cli tool to use go-pulsar library. It's like Java's Pulsar client tool.

$ mkdir work && cd work
$ export GOPATH="$(pwd)"
$ mkdir -p src/github.com/t2y/go-pulsar
$ cd src/github.com/t2y/go-pulsar
$ git clone git@github.com:t2y/go-pulsar.git .

Get dependency libraries and build it.

$ make install-glide
$ make deps
$ make
$ ./bin/pulsar-client --help
Usage:
  pulsar-client [OPTIONS]

Application Options:
      --serviceUrl=                 pulsar service url [$PULSAR_SERVICE_URL]
      --authMethod=                 authentication method [$PULSAR_AUTH_METHOD]
      --authParams=                 authentication params [$PULSAR_AUTH_PARAMS]
      --useTls                      use tls to connect [$USE_TLS]
      --tlsAllowInsecureConnection  allow insecure tls connection [$TLS_ALLOW_INSECURE_CONNECTION]
      --athenzConf=                 path to athenz config file [$PULSAR_ATHENZ_CONF]
      --athenzAuthHeader=           athenz authentication header [$PULSAR_ATHENZ_AUTH_HEADER]
      --conf=                       path to pulsar config file [$PULSAR_CONF]
      --verbose                     use verbose mode [$VERBOSE]
      --timeout=                    timeout to communicate with pulsar broker [$PULSAR_TIMEOUT]
      --command=                    produce or consume [$PULSAR_COMMAND]
      --topic=                      topic name [$PULSAR_TOPIC]
      --messages=                   messages to produce [$PULSAR_MESSAGES]
      --properties=                 properties to produce. e.g) key1:value1,key2:value2 [$PULSAR_PROPERTIES]
      --numMessages=                number of messages to consume (default: 1) [$PULSAR_NUM_MESSAGES]
      --subscriptionName=           subscription name [$PULSAR_SUBSCRIPTION_NAME]
      --subscriptionType=           subscription type: exclusive, shared, failover (default: exclusive) [$PULSAR_SUBSCRIPTION_TYPE]

Help Options:
  -h, --help                        Show this help message

Some options can be set by ini file. There're a sample file in example directory.

$ cat example/default.ini
log_level = info
service_url = pulsar://localhost:6650/
timeout = 40s
min_connection_num = 2
max_connection_num = 20

Code generation for pulsar protocol

pulsar protocol is defined using Protocol Buffers.

• Pulsar binary protocol specification

go-pulsar also uses PulsarApi.proto and generates go source code: PulsarApi.pb.go.

First of all, install protoc command for your platform. For example, use Homebrew on macOS.

$ brew install protobuf

To get latest proto file and re-generate go source code, make as follows.

$ make install-pb
$ make gen-pb

The following files are updated

• proto/PulsarApi.proto
• proto/pb/PulsarApi.pb.go

Getting started

Startup Pulsar server

Build and install pulsar server.

$ git clone https://github.com/apache/incubator-pulsar.git
$ cd pulsar
$ mvn install -DskipTests

Start up a standalone pulsar server for development.

$ ./bin/pulsar standalone

• Getting started with Pulsar

Producer

$ ./bin/pulsar-client --conf example/default.ini --command produce --topic "persistent://sample/standalone/ns1/my-topic" --messages "Hello Pulsar"
INFO[2017-06-15T08:43:47.887709192+09:00] read and parse ini file                       iniConf=&{info pulsar://localhost:6650/ 40s 2 20 pulsar://localhost:6650/ info} path="example/default.ini"
INFO[2017-06-15T08:43:49.222165203+09:00] messages successfully produced                messages=[Hello Pulsar] properties=[]

The --verbose option makes debug easy. It shows communications between producers/consumers and brokers.

Consumer

$ ./bin/pulsar-client --conf example/default.ini --command consume --topic "persistent://sample/standalone/ns1/my-topic" --subscriptionName sub
INFO[2017-06-15T08:50:33.467806336+09:00] read and parse ini file                       iniConf=&{info pulsar://localhost:6650/ 40s 2 20 pulsar://localhost:6650/ info} path="example/default.ini"
INFO[2017-06-15T08:50:34.515306354+09:00] messages successfully consumed                key-value=[] message="Hello Pulsar"

Authentication

go-pulsar supports Athenz authentication.

• Pulsar Authentication: Athenz

Read above documentation for each athenz parameters. Then, you can set some or all parameters on your needs into ini file. There're a sample file for athenz in example directory.

$ cat example/athenz.ini 
log_level = info
service_url = pulsar://localhost:6650/
timeout = 40s
min_connection_num = 2
max_connection_num = 20

auth_method=athenz
auth_params=tenantDomain:${yourDomain},tenantService:${yourService},providerDomain:${yourProviderDomain},privateKeyPath:${pathToPrivateKeyFile},keyId:0
use_tls=true
tls_allow_insecure_connection=false
tls_trust_certs_filepath=${pathToRootCaCertsFileIfNeeded}

athenz_conf=${pathToAthenzConfFile}
athenz_auth_header=${yourAthenzAuthenticationHeader}

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0