openGemini

README

简体中文 | English

Official Website | Docs | Contribution Guide | RoadMap | Performance

About openGemini

openGemini is a cloud-native distributed time series database, mainly for the Internet of Things and operation and maintenance monitoring scenarios. It focuses on the storage and analysis of massive telemetry data. It can be widely used in logistics, Internet of Vehicles, Internet, intelligent manufacturing, modern agriculture, transportation, and energy, electric power, medicine and other industries, it has excellent reading and writing performance and efficient data analysis capabilities. It uses SQL-like query language, easy to install, and convenient in operation and maintenance.

If you have used openGemini, please let us know and click Submit Usage Information

Why openGemini

Compared with other time series database, openGemini has the following advantages:

• High Performance

  Store and query data quickly and efficiently with automatic partitioning, LSM-based storage techniques, and better data process engineering, seeperformance reports

• High Scalability

  Adopting MPP architecture, support distributed cluster deployment and can be flexibly expanded as business grows to meet higher load requirements, see Architecture

• High Cardinality

  A new high cardinality storage engine solves problems such as excessive index memory usage and low read and write performance, see high cardinality storage engine

• Data Compression

  Data is stored in column format, and different data types use dedicated data compression algorithms. The data compression ratio is as high as 15:1 or higher, see Data compression algorithm

• Flexible Deployment

  Supports stand-alone and cluster deployment, and supports various forms of deployment such as virtual machines, physical machines, containers, and clouds. Provides one-click container deployment tool (openGemini-operator) and one-click normal installation and deployment tool (gemix)

• Ecosystem compatibility

  Compatible with InfluxDB v1.x Line Protocol, Influxql and read/write APIs, also compatible with existing InfluxDB toolchain, supports prometheus remote read/write API

Third-party supports

currently openGemini only supports Linux OS, and supports InfluxDB ecosystem toolchains, such as:

Drivers: JavaScript, Java, C/C++, Ruby, Go, Python, PHP

Client: ts-cli

Data access tools: StatsD, EMQ, Telegraf, Icinga2, HiveMQ, Kafka, MQTT

Data insight tools: Chronograf, Grafana

Big data and data analysis systems: Zeppelin, Flink, Spark, Kapacitor, etc.

Quick Start

For a more detailed introduction, please visit our official website User Guide

This section mainly contains the following:

• How to compile openGemini source code
• How to run openGemini

Compiling environment information

GO version v1.19+

Python version v3.7+

How to set GO environment variables

Open ~/.profile configuration file and add the following configurations to the end of the file:

export GOPATH=/path/to/dir
export GOBIN=$GOPATH/bin
export GO111MODULE=on
export GONOSUMDB=*
export GOSUMDB=off

Compiling

1. Clone source codes from Github

> cd $GOPATH
> mkdir -p {pkg,bin,src}
> cd src
> git clone https://github.com/openGemini/openGemini.git

1. Enter the home directory

> cd openGemini

1. Compiling

> export CGO_LDFLAGS="-Wl,-z,now -Wl,-z,relro -Wl,-z,noexecstack -fPIE -ftrapv"
> export CGO_CFLAGS="-fstack-protector-strong -D_FORTIFY_SOURCE=2 -O2"
> python build.py

The compiled binary file is in the build directory

> ls build
ts-cli ts-meta ts-monitor ts-server  ts-sql  ts-store

Configuration

The configuration file is in the conf directory. For more information about the configuration items, please refer to User Guide --> Reference --> Configuration Files

Run openGemini

Standalone operation

> cd openGemini
> mkdir -p /tmp/openGemini
> sh scripts/install.sh

Refer to cluster deployments in User Guide

Using openGemini

Use the client to connect to openGemini

> ts-cli --host 127.0.0.1 --port 8086

After successful login, the following message will be displayed

> ts-cli --host 127.0.0.1 --port 8086
openGemini CLI 0.1.0 (rev-revision)
Please use 'quit', 'exit' or 'Ctrl-D' to exit this program
> 

Create a database

> create database sensordb
> use sensordb

The openGemini supports three ways for creating a measurement.

• Implicit creation, automatic creation measurement when data is written. By default, partition is based on time.
• explicit creation, without specifying the partition key, as the same as implicit creation.

> create measurement sensor

• explicit creation, specifying partition keys during table creation, During data storage, the table will be primarily partitioned by time, and then secondarily partitioned according to the specified partition keys.

> create measurement sensor with shardkey farmID

Write data

insert sensor,farmID=f1,deviceID=d0 sensorID="s20",value=50.98

Query data

> select * from sensor
name: sensor
+---------------------+----------+--------+----------+-------+
| time                | deviceID | farmID | sensorID | value |
+---------------------+----------+--------+----------+-------+
| 1657959880895515464 | d0       | f1     | s20      | 50.98 |
+---------------------+----------+--------+----------+-------+
5 columns,1 rows in set
Elapsed: 7.723332ms  

Join & Contribute

Tips for Contribution

Contact Us

1. Slack

2. Twitter

3. Email

4. mailing list

License

openGemini is licensed under the Apache License 2.0. Refer to LICENSE for more details.

For third-party software usage notice, see Open_Source_Software_Notice