compass

Compass

Compass is a search and discovery engine built for querying application deployments, datasets and meta resources. It can also optionally track data flow relationships between these resources and allow the user to view a representation of the data flow graph.

Key Features

Discover why users choose Compass as their main data discovery and lineage service

• Full text search Faster and better search results powered by ElasticSearch full text search capability.
• Search Tuning Narrow down your search results by adding filters, getting your crisp results.
• Data Lineage Understand the relationship between metadata with data lineage interface.
• Scale: Compass scales in an instant, both vertically and horizontally for high performance.
• Extensibility: Add your own metadata types and resources to support wide variety of metadata.
• Runtime: Compass can run inside VMs or containers in a fully managed runtime environment like kubernetes.

Documentation

Explore the following resources to get started with Compass:

• Guides provides guidance on ingesting and querying metadata from Compass.
• Concepts describes all important Compass concepts.
• Reference contains details about configurations, metrics and other aspects of Compass.
• Contribute contains resources for anyone who wants to contribute to Compass.

Installation

Install Compass on macOS, Windows, Linux, OpenBSD, FreeBSD, and on any machine.
Refer this for installations and configurations

Binary (Cross-platform)

Download the appropriate version for your platform from releases page. Once downloaded, the binary can be run from anywhere. You don’t need to install it into a global location. This works well for shared hosts and other systems where you don’t have a privileged account. Ideally, you should install it somewhere in your PATH for easy use. /usr/local/bin is the most probable location.

macOS

compass is available via a Homebrew Tap, and as downloadable binary from the releases page:

brew install raystack/tap/compass

To upgrade to the latest version:

brew upgrade compass

Check for installed compass version

compass version

Linux

compass is available as downloadable binaries from the releases page. Download the .deb or .rpm from the releases page and install with sudo dpkg -i and sudo rpm -i respectively.

Windows

compass is available via scoop, and as a downloadable binary from the releases page:

scoop bucket add compass https://github.com/raystack/scoop-bucket.git

To upgrade to the latest version:

scoop update compass

Docker

We provide ready to use Docker container images. To pull the latest image:

docker pull raystack/compass:latest

To pull a specific version:

docker pull raystack/compass:v0.3.2

If you like to have a shell alias that runs the latest version of compass from docker whenever you type compass:

mkdir -p $HOME/.config/raystack
alias compass="docker run -e HOME=/tmp -v $HOME/.config/raystack:/tmp/.config/raystack --user $(id -u):$(id -g) --rm -it -p 3306:3306/tcp raystack/compass:latest"

Usage

Compass is purely API-driven. It is very easy to get started with Compass. It provides CLI, HTTP and GRPC APIs for simpler developer experience.

CLI

Compass CLI is fully featured and simple to use, even for those who have very limited experience working from the command line. Run compass --help to see list of all available commands and instructions to use.

List of commands

compass --help

Print command reference

compass reference

API

Compass provides a fully-featured GRPC and HTTP API to interact with Compass server. Both APIs adheres to a set of standards that are rigidly followed. Please refer to proton for GRPC API definitions.

Building Compass

Begin by cloning this repository then you have two ways in which you can build compass

• As a native executable
• As a docker image

To build compass as a native executable, run make inside the cloned repository.

make

This will create the compass binary in the root directory

Building compass' Docker image is just a simple, just run docker build command and optionally name the image

docker build . -t compass

Migration

Before serving Compass app, we need to run the migration first. Run this docker command to migrate Compass.

$ docker run --rm --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password raystack/compass compass server migrate

If you are using Compass binary, you can run this command.

./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> server migrate"

Serving locally

Once the migration has been done, Compass server can be started with this command.

docker run --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password raystack/compass compass server start

If you are using Compass binary, you can run this command.

./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> server start"

Running tests

Running all unit tests

make test

The tests combine both unit and integration tests, the test suite requires docker to run elasticsearch. In case you wish to test against an existing elasticsearch cluster, set the value of ES_TEST_SERVER_URL to the URL of the elasticsearch server.

Contribute

Development of Compass happens in the open on GitHub, and we are grateful to the community for contributing bugfixes and improvements. Read below to learn how you can take part in improving Compass.

Read our contributing guide to learn about our development process, how to propose bugfixes and improvements, and how to build and test your changes to Compass.

To help you get your feet wet and get you familiar with our contribution process, we have a list of good first issues that contain bugs which have a relatively limited scope. This is a great place to get started.

This project exists thanks to all the contributors.

License

Compass is Apache 2.0 licensed.