desq

DESQ: Decentralized Event-Sourcing & Querying

This repository contains an experimental architecture and proof-of-concept implementation for event-sourcing acros non-hierarchical 'nodes' and history-aware queries across them.

Local development with microk8s

To develop on your local machine, install a single-node kubernetes cluster such as microk8s.

Make sure to install the dns, storage and ingress addons in microk8s:

microk8s enable dns storage ingress

Make sure your local kubectl executable is able to use the microk8s kubeconfig. You can export the config via microk8s config and add this information to your local kubeconfig (~/.kube/config for example).

Follow the steps in events-db/README.md "Local development aka kubedev" to setup postgres-operator.

Then run:

kubectl config use-context microk8s
kubectl create ns desq-zone-groenestede
skaffold dev -f=skaffold.zone.yaml --cleanup=false -p kubedev-groenestede

It may take some time for all resources to become healthy.

To test that the zone-api is up, make sure you can connect to it, for example using a port-forward

kubectl port-forward service/zone-api 8081:8000 --namespace=desq-zone-groenestede
curl localhost:8081/status

To start the second development zone, open another terminal and run:

kubectl create ns desq-zone-heidedal
skaffold dev -f=skaffold.zone.yaml --cleanup=false -p kubedev-heidedal

Using the management UI

A simple management UI for the XBRP prototype.

To make the management-ui work with a nice domain, add the host ip address of the microk8s cluster to your /etc/hosts file. Find the ip of your microk8s cluster, for example using kubectl cluster-info and add the following lines to your hosts file:

# DESQ
<your microk8s node ip> command-api.bovenhove.desq.demo
<your microk8s node ip> management-api.bovenhove.desq.demo
<your microk8s node ip> bovenhove.desq.demo

<your microk8s node ip> command-api.oudescha.desq.demo
<your microk8s node ip> management-api.oudescha.desq.demo
<your microk8s node ip> oudescha.desq.demo

<your microk8s node ip> command-api.groenestede.desq.demo
<your microk8s node ip> management-api.groenestede.desq.demo
<your microk8s node ip> groenestede.desq.demo

<your microk8s node ip> command-api.heidedal.desq.demo
<your microk8s node ip> management-api.heidedal.desq.demo
<your microk8s node ip> heidedal.desq.demo

For more information on running the management UI see the Management UI README.

Architecture and components in DESQ

This repository is a mono-repo. It contains all components required to run DESQ.

• There is a clear distinction between types of components, in line with the vision Common Ground:
  • UI components provide user interfaces
  • Workers are independent process components
  • API's expose services
  • DB components store data
• The common folder contains shared packages.
• The example-desq-client folder contains a few rudimentary example clients.

Zone components

A zone is an independent region of data storage and computation that corresponds to the responsibilities of one legal actor (for example a municipality).

command-api

The command-api is reponsible for processing Commands.

event-db

The event-db is responsible for storing Events.

event-disclose-api

The event-disclose-api is responsible for disclosing Events to other zones within the network.

event-worker

The event-worker watches the event-db. The event-worker is responsible for performing work that needs to be done based on the Events that are added.

journal-db

The journal-db is responsible for storing Journals.

journal-update-api

The journal-update-api is responsible for processing updates to Journals.

management_ui

The management_ui is provides the user interface that can be used to see and manage a Zone.

management-api

The management-api is responsible for performing management tasks in a Zone.

query-api

The query-api is responsible for answering Queries.

zone-api

The zone-api is responsible for providing services related to a Zone.

Catalog components

A catalog is a resource that is shared by all participants in the DESQ network. It is used to provide discovery services in the network.

catalog-api

The catalog-api provides a service for determining which zone is responsible ('bijhoudingsverantwoordelijk') for each person journal in the system.