gateways

gateways

Clean-architecture SDK for software-defined multi-cloud backend environments including automated-deployment

Intro

The concept is that customers use the SDK to quickly create their own REST API and minimise effort writing modern resilient backends

This example demonstrates configuration of a three region, multicloud back end.

The backend environment is software defined through the json and built on demand.

The module code is written to make it relatively trivial to

• add new clouds, datastores or brokers

• operate one, two, three or more regions

• operate different architectural patterns within the same environment

Each collection(entity) SDK can be configured as CQRS, datastore or broker

Explanation of example.json (configV2 format)

Example.json offers three regions, two Google Cloud and one Microsoft.

    "environmentPrefix": "dev",
    "regionList": ["gcp1", "gcp2", "ms1"],

We configure a defaultServiceConfig per region

    "defaultServiceConfig" : {
        "gcp1" : "GCP_CQRS_1",
        "gcp2" : "GCP_CQRS_2",
        "ms1"  : "MS_CQRS_1"
    },

Each Service Config can be defined to be CQRS (datastore+broker), DATASTORE-only or BROKER-only.

There are cloud parameters to login to the project or subscription, and then specific parameters defining the product, product auth and any build defaults.

Below we see the Google Firestore and PubSub example

        "GCP_CQRS_1": {
            "serviceType": "SERVICETYPE_CQRS",
            "region": "gcp1",
            "cloudParameters": {
                "cloudType": "GOOGLE_CLOUD",
                "gcpConfig": {
                    "projectID": "gcp1europewest",
                    "projectOwnerAccount": "gcp1europewest-adm",
                    "projectApplicationCredentialsFile": "/opt/projX/secrets/gcp1europewest-adm.json"
                }        
            },
            "brokerParameters": {
                "brokerType": "BROKER_PUBSUB",
                "multiRegionForwarder": true,
                "gcpBroker": {
                    "ownerAccount": "gcp1europewest-pubsub-adm",
                    "applicationCredentialsFile": "/opt/projX/secrets/gcp1europewest-pubsub-adm.json"
                }
            },
            "datastoreParameters": {
                "datastoreType": "DATASTORE_FIRESTORE",
                "gcpDatastore": {
                    "ownerAccount": "gcp1europewest-firestore-adm",
                    "applicationCredentialsFile": "/opt/projX/secrets/gcp1europewest-firestore-adm.json",
                    "firestoreRegion": "europe-west"
                }
            }
        },

The Microsoft serviceType configures Cosmos Db with Mongodb Client and ServiceBus for the broker.

        "MS_CQRS_1": {
            "serviceType": "SERVICETYPE_CQRS",
            "region": "ms1",
            "cloudParameters": {
                "cloudType": "AZURE",
                "azureConfig": {
                    "subscriptionId": "<azure_subscription_id>",
                    "resourceGroup": "projX-we2-rg",
                    "azureAuthLocation": "/opt/projX/secrets/adm-SPN.json"
                }
            },
            "brokerParameters": {
                "brokerType": "BROKER_SERVICEBUS",
                "multiRegionForwarder": true,
                "azureBroker": {
                    "brokerPrefix": "projX-b2",
                    "brokerSku": "Standard",
                    "connectionString": "Endpoint=sb://projX-namespace.servicebus.windows.net/;SharedAccessKeyName=projX-authorule;SharedAccessKey=<shared_access_key>="
                }
            }, 
            "datastoreParameters": {
                "datastoreType": "DATASTORE_COSMOSDB_MONGODB",
                "azureDatastore": {
                    "datastorePrefix": "projX-d2",
                    "cosmosDbAccountName": "projX",
                    "cosmosDbKind": "Mongodb",
                    "cosmosDbVersion": "4.0"
                },
                "mongoDb": {
                    "connectionString": "mongodb://projX-cosmosdb:HASHCODE==@projX-cosmosdb.mongo.cosmos.azure.com:10255/?ssl=true&replicaSet=globaldb&retrywrites=false&maxIdleTimeMS=120000&appName=@projX-cosmosdb@"
                }
            }
        },

Five collections are defined: country, organisation, rbac, topology, user. This refers to a collection of json documents

defaultServiceConfig applies except where they are overridden, e.g. country is just stored in the DATASTORES and not via CQRS. The same approach applies to brokers.

    "collections": {
        "country": {
            "enabled": "true",
            "datastoreCollection": "country",
            "regionalConfig" : {
                "gcp1" : "GCP_DATASTORE_1",
                "gcp2" : "GCP_DATASTORE_2",
                "ms1"  : "MS_DATASTORE_1"
            }
        },
        "organisation": {
            "enabled": "true",
            "brokerTopic": "organisation",
            "brokerSubscription": "organisation",
            "datastoreCollection": "organisation",
            "regionalConfig" : { }
        },
        "rbac": { },
        "topology": { },
        "user": { },

We also have datastore and broker defaults. When we "install", these prefixes and suffixes are added to the collection name to name the objects (topics, subscriptions etc)

    "datastoreDefaults": {
        "datastoreCollectionPrefix": "c"
    },
    "brokerDefaults": {
        "cloudEventDomain": "mydomain.com",
        "brokerTopicPrefix": "t",
        "brokerSubscriptionPrefix": "s",
        "brokerDeadLetterSuffix": "DL",
        "brokerForwarderSuffix": "FW",
        "brokerMaxDeliveryAttempts": 5,
        "brokerMaxAckTimeInSeconds": 20,
        "forwarders": {
            "gcp1": ["gcp2", "ms1"],
            "gcp2": ["ms1", "gcp1"],
            "ms1": ["gcp1", "gcp2"]
        }
    },

Finally the capability to configure REST API defaults is present - note the router resides in the customer specific implementation code

    "apiDefaults": {
        "queryRowLimit": 10,
        "authEnabled": false,
        "corsEnabled": true,
        "refdataDir": "./refdataSeed"
    },