Hub-of-Hubs GitOps
The GitOps component of Hub-of-Hubs.
Go to the Contributing guide to learn how to get involved.
Overview
The Hub-of-Hubs (HOH) GitOps component shares a volume (persistent storage) with a
modified version of the
multicloud-operators-subscription operator,
where the subscriptions-operator is responsible for syncing Git objects via the ACM Subscriptions mechanism,
while the HOH GitOps component watches the files and processes them to provide support for customized gitops / non-k8s gitops.
Disclaimers:
- The component was implemented to demonstrate the mechanism. It is not fully implemented and is not tested for scale:
- Syncers should support un-deploying non-k8s resources upon deleting a subscription (e.g., when a repo is found but its subscription is not).
- Supported non-k8s resource implementations should be completed (currently only supports "INSERT" but not update/delete).
- Optimizations such as parallelized storage-walking / parallelized & batched DB job handling can be applied.
Prerequisites
Deploying the Shared Volume
-
Set the GITOPS_NODE_HOSTNAME
to the hostname of the node (e.g., ip-10-0-136-193
) that the storage, HOH-gitops and the
customized operator will run on:
$ export GITOPS_NODE_HOSTNAME=$(kubectl get node --selector='node-role.kubernetes.io/worker' -o=jsonpath='{.items[0].metadata.labels.kubernetes\.io\/hostname}')
-
Run the following command to deploy the hoh-gitops-pv
PersistentVolume and the hoh-gitops-pv-claim
PersistentVolumeClaim
that claims it to your hub of hubs cluster:
envsubst < deploy/hub-of-hubs-gitops-pv.yaml | kubectl apply -f -
Deploying the customized Subscriptions Operator
Deploying the modified Subscription CRD
kubectl -n open-cluster-management apply -f deploy/customized-subscriptions-operator/apps.open-cluster-management.io_subscriptions_crd_v1.yaml
Creating the namespace for accessible Subscription CRs
kubectl create namespace hoh-subscriptions
Deploying the modified operator
The subscriptions operator deployment is managed by the ACM for Kubernetes Operator. To have the latter deploy the customized version, modify the "multicluster-operators-standalone-subscription" deployment
to that present in standalone-subscriptions-operator-deployment.yaml.
The modified code has small modifications of the upstream stable release of the operator in Open Cluster Management organization,
therefore it is forked to a personal Git.
- Set the
MODIFIED_OPERATOR_IMAGE
environment variable to hold the URL of the image:
$ export MODIFIED_OPERATOR_IMAGE=quay.io/maroonayoub/multicloud-operators-subscription@sha256:1c57e1e77ea3c929c7176681d5b64eca43354bbaf00aeb7f7ddb01d3c6d15ad0
- Patch the ACM for K8s operator:
kubectl -n open-cluster-management patch ClusterServiceVersion advanced-cluster-management.v2.4.2 --type=merge --patch "$(envsubst < deploy/customized-subscriptions-operator/operators-subscriptions-deployments-patch.yaml)"
Visit examples for example Subscription deployments / Git objects
Getting Started
Build and push the image to docker registry
-
Set the REGISTRY
environment variable to hold the name of your docker registry:
$ export REGISTRY=...
-
Set the IMAGE_TAG
environment variable to hold the required version of the image.
default value is latest
, so in that case no need to specify this variable:
$ export IMAGE_TAG=latest
-
Run make to build and push the image:
$ make push-images
Deploy on the hub of hubs
-
If the hub-of-hubs-database-transport-bridge-secret
does not exist:
-
Set the DATABASE_URL
according to the PostgreSQL URL format: postgres://YourUserName:YourURLEscapedPassword@YourHostname:5432/YourDatabaseName?sslmode=verify-full&pool_max_conns=50
.
Remember to URL-escape the password, you can do it in bash:
python -c "import sys, urllib as ul; print ul.quote_plus(sys.argv[1])" 'YourPassword'
-
Create a secret with your database url:
kubectl create secret generic hub-of-hubs-database-transport-bridge-secret -n open-cluster-management --from-literal=url=$DATABASE_URL
-
Set the REGISTRY
environment variable to hold the name of your docker registry:
$ export REGISTRY=...
-
Set the IMAGE
environment variable to hold the name of the image.
$ export IMAGE=$REGISTRY/$(basename $(pwd)):latest
-
Run the following command to give the hub-of-hubs-gitops
service account "privileged" security context constraint permissions:
oc adm policy add-scc-to-user privileged -z hub-of-hubs-gitops -n open-cluster-management
-
Run the following command to deploy the hub-of-hubs-gitops
to your hub of hubs cluster:
envsubst < deploy/hub-of-hubs-gitops.yaml.template | kubectl apply -f -
Cleanup from the hub of hubs
-
Run the following command to clean hub-of-hubs-gitops
from your hub of hubs cluster:
envsubst < deploy/hub-of-hubs-gitops.yaml.template | kubectl delete -f -
-
Run the following command to remove the "privileged" security context constraint permissions from hub-of-hubs-gitops
service account :
oc adm policy remove-scc-from-user privileged -z hub-of-hubs-gitops -n open-cluster-management
-
If you wish to revert the ACM for K8s operator's customization, run the following:
kubectl -n open-cluster-management patch ClusterServiceVersion advanced-cluster-management.v2.4.2 \
--type=merge --patch $(cat deploy/customized-subscriptions-operator/revert-operators-subscriptions-deployments-patch.yaml)
-
Finally, delete PV and PVC:
kubectl -n open-cluster-management delete -f deploy/hub-of-hubs-gitops-pv.yaml