ceph-objectstore-broker

Service Broker for Ceph Object Storage

This broker is compliant with V2 of the Open Service Broker API. The broker provides access to Ceph's object storage, and can currently be deployed as CloudFoundry app, on Kubernetes or on OpenShift. Deployment as a Bosh release is planned for the future.

Table of Contents

• General Operation
• Deployment
  • Prerequisites
  • CloudFoundry
  • Kubernetes & OpenShift
  • Bosh Release
• Integration Tests

General Operation

The service provided by the broker and its plans are in the brokerConfig/service-config.json file. You can edit this to your liking before deploying.

When an instance is provisioned a user is created on Ceph. Then when an application binds to the broker, it returns access credentials for both the S3 and Swift APIs supported by Ceph.

The credentials made available to the application (usually through environment variables) after a bind are:-

• s3User
• s3AccessKey
• s3SecretKey
• s3Endpoint
• swiftUser
• swiftSecretKey
• swiftEndpoint

Unbinding and deprovisioning are simply reverse operations of the provision and bind stages.

Deployment

Deployment to all platforms is done through the deploy.sh file, so once prerequisites for a platform are fulfilled the script can be used to deploy the broker.

Prerequisites

Before deploying to a platform, you need to provide the required details about your Ceph installation. Specifically you will need a Ceph object gateway setup. The broker will use the admin user on the gateway to manage users there as required to operate the service, and so it requires a number of variables including the gateway's endpoint and access keys for the user.

To provide the required information you will need a file called vars-file.yml. A template for this file called vars-file-template.yml is available, and so can simply be copied, renamed and then the details filled in.

Lastly, you will need Go installed as its used in the deployment script and in case you want to build yourself or run the integration tests. The broker has been developed with Go V1.10.1. It should theoretically work with older releases, but keep in mind that is not verified.

CloudFoundry

Deployment of the broker as an app running on CloudFoundry is controlled by the manifest.yml file, which requires no edits. To deploy simply run ./deploy.sh cf ceph-objectstorage-broker, with the second argument being the name of the app on CF.

Once the broker is running on CF, it needs to be registered with CF and then the plans need to be made public. To register the broker use cf create-service-broker SERVICE_BROKER BROKER_USERNAME BROKER_PASSWORD BROKER_URL. Then to make the service public run cf enable-service-access ceph-object-storage, where 'ceph-object-storage' is the name of the service provided in brokerConfig/service-config.json.

Kubernetes & OpenShift

Deployment to k8s and OS are both done by using the following files:

• Automatically created/updated using your vars-file.yml via the update-cosb-vars/update-vars.go GO program, which is run on each deploy
  • config-map.yml
  • secret.yml
• template.yml
• route.yml (only for OS)
• broker.yml (Manually used to register after deployment)

Before you deploy, please make sure you have kubectl or oc installed and that you are logged in to your cluster, as they are used to deploy to k8s and OS, respectively.

To deploy use ./deploy.sh k8s or ./deploy.sh os. These commands will set the config-map, secret, deploy the broker application and then create a service for it. In the case of OS, it also creates a route for the broker and displays the url of the created route.

The default service created uses a NodePort to expose the broker, however depending on your platform you might want to use something like a loadbalancer, in which case you can just edit the relevant yaml files and then use the deployment script to deploy with your own configuration.

To register the broker you need to get the url of your broker (it could be deployed on a different platform), any certificates if you want encryption and then update the deployment-configs/k8s/broker.yml. If you don't use encryption then you simply need to set the url field. Once you have updated the broker file you can run oc apply -f "deployment-configs/k8s/broker.yml" or kubectl apply -f "deployment-configs/k8s/broker.yml", depending if you are using OpenShift or Kubernetes, respectively.

NOTE: To apply the broker file you need to have the Service Catalog installed on your Kubernetes cluster and be a user with sufficient privileges (e.g. system:admin on OpenShift).

Bosh Release

The BOSH release for the broker and related documentation can be found here.

Integration Tests

To run the tests:

1. Fulfill the required prerequisites
2. Run go run update-cosb-vars/update-vars.go
3. Run source tests/tests.env
4. Run go run main.go
5. In the tests folder run go test or go test -v for more details