ceph

package module
v0.16.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 1, 2023 License: MIT Imports: 0 Imported by: 0

README

go-ceph - Go bindings for Ceph APIs

Godoc license

Introduction

The go-ceph project is a collection of API bindings that support the use of native Ceph APIs, which are C language functions, in Go. These bindings make use of Go's cgo feature. There are three main Go sub-packages that make up go-ceph:

  • rados - exports functionality from Ceph's librados
  • rbd - exports functionality from Ceph's librbd
  • cephfs - exports functionality from Ceph's libcephfs
  • rgw/admin - interact with radosgw admin ops API

We aim to provide comprehensive support for the Ceph APIs over time. This includes both I/O related functions and management functions. If your project makes use of Ceph command line tools and is written in Go, you may be able to switch away from shelling out to the CLI and to these native function calls.

Installation

The code in go-ceph is purely a library module. Typically, one will import go-ceph in another Go based project. When building the code the native RADOS, RBD, & CephFS library and development headers are expected to be installed.

On debian based systems (apt) these may be:

libcephfs-dev librbd-dev librados-dev

On rpm based systems (dnf, yum, etc) these may be:

libcephfs-devel librbd-devel librados-devel

On MacOS you can use brew to install the libraries:

brew tap mulbc/ceph-client
brew install ceph-client

NOTE: CentOS users may want to use a CentOS Storage SIG repository to enable packages for a supported ceph version. Example: dnf -y install centos-release-ceph-pacific. (CentOS 7 users should use "yum" rather than "dnf")

To quickly test if one can build with go-ceph on your system, run:

go get github.com/amirhaa/go-ceph

Once compiled, code using go-ceph is expected to dynamically link to the Ceph libraries. These libraries must be available on the system where the go based binaries will be run. Our use of cgo and ceph libraries does not allow for fully static binaries.

go-ceph tries to support different Ceph versions. However some functions might only be available in recent versions, and others may be deprecated. In order to work with non-current versions of Ceph, it is required to pass build-tags to the go command line. A tag with the named Ceph release will enable/disable certain features of the go-ceph packages, and prevent warnings or compile problems. For example, to ensure you select the library features that match the "pacific" release, use:

go build -tags pacific ....
go test -tags pacific ....
Supported Ceph Versions
go-ceph version Supported Ceph Versions Deprecated Ceph Versions
v0.16.0 octopus, pacific† nautilus
v0.15.0 octopus, pacific nautilus
v0.14.0 octopus, pacific nautilus
v0.13.0 octopus, pacific nautilus
v0.12.0 octopus, pacific nautilus
v0.11.0 nautilus, octopus, pacific
v0.10.0 nautilus, octopus, pacific
v0.9.0 nautilus, octopus
v0.8.0 nautilus, octopus
v0.7.0 nautilus, octopus
v0.6.0 nautilus, octopus mimic
v0.5.0 nautilus, octopus luminous, mimic
v0.4.0 luminous, mimic, nautilus, octopus
v0.3.0 luminous, mimic, nautilus, octopus
v0.2.0 luminous, mimic, nautilus
(pre release) luminous, mimic (see note)

These tags affect what is supported at compile time. What version of the Ceph cluster the client libraries support, and vice versa, is determined entirely by what version of the Ceph C libraries go-ceph is compiled with.

† Preliminary support for Ceph Quincy is available in this release. Due to Quincy specific issues in our test suite some APIs can not be tested against Quincy at the moment. Thus, go-ceph should work with Quincy but we are not declaring it fully supported until all APIs can be tested on Quincy.

NOTE: Prior to 2020 the project did not make versioned releases. The ability to compile with a particular Ceph version before go-ceph v0.2.0 is not guaranteed.

Documentation

Detailed API documentation is available at https://pkg.go.dev/github.com/amirhaa/go-ceph.

Some API Hints and How-Tos are also available to quickly introduce how some of API calls work together.

Development

docker run --rm -it --net=host \
  --security-opt apparmor:unconfined \
  -v ${PWD}:/go/src/github.com/amirhaa/go-ceph:z \
  -v /home/nwatkins/src/ceph/build:/home/nwatkins/src/ceph/build:z \
  -e CEPH_CONF=/home/nwatkins/src/ceph/build/ceph.conf \
  ceph-golang

Run against a vstart.sh cluster without installing Ceph:

export CGO_CPPFLAGS="-I/ceph/src/include"
export CGO_LDFLAGS="-L/ceph/build/lib"
go build

Contributing

Contributions are welcome & greatly appreciated, every little bit helps. Make code changes via Github pull requests:

  • Fork the repo and create a topic branch for every feature/fix. Avoid making changes directly on master branch.
  • All incoming features should be accompanied with tests.
  • Make sure that you run go fmt before submitting a change set. Alternatively the Makefile has a flag for this, so you can call make fmt as well.
  • The integration tests can be run in a docker container, for this run:
make test-docker
Interactive "Office Hours"

The maintenance team plans to be available regularly for questions, comments, pings, etc for about an hour twice a week. The current schedule is:

  • 2:00pm EDT (currently 18:00 UTC) Mondays
  • 9:00am EDT (currently 13:00 UTC) Thursdays

We will use the #ceph-devel IRC channel

Documentation

Overview

Package ceph is the root of a set of packages that wrap the Ceph APIs.

Generally, this package only exists to host subsystem specific packages and contains no functions.

The "rados" sub-package wraps APIs that handle RADOS specific functions.

The "rbd" sub-package wraps APIs that handle RBD specific functions.

The "cephfs" sub-package wraps APIs that handle CephFS specific functions.

The "common" sub-package contains sub-packages related to implementing common interfaces and utilities shared across the above.

Consult the documentation for each package for additional details.

Directories

Path Synopsis
Package cephfs contains a set of wrappers around Ceph's libcephfs API.
Package cephfs contains a set of wrappers around Ceph's libcephfs API.
admin
Package admin is a convenience layer to support the administration of CephFS volumes, subvolumes, etc.
Package admin is a convenience layer to support the administration of CephFS volumes, subvolumes, etc.
common
admin/manager
Package manager from common/admin contains a set of APIs used to interact with and administer the Ceph manager (mgr).
Package manager from common/admin contains a set of APIs used to interact with and administer the Ceph manager (mgr).
admin/nfs
Package nfs from common/admin contains a set of APIs used to interact with and administer NFS support for ceph clusters.
Package nfs from common/admin contains a set of APIs used to interact with and administer NFS support for ceph clusters.
commands
Package commands provides types and utility functions that are used for interfacing with the JSON based command infrastructure in Ceph.
Package commands provides types and utility functions that are used for interfacing with the JSON based command infrastructure in Ceph.
log
Package log allows to enable go-ceph logging and integrate it with the logging of the go-ceph consuming code.
Package log allows to enable go-ceph logging and integrate it with the logging of the go-ceph consuming code.
internal
errutil
Package errutil provides common functions for dealing with error conditions for all ceph api wrappers.
Package errutil provides common functions for dealing with error conditions for all ceph api wrappers.
log
Package log is the internal package for go-ceph logging.
Package log is the internal package for go-ceph logging.
Package rados contains a set of wrappers around Ceph's librados API.
Package rados contains a set of wrappers around Ceph's librados API.
rbd
Package rbd contains a set of wrappers around Ceph's librbd API.
Package rbd contains a set of wrappers around Ceph's librbd API.
admin
Package admin is a convenience layer to support the administration of rbd volumes, snapshots, scheduling etc that is outside of the C api for librbd.
Package admin is a convenience layer to support the administration of rbd volumes, snapshots, scheduling etc that is outside of the C api for librbd.
rgw
Package rgw contains sub-packages pertaining to the administration of RGW.
Package rgw contains sub-packages pertaining to the administration of RGW.
admin
Package admin contains a set of wrappers around Ceph's RGW Admin Ops API.
Package admin contains a set of wrappers around Ceph's RGW Admin Ops API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL