cloudnative-pg

module
v1.23.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 30, 2024 License: Apache-2.0

README

CNCF Landscape Latest Release GitHub License Documentation Stack Overflow

Welcome to the CloudNativePG project!

CloudNativePG is a comprehensive open source platform designed to seamlessly manage PostgreSQL databases within Kubernetes environments, covering the entire operational lifecycle from initial deployment to ongoing maintenance. The main component is the CloudNativePG operator.

CloudNativePG was originally built and sponsored by EDB.

Table of content

Getting Started

The best way to get started is with the "Quickstart" section in the documentation.

Scope

The goal of CloudNativePG is to increase the adoption of PostgreSQL, one of the most loved DBMS in traditional VM and bare metal environments, inside Kubernetes, thus making the database an integral part of the development process and GitOps CI/CD automated pipelines.

In scope

CloudNativePG has been designed by Postgres experts with Kubernetes administrators in mind. Put simply, it leverages Kubernetes by extending its controller and by defining, in a programmatic way, all the actions that a good DBA would normally do when managing a highly available PostgreSQL database cluster.

Since the inception, our philosophy has been to adopt a Kubernetes native approach to PostgreSQL cluster management, making incremental decisions that would answer the fundamental question: "What would a Kubernetes user expect from a Postgres operator?".

The most important decision we made is to have the status of a PostgreSQL cluster directly available in the Cluster resource, so to inspect it through the Kubernetes API. We've fully embraced the operator pattern and eventual consistency, two of the core principles upon which Kubernetes is built for managing complex applications.

As a result, the operator is responsible for managing the status of the Cluster resource, keeping it up to date with the information that each PostgreSQL instance manager regularly reports back through the API server. Changes to the cluster status might trigger, for example, actions like:

  • a PostgreSQL failover where, after an unexpected failure of a cluster's primary instance, the operator itself elects the new primary, updates the status, and directly coordinates the operation through the reconciliation loop, by relying on the instance managers

  • scaling up or down the number of read-only replicas, based on a positive or negative variation in the number of desired instances in the cluster, so that the operator creates or removes the required resources to run PostgreSQL, such as persistent volumes, persistent volume claims, pods, secrets, config maps, and then coordinates cloning and streaming replication tasks

  • updates of the endpoints of the PostgreSQL services that applications rely on to interact with the database, as Kubernetes represents the single source of truth and authority

  • updates of container images in a rolling fashion, following a change in the image name, by first updating the pods where replicas are running, and then the primary, issuing a switchover first

The latter example is based on another pillar of CloudNativePG: immutable application containers - as explained in the blog article "Why EDB Chose Immutable Application Containers".

The above list can be extended. However, the gist is that CloudNativePG exclusively relies on the Kubernetes API server and the instance manager to coordinate the complex operations that need to take place in a business continuity PostgreSQL cluster, without requiring any assistance from an intermediate management tool responsible for high availability and failover management like similar open source operators.

CloudNativePG also manages additional resources to help the Cluster resource manage PostgreSQL - currently Backup, ClusterImageCatalog, ImageCatalog, Pooler, and ScheduledBackup.

Fully embracing Kubernetes means adopting a hands-off approach during temporary failures of the Kubernetes API server. In such instances, the operator refrains from taking action, deferring decisions until the API server is operational again. Meanwhile, Postgres instances persist, maintaining operations based on the latest known state of the cluster.

Out of scope

CloudNativePG is exclusively focused on the PostgreSQL database management system maintained by the PostgreSQL Global Development Group (PGDG). We are not currently considering adding to CloudNativePG extensions or capabilities that are included in forks of the PostgreSQL database management system, unless in the form of extensible or pluggable frameworks.

CloudNativePG doesn't intend to pursue database independence (e.g. control a MariaDB cluster).

Communications

Resources

Adopters

A list of publicly known users of the CloudNativePG operator is in ADOPTERS.md. Help us grow our community and CloudNativePG by adding yourself and your organization to this list!

Maintainers

The current maintainers of the CloudNativePG project are:

  • Gabriele Bartolini (EDB)
  • Francesco Canovai (EDB)
  • Leonardo Cecchi (EDB)
  • Jonathan Gonzalez (EDB)
  • Marco Nenciarini (EDB)
  • Armando Ruocco (EDB)
  • Philippe Scorsolini (Upbound)

They are listed in the CODEOWNERS file.

CloudNativePG at KubeCon

Star History

Star History Chart

Trademarks

Postgres, PostgreSQL and the Slonik Logo are trademarks or registered trademarks of the PostgreSQL Community Association of Canada, and used with their permission.

Directories

Path Synopsis
api
v1
Package v1 contains API Schema definitions for the postgresql v1 API group +kubebuilder:object:generate=true +groupName=postgresql.cnpg.io
 Package v1 contains API Schema definitions for the postgresql v1 API group +kubebuilder:object:generate=true +groupName=postgresql.cnpg.io
cmd
kubectl-cnpg
kubectl-cnp is a plugin to manage your CloudNativePG clusters
 kubectl-cnp is a plugin to manage your CloudNativePG clusters
manager
The manager command is the main entrypoint of CloudNativePG operator.
 The manager command is the main entrypoint of CloudNativePG operator.
Package controllers contains the controller of the CRD
 Package controllers contains the controller of the CRD
internal
cmd/manager/backup
Package backup implement the "controller backup" command
 Package backup implement the "controller backup" command
cmd/manager/bootstrap
Package bootstrap implement the "controller bootstrap" command
 Package bootstrap implement the "controller bootstrap" command
cmd/manager/controller
Package controller implement the command used to start the operator
 Package controller implement the command used to start the operator
cmd/manager/debug
Package debug implement the debug command subfeatures
 Package debug implement the debug command subfeatures
cmd/manager/debug/architectures
Package architectures implement the show-architectures command
 Package architectures implement the show-architectures command
cmd/manager/instance
Package instance implements the "instance" subcommand of the operator
 Package instance implements the "instance" subcommand of the operator
cmd/manager/instance/initdb
Package initdb implements the "instance init" subcommand of the operator
 Package initdb implements the "instance init" subcommand of the operator
cmd/manager/instance/join
Package join implements the "instance join" subcommand of the operator
 Package join implements the "instance join" subcommand of the operator
cmd/manager/instance/pgbasebackup
Package pgbasebackup implement the pgbasebackup bootstrap method
 Package pgbasebackup implement the pgbasebackup bootstrap method
cmd/manager/instance/restore
Package restore implements the "instance restore" subcommand of the operator
 Package restore implements the "instance restore" subcommand of the operator
cmd/manager/instance/restoresnapshot
Package restoresnapshot implements the "instance restoresnapshot" subcommand of the operator
 Package restoresnapshot implements the "instance restoresnapshot" subcommand of the operator
cmd/manager/instance/run
Package run implements the "instance run" subcommand of the operator
 Package run implements the "instance run" subcommand of the operator
cmd/manager/instance/run/lifecycle
Package lifecycle contains the code to run and manage the lifecycle of a postgres Instance
 Package lifecycle contains the code to run and manage the lifecycle of a postgres Instance
cmd/manager/instance/status
Package status implement the "instance status" subcommand of the operator
 Package status implement the "instance status" subcommand of the operator
cmd/manager/pgbouncer
Package pgbouncer implements the "pgbouncer" subcommand of the operator
 Package pgbouncer implements the "pgbouncer" subcommand of the operator
cmd/manager/pgbouncer/run
Package run implements the "pgbouncer run" subcommand of the operator
 Package run implements the "pgbouncer run" subcommand of the operator
cmd/manager/show
Package show implement the show command subfeatures
 Package show implement the show command subfeatures
cmd/manager/show/walarchivequeue
Package walarchivequeue implement the wal-archive-queue command
 Package walarchivequeue implement the wal-archive-queue command
cmd/manager/walarchive
Package walarchive implement the wal-archive command
 Package walarchive implement the wal-archive command
cmd/manager/walrestore
Package walrestore implement the walrestore command
 Package walrestore implement the walrestore command
cmd/plugin
Package plugin contains the common behaviors of the kubectl-cnpg subcommand
 Package plugin contains the common behaviors of the kubectl-cnpg subcommand
cmd/plugin/backup
Package backup implements a command to request an on-demand backup for a PostgreSQL cluster
 Package backup implements a command to request an on-demand backup for a PostgreSQL cluster
cmd/plugin/certificate
Package certificate implement the kubectl-cnpg certificate command
 Package certificate implement the kubectl-cnpg certificate command
cmd/plugin/destroy
Package destroy implements a command to destroy an instances of a cluster and its associated PVC
 Package destroy implements a command to destroy an instances of a cluster and its associated PVC
cmd/plugin/fence
Package fence implements a command to fence instances in a cluster
 Package fence implements a command to fence instances in a cluster
cmd/plugin/fio
Package fio implements the fio job creation via deployment
 Package fio implements the fio job creation via deployment
cmd/plugin/hibernate
Package hibernate implements the hibernation feature
 Package hibernate implements the hibernation feature
cmd/plugin/install
Package install implements the install plugin command
 Package install implements the install plugin command
cmd/plugin/logical
Package logical contains the common features of the publication/subscription management
 Package logical contains the common features of the publication/subscription management
cmd/plugin/logical/publication
Package publication contains the implementation of the kubectl cnpg publication command
 Package publication contains the implementation of the kubectl cnpg publication command
cmd/plugin/logical/publication/create
Package create contains the implementation of the kubectl cnpg publication create command
 Package create contains the implementation of the kubectl cnpg publication create command
cmd/plugin/logical/publication/drop
Package drop contains the implementation of the kubectl cnpg publication drop command
 Package drop contains the implementation of the kubectl cnpg publication drop command
cmd/plugin/logical/subscription
Package subscription contains the implementation of the kubectl cnpg subscription command
 Package subscription contains the implementation of the kubectl cnpg subscription command
cmd/plugin/logical/subscription/create
Package create contains the implementation of the kubectl cnpg subscription create command
 Package create contains the implementation of the kubectl cnpg subscription create command
cmd/plugin/logical/subscription/drop
Package drop contains the implementatoin of the cnpg subscription drop command
 Package drop contains the implementatoin of the cnpg subscription drop command
cmd/plugin/logical/subscription/syncsequences
Package syncsequences contains the implementation of the kubectl cnpg subscription sync-sequences command
 Package syncsequences contains the implementation of the kubectl cnpg subscription sync-sequences command
cmd/plugin/logs
Package logs implements the kubectl-cnpg logs command
 Package logs implements the kubectl-cnpg logs command
cmd/plugin/maintenance
Package maintenance implements the kubectl-cnpg maintenance sub-command
 Package maintenance implements the kubectl-cnpg maintenance sub-command
cmd/plugin/pgadmin
Package pgadmin implements the creation of a pgadmin deployment
 Package pgadmin implements the creation of a pgadmin deployment
cmd/plugin/pgbench
Package pgbench implements the pgbench job creation
 Package pgbench implements the pgbench job creation
cmd/plugin/promote
Package promote implement the kubectl-cnpg promote command
 Package promote implement the kubectl-cnpg promote command
cmd/plugin/psql
Package psql implements the `kubectl cnpg psql` command
 Package psql implements the `kubectl cnpg psql` command
cmd/plugin/reload
Package reload implements a command to trigger a reconciliation loop for a cluster
 Package reload implements a command to trigger a reconciliation loop for a cluster
cmd/plugin/report
Package report implements the kubectl-cnpg report command
 Package report implements the kubectl-cnpg report command
cmd/plugin/restart
Package restart implements a command to rollout restart a cluster or restart a single instance
 Package restart implements a command to rollout restart a cluster or restart a single instance
cmd/plugin/snapshot
Package snapshot implements the snapshot feature
 Package snapshot implements the snapshot feature
cmd/plugin/status
Package status implements the kubectl-cnpg status command
 Package status implements the kubectl-cnpg status command
cmd/versions
Package versions builds the version subcommand for both manager and plugins
 Package versions builds the version subcommand for both manager and plugins
cnpi/plugin
Package plugin contains the logics that acts as bridge between cnpg-i and the operator
 Package plugin contains the logics that acts as bridge between cnpg-i and the operator
cnpi/plugin/client
Package client contains a set of helper structures for CNPG to use the plugins exposing the CNPI interface
 Package client contains a set of helper structures for CNPG to use the plugins exposing the CNPI interface
cnpi/plugin/operatorclient
Package operatorclient contains an extended kubernetes client that supports plugin API calls
 Package operatorclient contains an extended kubernetes client that supports plugin API calls
configuration
Package configuration contains the configuration of the operator, reading if from environment variables and from the ConfigMap
 Package configuration contains the configuration of the operator, reading if from environment variables and from the ConfigMap
management/cache
Package cache contains the constants and functions for reading/writing to the process local cache some specific supported objects
 Package cache contains the constants and functions for reading/writing to the process local cache some specific supported objects
management/cache/client
Package client contains the constants and functions for reading supported objects from cache or building them in case of cache miss.
 Package client contains the constants and functions for reading supported objects from cache or building them in case of cache miss.
management/controller
Package controller contains the functions in PostgreSQL instance manager that reacts to changes to the Cluster resource.
 Package controller contains the functions in PostgreSQL instance manager that reacts to changes to the Cluster resource.
management/controller/externalservers
Package externalservers contains the reconciler of external servers, taking care of enabling connectivity to every server defined in the relative section of the CR
 Package externalservers contains the reconciler of external servers, taking care of enabling connectivity to every server defined in the relative section of the CR
management/controller/roles
Package roles contains the code needed to reconcile roles with PostgreSQL
 Package roles contains the code needed to reconcile roles with PostgreSQL
management/controller/slots/infrastructure
Package infrastructure contains the structs and interfaces needed to manage replication slots
 Package infrastructure contains the structs and interfaces needed to manage replication slots
management/controller/slots/reconciler
Package reconciler contains all the logic needed to reconcile replication slots
 Package reconciler contains all the logic needed to reconcile replication slots
management/controller/slots/runner
Package runner contains the runner that replicates slots from the primary to the replicas
 Package runner contains the runner that replicates slots from the primary to the replicas
management/controller/tablespaces
Package tablespaces contains the runner to declarative tablespace
 Package tablespaces contains the runner to declarative tablespace
management/controller/tablespaces/infrastructure
Package infrastructure contains the structs and interfaces needed to manage declarative tablespace
 Package infrastructure contains the structs and interfaces needed to manage declarative tablespace
management/istio
Package istio implements functions needed to integrate with istio-proxy
 Package istio implements functions needed to integrate with istio-proxy
management/linkerd
Package linkerd implements functions needed to integrate with linkerd-proxy
 Package linkerd implements functions needed to integrate with linkerd-proxy
management/utils
Package utils contains uncategorized utilities only used by the instance manager of PostgreSQL and PgBouncer
 Package utils contains uncategorized utilities only used by the instance manager of PostgreSQL and PgBouncer
pgbouncer/management/controller
Package controller contains the functions in pgbouncer instance manager that reacts to changes in the Pooler resource.
 Package controller contains the functions in pgbouncer instance manager that reacts to changes in the Pooler resource.
plugin/resources
Package resources contains reusable functions for the plugin commands
 Package resources contains reusable functions for the plugin commands
scheme
Package scheme offers a builder capable of generating a scheme with the resources known by the CNP manager
 Package scheme offers a builder capable of generating a scheme with the resources known by the CNP manager
pkg
certs
Package certs handle the PKI infrastructure of the operator
 Package certs handle the PKI infrastructure of the operator
cnpgerrors
Package cnpgerrors contains error types for CNPG
 Package cnpgerrors contains error types for CNPG
concurrency
Package concurrency contains utilities for goroutines coordination
 Package concurrency contains utilities for goroutines coordination
conditions
Package conditions contains functions useful to update the conditions on the resources managed by the operator
 Package conditions contains functions useful to update the conditions on the resources managed by the operator
configfile
Package configfile contains primitives needed to manage a configuration file with the syntax of PostgreSQL
 Package configfile contains primitives needed to manage a configuration file with the syntax of PostgreSQL
configparser
Package configparser contains the code required to fill a Go structure representing the configuration information from several sources like:
 Package configparser contains the code required to fill a Go structure representing the configuration information from several sources like:
executablehash
Package executablehash detect the SHA256 of the running binary
 Package executablehash detect the SHA256 of the running binary
fileutils
Package fileutils contains the utility functions about file management
 Package fileutils contains the utility functions about file management
fileutils/compatibility
Package compatibility provides a layer to cross-compile with other OS than Linux
 Package compatibility provides a layer to cross-compile with other OS than Linux
management
Package management contains all the features needed by the instance manager that runs in each Pod as PID 1
 Package management contains all the features needed by the instance manager that runs in each Pod as PID 1
management/barman
Package barman contains the utilities to interact with barman-cloud.
 Package barman contains the utilities to interact with barman-cloud.
management/barman/archiver
Package archiver manages the WAL archiving process
 Package archiver manages the WAL archiving process
management/barman/capabilities
Package capabilities stores the definition of the type for Barman capabilities
 Package capabilities stores the definition of the type for Barman capabilities
management/barman/credentials
Package credentials is used to build environment for barman cloud commands
 Package credentials is used to build environment for barman cloud commands
management/barman/restorer
Package restorer manages the WAL restore process
 Package restorer manages the WAL restore process
management/barman/spool
Package spool implements a WAL pooler keeping track of which WALs we have archived
 Package spool implements a WAL pooler keeping track of which WALs we have archived
management/catalog
Package catalog is the implementation of a backup catalog
 Package catalog is the implementation of a backup catalog
management/execlog
Package execlog handles stdout and stderr pipes of started commands and logs them in JSON using the provided logger
 Package execlog handles stdout and stderr pipes of started commands and logs them in JSON using the provided logger
management/external
Package external contains the functions needed to manage servers which are external to this PostgreSQL cluster
 Package external contains the functions needed to manage servers which are external to this PostgreSQL cluster
management/external/internal/pgpass
Package pgpass allows the user to generate a PostgreSQL .pgpass file See: https://www.postgresql.org/docs/current/libpq-pgpass.html
 Package pgpass allows the user to generate a PostgreSQL .pgpass file See: https://www.postgresql.org/docs/current/libpq-pgpass.html
management/log
Package log contains the logging subsystem of the instance manager
 Package log contains the logging subsystem of the instance manager
management/logtest
Package logtest contains the testing utils for the logging subsystem of the instance manager
 Package logtest contains the testing utils for the logging subsystem of the instance manager
management/pgbouncer/config
Package config contains the code related to the generation of the PgBouncer configuration
 Package config contains the code related to the generation of the PgBouncer configuration
management/pgbouncer/metricsserver
Package metricsserver contains the web server powering metrics
 Package metricsserver contains the web server powering metrics
management/postgres
Package postgres contains the function about starting up, shutting down and managing a PostgreSQL instance.
 Package postgres contains the function about starting up, shutting down and managing a PostgreSQL instance.
management/postgres/constants
Package constants provides the needed constants in the postgres package
 Package constants provides the needed constants in the postgres package
management/postgres/logicalimport
Package logicalimport contains the logic needed to import a logical snapshot
 Package logicalimport contains the logic needed to import a logical snapshot
management/postgres/logpipe
Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.
 Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.
management/postgres/metrics
Package metrics enables to expose a set of metrics and collectors on a given postgres instance
 Package metrics enables to expose a set of metrics and collectors on a given postgres instance
management/postgres/metrics/histogram
Package histogram contain histogram-metrics related functions
 Package histogram contain histogram-metrics related functions
management/postgres/pool
Package pool contain an implementation of a connection pool to multiple database pointing to the same instance
 Package pool contain an implementation of a connection pool to multiple database pointing to the same instance
management/postgres/utils
Package utils holds generic utils about postgres instances
 Package utils holds generic utils about postgres instances
management/postgres/webserver
Package webserver contains the web server powering probes, backups and metrics
 Package webserver contains the web server powering probes, backups and metrics
management/postgres/webserver/metricserver
Package metricserver contains the web server powering metrics
 Package metricserver contains the web server powering metrics
management/upgrade
Package upgrade manages the in-place upgrade of the instance manager
 Package upgrade manages the in-place upgrade of the instance manager
management/url
Package url holds the constants for webserver routing
 Package url holds the constants for webserver routing
multicache
Package multicache implements a cache that is able to work on multiple namespaces but also able to read data from a namespace which is beside the specified ones.
 Package multicache implements a cache that is able to work on multiple namespaces but also able to read data from a namespace which is beside the specified ones.
podspec
Package podspec contains various utilities to deal with Pod Specs
 Package podspec contains various utilities to deal with Pod Specs
postgres
Package postgres contains the function covering the PostgreSQL integrations and the relative data types
 Package postgres contains the function covering the PostgreSQL integrations and the relative data types
reconciler/backup/volumesnapshot
Package volumesnapshot contains the VolumeSnapshot reconciler that is used to take VolumeSnapshot backups
 Package volumesnapshot contains the VolumeSnapshot reconciler that is used to take VolumeSnapshot backups
reconciler/hibernation
Package hibernation contains all the logic to hibernate a CNPG cluster
 Package hibernation contains all the logic to hibernate a CNPG cluster
reconciler/instance
Package instance contains all the logic to reconcile an instance pod
 Package instance contains all the logic to reconcile an instance pod
reconciler/persistentvolumeclaim
Package persistentvolumeclaim contains all the logic to reconcile and build PVCS
 Package persistentvolumeclaim contains all the logic to reconcile and build PVCS
reconciler/replicaclusterswitch
Package replicaclusterswitch contains the logic needed to turn on the replica cluster feature on an existing cluster
 Package replicaclusterswitch contains the logic needed to turn on the replica cluster feature on an existing cluster
resources
Package resources contains a set of Kubernetes generic utilities that are used by the operator
 Package resources contains a set of Kubernetes generic utilities that are used by the operator
resources/instance
Package instance contains the client capable of querying the HTTP instances endpoints
 Package instance contains the client capable of querying the HTTP instances endpoints
servicespec
Package servicespec contains various utilities to deal with Service Specs
 Package servicespec contains various utilities to deal with Service Specs
specs
Package specs contains the specification of the K8s resources generated by the CloudNativePG operator
 Package specs contains the specification of the K8s resources generated by the CloudNativePG operator
specs/pgbouncer
Package pgbouncer contains the specification of the K8s resources generated by the CloudNativePG operator related to pgbouncer poolers
 Package pgbouncer contains the specification of the K8s resources generated by the CloudNativePG operator related to pgbouncer poolers
stringset
Package stringset implements a basic set of strings
 Package stringset implements a basic set of strings
system
Package system provides an interface with the operating system
 Package system provides an interface with the operating system
system/compatibility
Package compatibility provides a layer to cross-compile with other OS than Linux
 Package compatibility provides a layer to cross-compile with other OS than Linux
utils
Package utils contains otherwise uncategorized kubernetes relative functions
 Package utils contains otherwise uncategorized kubernetes relative functions
utils/hash
Package hash allows the user to get a hash number for a given Kubernetes object.
 Package hash allows the user to get a hash number for a given Kubernetes object.
utils/logs
Package logs contains code to fetch logs from Kubernetes pods
 Package logs contains code to fetch logs from Kubernetes pods
versions
Package versions contains the version of the CloudNativePG operator and the software that is used by it
 Package versions contains the version of the CloudNativePG operator and the software that is used by it
Package releases contains the filesystem with operator manifests with all the released manifests
 Package releases contains the filesystem with operator manifests with all the released manifests
Package tests contains the test infrastructure of the CloudNativePG operator
 Package tests contains the test infrastructure of the CloudNativePG operator
utils
Package utils contains helper functions/methods for e2e
 Package utils contains helper functions/methods for e2e
utils/nodes
Package nodes contains the helper methods/functions for nodes
 Package nodes contains the helper methods/functions for nodes

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.