flink-operator

command module
v0.9.9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 13, 2023 License: Apache-2.0 Imports: 12 Imported by: 0

README

This is not an officially supported Google product.

Kubernetes Operator for Apache Flink is a control plane for running Apache Flink on Kubernetes.

Community

Project Status

Beta

The operator is under active development, backward compatibility of the APIs is not guaranteed for beta releases.

Prerequisites

  • Version >= 1.15 of Kubernetes
  • Version >= 1.15 of kubectl (with kustomize)
  • Version >= 1.7 of Apache Flink

Overview

The Kubernetes Operator for Apache Flink extends the vocabulary (e.g., Pod, Service, etc) of the Kubernetes language with custom resource definition FlinkCluster and runs a controller Pod to keep watching the custom resources. Once a FlinkCluster custom resource is created and detected by the controller, the controller creates the underlying Kubernetes resources (e.g., JobManager Pod) based on the spec of the custom resource. With the operator installed in a cluster, users can then talk to the cluster through the Kubernetes API and Flink custom resources to manage their Flink clusters and jobs.

Features

  • Support for both Flink job cluster and session cluster depending on whether a job spec is provided
  • Custom Flink images
  • Flink and Hadoop configs and container environment variables
  • Init containers and sidecar containers
  • Remote job jar
  • Configurable namespace to run the operator in
  • Configurable namespace to watch custom resources in
  • Configurable access scope for JobManager service
  • Taking savepoints periodically
  • Taking savepoints on demand
  • Restarting failed job from the latest savepoint automatically
  • Cancelling job with savepoint
  • Cleanup policy on job success and failure
  • Updating cluster or job
  • Batch scheduling for JobManager and TaskManager Pods
  • GCP integration (service account, GCS connector, networking)
  • Support for Beam Python jobs

Installation

The operator is still under active development, there is no Helm chart available yet. You can follow either

  • User Guide to deploy a released operator image on gcr.io/flink-operator to your Kubernetes cluster or
  • Developer Guide to build an operator image first then deploy it to the cluster.

Documentation

Quickstart guides
API
How to
Tech talks
  • CNCF Webinar: Apache Flink on Kubernetes Operator (video, slides)

Contributing

Please check CONTRIBUTING.md and the Developer Guide out.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
api
v1alpha1
Package v1alpha1 contains API Schema definitions for the flinkoperator v1alpha1 API group +kubebuilder:object:generate=true +groupName=flinkoperator.streamnative.io
Package v1alpha1 contains API Schema definitions for the flinkoperator v1alpha1 API group +kubebuilder:object:generate=true +groupName=flinkoperator.streamnative.io
v1beta1
Package v1beta1 contains API Schema definitions for the flinkoperator v1beta1 API group +kubebuilder:object:generate=true +groupName=flinkoperator.streamnative.io
Package v1beta1 contains API Schema definitions for the flinkoperator v1beta1 API group +kubebuilder:object:generate=true +groupName=flinkoperator.streamnative.io

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL