Add ClickhouseBackup Plugin Support#1798
Draft
jitendra1411 wants to merge 1 commit into
Draft
Conversation
Signed-off-by: jitendra1411 <jackrh2017@gmail.com>
|
Will this get accepted. |
Contributor
|
Would be really cool to see this getting merged |
discostur
added a commit
to discostur/clickhouse-operator
that referenced
this pull request
Jun 29, 2026
Introduces operator-managed backup and restore for ClickHouse using clickhouse-backup, exposed through three new custom resources in the clickhouse.altinity.com/v1 API group: - ClickHouseBackup (chb): one-off backup -> Kubernetes Job - ClickHouseBackupSchedule (chbs): recurring backup -> managed CronJob - ClickHouseRestore (chr): one-off restore -> Kubernetes Job The controllers follow the existing ClickHouseKeeper controller-runtime pattern. clickhouse-backup runs as a sidecar (a documented prerequisite); the generated jobs trigger it remotely through the system.backup_actions integration table, so no backup logic is reimplemented in the operator. Cluster-aware: backs up one replica per shard for Replicated* tables (AllReplicas opt-in for non-replicated data); on restore it applies the schema on all replicas and the data on the first replica of each shard, letting native replication synchronize the rest. Restore safety follows the conventions of mature DB operators: preflight validation (target CHI Completed, topology reachable) and an overwrite guard that refuses a non-empty target unless overwrite=true. Includes the CRDs, RBAC (incl. batch jobs/cronjobs), regenerated install bundles and Helm chart, documentation and examples, Go unit tests and a TestFlows e2e test. Refs Altinity#1795, Altinity#862. Supersedes the gRPC-plugin approach of Altinity#1798. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Kilian Ries <mail@kilian-ries.de>
discostur
added a commit
to discostur/clickhouse-operator
that referenced
this pull request
Jun 30, 2026
Introduces operator-managed backup and restore for ClickHouse using clickhouse-backup, exposed through three new custom resources in the clickhouse.altinity.com/v1 API group: - ClickHouseBackup (chb): one-off backup -> Kubernetes Job - ClickHouseBackupSchedule (chbs): recurring backup -> managed CronJob - ClickHouseRestore (chr): one-off restore -> Kubernetes Job The controllers follow the existing ClickHouseKeeper controller-runtime pattern. clickhouse-backup runs as a sidecar (a documented prerequisite); the generated jobs trigger it remotely through the system.backup_actions integration table, so no backup logic is reimplemented in the operator. Cluster-aware: backs up one replica per shard for Replicated* tables (AllReplicas opt-in for non-replicated data); on restore it applies the schema on the first replica per shard via ON CLUSTER (requires the sidecar's restore_schema_on_cluster) and the data on the first replica, letting native replication synchronize the remaining replicas. Restore safety follows the conventions of mature DB operators: preflight validation (target CHI Completed, topology reachable) and an overwrite guard that refuses a non-empty target unless overwrite=true. Also adds: selective (tables/partitions) and incremental (--diff-from-remote) backups; remote-backup retention (keepLastRemote); optional post-backup verification; Prometheus metrics on the operator's existing :9999 endpoint plus Kubernetes Events; and annotation-driven bootstrap-from-backup for new installations. Compression and encryption are documented as clickhouse-backup sidecar settings. Includes the CRDs, RBAC (incl. batch jobs/cronjobs), regenerated install bundles and Helm chart, documentation and examples, Go unit tests and a TestFlows e2e test. Refs Altinity#1795, Altinity#862. Supersedes the gRPC-plugin approach of Altinity#1798. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Kilian Ries <mail@kilian-ries.de>
discostur
added a commit
to discostur/clickhouse-operator
that referenced
this pull request
Jun 30, 2026
Introduces operator-managed backup and restore for ClickHouse using clickhouse-backup, exposed through three new custom resources in the clickhouse.altinity.com/v1 API group: - ClickHouseBackup (chb): one-off backup -> Kubernetes Job - ClickHouseBackupSchedule (chbs): recurring backup -> managed CronJob - ClickHouseRestore (chr): one-off restore -> Kubernetes Job The controllers follow the existing ClickHouseKeeper controller-runtime pattern. clickhouse-backup runs as a sidecar (a documented prerequisite); the generated jobs trigger it remotely through the system.backup_actions integration table, so no backup logic is reimplemented in the operator. Cluster-aware: backs up one replica per shard for Replicated* tables (AllReplicas opt-in for non-replicated data); on restore it applies the schema on the first replica per shard via ON CLUSTER (requires the sidecar's restore_schema_on_cluster) and the data on the first replica, letting native replication synchronize the remaining replicas. Restore safety follows the conventions of mature DB operators: preflight validation (target CHI Completed, topology reachable) and an overwrite guard that refuses a non-empty target unless overwrite=true. Also adds: selective (tables/partitions) and incremental (--diff-from-remote) backups; remote-backup retention (keepLastRemote); optional post-backup verification; Prometheus metrics on the operator's existing :9999 endpoint plus Kubernetes Events; and annotation-driven bootstrap-from-backup for new installations. Compression and encryption are documented as clickhouse-backup sidecar settings. Includes the CRDs, RBAC (incl. batch jobs/cronjobs), regenerated install bundles and Helm chart, documentation and examples, Go unit tests and a TestFlows e2e test. Refs Altinity#1795, Altinity#862. Supersedes the gRPC-plugin approach of Altinity#1798. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Kilian Ries <mail@kilian-ries.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR introduces a new Backup plugin architecture to the ClickHouse Operator, enabling automated backup operations via an external plugin service. Issue: #1795
Current PR Includes
1. Operator Enhancements
CRDs
Controller Updates
2. Go Plugin Skeleton
BackupRequestandBackupResultmessages.Motivation & Benefits
TODO
Testing Notes