Commit Graph
3942 Commits
Author SHA1 Message Date
Travis Nielsen 09c233ffb0 core: skip node watcher during initial cache sync
When the operator starts, there is no need to handle the
node watcher immediately while the controller runtime
cache is being initialized. The cluster will anyway be
fully reconciled and create new OSDs if needed. The node
watcher is only intended for triggering a reconcile when
a node is added or updated at a later time after a sync
would anyway be triggered.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 7803fac10d)
2026-06-23 23:34:18 +00:00
subhamkrai 89335cb311 csi: check for operator chart label for ownership
adding check if operator is installed via helm with
existing cluster chart check so that if some users
install only operator chart and not the cluster chart
we are still able to transfer the ownership of old
yaml based csi resource to new helm based csi.

Signed-off-by: subhamkrai <srai@redhat.com>
2026-06-16 17:27:15 +05:30
Santosh ffc60861aa core: delete stale MDS and RGW pdbs
When RGW and MDS intances are reduce to 1, the older RGW and MDS PDB
instance (with maxUnavailabe=1) are still available. This blocks node
drain when the node, on which the single RGW and MDS instances are
running, is drained.

This PR deletes any stale MDS and RGW PDB when the only 1 MDS and 1 RGW
instances are available.

Signed-off-by: Santosh <sapillai@redhat.com>
(cherry picked from commit 3396198e9c)
2026-06-10 16:55:14 +00:00
Yuxiang He 2079062c64 osd: support using node labels for OSD device class assignment
Add support for the osd.rook.io/device-class node label to assign
CRUSH device classes to all OSDs on a node.
The operator reads the label during OSD provisioning and
deployment reconciliation. If CR-level deviceClass conflicts
with the node label, the operator will skip reconciling the
node and logs an error.

Resolves: #17131
Signed-off-by: Yuxiang He <yuxiang@dropbox.com>
(cherry picked from commit d9357dbfdc)
2026-06-09 03:27:39 +00:00
Sunnatillo 33de72b18f osd: set require-osd-release from the health monitor
applyUpgradeOSDFunctionality() may run before all OSDs have registered
their new version, leaving require-osd-release unset and the cluster in
HEALTH_WARN.

Check for a single converged OSD version on each health-monitor tick and
set require-osd-release when it changes. The release name is cached to
avoid redundant ceph commands

Signed-off-by: Sunnatillo <sunnat.samadov@est.tech>
(cherry picked from commit f6fa25e844)
2026-06-04 15:31:11 +00:00
Joshua Hoblitt 560e6514f1 core: add cluster label to all servicemonitor scraped metrics
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit cc05619dd8)
2026-05-26 19:09:03 +00:00
Santosh 42f6f90990 osd: fix flaky TestAddRemoveNode failure
User RaceFreeFakeWatcher isntead of FakeWatcher in TestAddRemoveNode
function. The test runs c.Start() in a goroutine whch stops the watcher
when done. If mockNodeOrchestrationCompeltion sends a watch ever after,
that, FakeWatcher panics with "send on a closed channel".
RaceFreeFakeWatcher safely ignores the sends on a stopped channel.

Signed-off-by: Santosh <sapillai@redhat.com>
(cherry picked from commit 6781d2e58d)
2026-05-15 19:39:59 +00:00
Artem Torubarov 9567cc44a7 osd: honor per-device deviceClass in raw-mode prepare and reconcile
Fixes #16834 and #17384. Per-OSD CR deviceClass was ignored in
osd prepare job when reporting the osd config back to the operator,
and in the operator's reconcile override.
Fixed by resolving the per-device class from the CR devices list
in GetCephVolumeRawOSDs and in makeDeployment.

Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
(cherry picked from commit b41d63788d)
2026-05-14 18:04:53 +00:00
Travis Nielsen 458ab183ce Merge pull request #17516 from rook/mergify/bp/release-1.19/pr-17472
mgr: retry self-signed cert creation on wrapped context deadline errors (backport #17472)
2026-05-12 13:37:39 -10:00
immanuwell 209afb081c docs: fix duplicated words in docs and comments
Clean up duplicated words and a typo in object storage docs, comments, and user-facing messages.

Signed-off-by: immanuwell <pchpr.00@list.ru>
(cherry picked from commit c3a9f5fdd9)
2026-05-12 22:41:39 +00:00
texasich 44123d7472 mgr: retry self-signed cert creation on wrapped deadline errors
Signed-off-by: texasich <texasich@users.noreply.github.com>
(cherry picked from commit beb6c0f373)
2026-05-12 22:37:48 +00:00
Prabhala Tara Aasrita 7d35e21f7c mon: check node existence on every health check iteration
Previously, shouldFailoverMonImmediately() was called only when a mon first went out of quorum. If the node gets deleted after the timeout starts, it does not get detected until the timeout expires. Moved the call outside the timeout initialization block so it runs on every health check iteration and detects node deletion immediately.

Fixes #15953

Signed-off-by: Prabhala Tara Aasrita <taraasrita@ibm.com>
(cherry picked from commit aeae61e5be)
2026-05-07 06:03:33 +00:00
Santosh eea22fa5e5 osd: detect encryption label from dmcrypt path
Detect OSD encryption from the dmcrypt block path in `getOSDInfo` as a
fallback when the `encrypted` label is missing on pre-label deployments.

Include `osd.Encrypted` in the label check in `makeDeployment` so the
label reflects the actual OSD state on upgrade.

Fixes a bug where upgrading to a Rook version with the `encrypted` label
would set the label to `false` on already-encrypted OSDs, because the
label didn't exist yet and `getOSDInfo` defaulted to `false`.

Signed-off-by: Santosh <sapillai@redhat.com>
(cherry picked from commit fc9e3b54f3)
2026-04-30 08:08:09 +00:00
Rohan Gupta 62215c7061 exporter: set correct DNSPolicy for rook-ceph-exporter
rook-ceph-exporter should use DNSPolicy ClusterFirstWithHostNet when
hostNetwork is enabled

Signed-off-by: Rohan Gupta <rohgupta@redhat.com>
(cherry picked from commit 17873d3af2)
2026-04-29 18:11:39 +00:00
Santosh c2cf45e716 core: remove newlines from liveness probe scripts
Strip unnecessary leading and trailing newlines from osdLivenessProbeScript and livenessProbeScript raw string literals in spec.go

Signed-off-by: Santosh <sapillai@redhat.com>
(cherry picked from commit 925acb3e92)
2026-04-24 13:44:34 +00:00
Travis Nielsen 2fb3f54c73 Merge pull request #17289 from subhamkrai/add-helm-annotation-to-csi-1.19
csi: add helm ownership annotation to csi resources
2026-04-24 07:40:45 -06:00
subhamkrai cf42d313a9 csi: add helm ownership annotation to csi resources
We are moving the CSI management to the admin and currently, the
CSI resources are created at runtime so the cluster installed with
helm will face an error that resources are not managed by helm.
To avoid this error, we are adding the annotation at runtime so
that during upgrade users do not face issues.

Signed-off-by: subhamkrai <srai@redhat.com>
2026-04-23 20:07:33 +05:30
David Orman 9a7c43cb60 osd: fix CRUSH device class not applied during OSD re-discovery
When an OSD is purged and reprovisioned, it comes back without the
correct CRUSH device class. NVMe devices configured with
deviceClass: nvme in the CR end up as ssd.

Root cause: the operator sets ROOK_OSD_CRUSH_DEVICE_CLASS on prepare
jobs but not on OSD deployments. During re-discovery
(GetCephVolumeRawOSDs), the daemon reads this env var, finds it
absent, and falls back to auto-detected deviceType which returns ssd
for NVMe.

Fix: add crushDeviceClassEnvVar(osd.DeviceClass) to the OSD
deployment spec in spec.go, alongside the existing
dataDeviceClassEnvVar. Existing deployments pick up the new env var
on operator reconciliation without manual intervention.

Fixes #17227
Signed-off-by: David Orman <ormandj@corenode.com>
(cherry picked from commit 1357d71177)
2026-04-21 12:21:03 +00:00
Denis Egorenko 99e0268bde mds: fix incorrect behaviour for CephFS health state when no active standby
Set expected standby mds daemons, when changing activeStandby mds parameter.

Resolves #17372

Signed-Off-By: Denis Egorenko <degorenko@mirantis.com>
(cherry picked from commit b3ffac9e0d)
2026-04-20 20:16:12 +00:00
Travis Nielsen d5852b3b32 mon: prevent mon drains more reliably when mons are down
The mon PDB was previously only being adjusted
during mon failover, where the maxunavailable was set to 0
before failover, then reset to 1 after failover. This allowed
for a race condition where one mon had been drained and is
not fully back in quorum before the next mon is allowed to drain.
This is now prevented by disallowing drains until a mon
is fully back in quorum. Anytime a mon falls out of quorum,
the mon pdb will be adjusted to prevent further drains
from causing a mon quorum outage.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 80b50969b2)
2026-04-17 18:15:53 +00:00
Sonny Sasaka 7128bd9178 csi: fix swapped provisionerPriorityClassName with pluginPriorityClassName
Previously provisionerPriorityClassName config is applied to node plugin
but pluginPriorityClassName is applied to controller plugin. This causes
a misconfiguration since the default values for
provisionerPriorityClassName is system-cluster-critical and
pluginPriorityClassName is system-node-critical.

The default priority for node plugin ought to be system-node-critical so
we can leverage this information to kill the pods in this priority last,
allowing other pods that may mount RBD-backed volume to unmount cleanly
first, otherwise the shutdown sequence may hang due to failure
unmounting RBD-backed volume when the node plugin pod is already killed.

Signed-off-by: Sonny Sasaka <sonnysasaka@gmail.com>
(cherry picked from commit dff04494fe)
2026-04-17 04:37:23 +00:00
Alexander Degenhart 9a5acd919b csi: add 'CSIMetadataRadosNamespace' parameter to CephFilesystemSubVolumeGroup
This parameter is written to .spec.cephFS.radosNamespace of the generated
ClientProfile.

The parameter is called 'CSIMetadataRadosNamespace' to clarify that it is used
for CSI metadata and unrelated to the actual CephFS (meta)data.

Context:

The ceph-csi CephFS plugin stores additional metadata related to PV(C)s
in RADOS objects in the metadata pool of the CephFS.
By default those objects are stored in the 'csi' RADOS namespace.

For a multi-tenant CephFS setup (that is used by multiple K8s Clusters/
ceph-csi drivers) segregating those additional objects into separate RADOS
namespaces is desireable so the clusters can not modify the metadata of other
clusters (enforced by appropriate ceph client capabilities).

'ceph-csi' implemented this via the `cephFS.radosNamespace` config option for
cluster entries in the `ceph-csi-config` ConfigMap with https://github.com/ceph/ceph-csi/pull/4661.

The `ceph-csi-operator` added support for that config entry by adding
'radosNamespace' to the 'CephFsConfigSpec' of the 'ClientProfile' with
https://github.com/ceph/ceph-csi-operator/pull/165.

Signed-off-by: Alexander Degenhart <degenhart@fim.uni-passau.de>
(cherry picked from commit 17b98da89e)
2026-04-15 23:14:39 +00:00
Travis Nielsen 89ead2f314 Merge pull request #17299 from rook/mergify/bp/release-1.19/pr-17160
test: add integration tests for CephObjectStoreUser capabilities (backport #17160)
2026-04-08 13:09:00 -06:00
Travis Nielsen 99c90c6d8e osd: add logging when detecting osd versions
When osd versions are being detected, log more info
about the versions to understand better if the
require-osd-version release is being set as
expected.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit a80ca49855)
2026-04-08 17:07:28 +00:00
Joshua Hoblitt 01eeb21aeb test: add integration tests for CephObjectStoreUser capabilities
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 0139637e34)
2026-04-02 20:41:07 +00:00
hjk068 ff79144fef object: call generateUserCaps after AdminOpsClient calls
This prevents overwriting the user-specified capabilities
and correctly copies the Capabilities into UserCapabilities
after the AdminOpsClient calls.

Signed-off-by: hjk068 <hello.hyunjin@gmail.com>
(cherry picked from commit b113e37dae)
2026-04-02 15:15:22 +00:00
Tarun Gupta Akirala 89129f03cd cosi: update default sidecar image version
Encountered this bug in current image kubernetes-sigs/container-object-storage-interface#173
which is fixed in recent release via kubernetes-sigs/container-object-storage-interface#197

Signed-off-by: Tarun Gupta Akirala <tarun.akirala@nutanix.com>
(cherry picked from commit 9803964d03)
2026-04-01 11:28:33 -06:00
majiayu000 c9e0bca7ff object: add labels field to CephObjectStore RGW service spec
Allow users to configure custom labels on CephObjectStore RGW services
by adding a Labels field to the RGWServiceSpec, following the existing
pattern used for Annotations. This enables use cases such as service
mesh integration and monitoring discovery that require specific labels
on Kubernetes services.

Fixes: #17235
Signed-off-by: majiayu000 <1835304752@qq.com>
(cherry picked from commit ac6ec79865)
2026-03-30 16:38:49 +00:00
Oded Viner 7ec6cb6aef pool: clean up erasure code profile on pool deletion
deleting an erasure coded CephBlockPool left the ec profile
orphaned. recreating with different settings failed because
the old profile could not be overridden.
- delete ec profile during CephBlockPool deletion
- disable --format json on ec profile set command
- fix ec profile name mismatch in CephObjectStore deletion

Signed-off-by: Oded Viner <oviner@redhat.com>
(cherry picked from commit b88648d10f)
2026-03-19 16:05:04 +00:00
Oded Viner 5e62a9bedd pool: set ec pool status to ready after reconcile
ec pools skip the mirroring configuration path where
the status is set to ready, leaving them stuck in
progressing state forever. add status update to ready
in the else branch when mirroring is skipped.

Signed-off-by: Oded Viner <oviner@redhat.com>
(cherry picked from commit ba9fad77a0)
2026-03-17 21:34:56 +00:00
Niraj Yadav 333904ac0a csi: update ceph-csi image to v3.16.2
This patch updates the ceph-csi image version
to v3.16.2

Signed-off-by: Niraj Yadav <niryadav@redhat.com>
(cherry picked from commit c5613a3320)
2026-03-13 15:50:47 +00:00
Artem Torubarov eb17bb490f rgw: not update shared pools if prev pool is not empty
Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
(cherry picked from commit 845293c3cd)
2026-03-11 22:13:43 +00:00
Artem Torubarov e2ce1ea2b0 rgw: support new RGW pools in shared pools zone json config
Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
(cherry picked from commit a134643352)
2026-03-11 22:13:42 +00:00
Muhammad Adil Ghaffar ee5eb68223 exporter: delete orphaned ceph-exporter deployments on reconcile
Clean up stale exporter deployments whose target node no longer exists.
This handles the case where the operator was down when a node was deleted,
so the deletion event was missed and the stale deployment was never removed.

Signed-off-by: Muhammad Adil Ghaffar <muhammad.adil.ghaffar@est.tech>
(cherry picked from commit 0b8ed65749)
2026-03-10 21:26:06 +00:00
Travis Nielsen 9ca5560d22 exporter: continue reconciling all clusters when disabled in the first
When reconciling the crash collectors and exporters, if one cluster
has disabled them, they will be deleted, then the reconcile
will complete, thus skipping reconcile of the crash collectors
and exporters for the remaining clusters.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 71e1dbffa4)
2026-03-10 17:56:41 +00:00
Travis Nielsen 46d9c79f8a exporter: remove exporter resources as best effort
Remove the exporter deployment when needed as best effort
in case the removal fails, there is no need to block other
reconcile.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 256a338c14)
2026-03-10 17:56:41 +00:00
Travis Nielsen 0cce6bac21 exporter: remove obsolete version checking for ceph v18
Rook no longer supports earlier than Ceph v18, so remove the
obsolete version checking in the ceph exporter reconcile.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 733406b175)
2026-03-10 17:56:41 +00:00
parth-gr 379f313e97 pool: skip miroring if the pool is ec
the ec pool doesnt support mirroring from ceph end
so skip if the mirroring spec is set on the ec pools

Signed-off-by: parth-gr <partharora1010@gmail.com>
(cherry picked from commit 9f932d477e)
2026-03-06 14:41:49 +00:00
mergify[bot] dd72e9917a Merge pull request #17158 from rook/mergify/bp/release-1.19/pr-17116
exporter: add configurable port for ceph exporter (backport #17116)
2026-03-05 19:22:51 +00:00
Oded Viner b3f26bc27a exporter: add configurable port for ceph exporter
the ceph exporter port was hardcoded to 9926, preventing
co-location of exporters from different ceph clusters on
the same node. add a port field to cephexporterspec so
users can override it via spec.monitoring.exporter.port.

Signed-off-by: Oded Viner <oviner@redhat.com>
(cherry picked from commit c2a2fc4241)
2026-03-05 17:11:45 +00:00
Travis Nielsen 0e21503455 test: increase osd reconcile wait timeout
The osd unit tests periodically fail with a timing issue
when canceling an orchestration. Attempt to stabilize
the test with a longer timeout before canceling the
test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 96f4320a34)
2026-03-05 16:24:34 +00:00
mergify[bot] 1d243854d4 Merge pull request #17127 from rook/mergify/bp/release-1.19/pr-17119
deploy: update CSI sidecars to latest versions available (backport #17119)
2026-02-26 19:02:19 +00:00
Travis Nielsen 9c1100abc7 Merge pull request #17125 from rook/mergify/bp/release-1.19/pr-17112
osd: update lockbox key rotation for encrypted OSDs (backport #17112)
2026-02-26 11:45:15 -07:00
Praveen M 95e4a8a082 csi: update CSI sidecars to latest versions available
Updated the following csi sidecars to their latest available versions:
- csi-attacher: v4.11.0
- csi-snapshotter: v8.5.0
- csi-resizer: v2.1.0
- csi-provisioner: v6.1.1
- csi-node-driver-registrar: v2.16.0

Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit 6b55092abf)
2026-02-26 17:51:03 +00:00
Artem Torubarov 7dd938b222 rgw: skip non-raw commands on object deletion path when pools not exists
Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
(cherry picked from commit a7f176b366)
2026-02-26 17:50:08 +00:00
Artem Torubarov 16ea704cb6 rgw: force objectStore controller to wait until zone and sharedPools are ready
Fixes race condition for rgw multisite deployments caused by ObjectStore
and Zone controllers calling period commit concurrently. See: https://github.com/rook/rook/issues/17013

Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
(cherry picked from commit b47d2c2f79)
2026-02-26 17:50:08 +00:00
Blaine Gardner ef9e8be357 osd: update lockbox key rotation for encrypted OSDs
Update `lockbox.keyring` rotation for encrypted OSDs to ensure the key
is not lost if the init container fails but continues assuming
(reasonably) that the key is likely still relevant.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 931006fb91)
2026-02-26 15:41:51 +00:00
Satoru Takeuchi ec6008e2ea osd: set device-type label on update
We should set OSDInfo.DeviceType if "device-type" label is
set on the corresponding Deployment. Otherwise, Rook
deletes this label on Update.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
(cherry picked from commit dac71d9edd)
2026-02-25 12:27:01 +00:00
Praveen M 08985245ad csi: update imagePullPolicy in operatorconfig and driver CR
Adds ImagePullPolicy configuration to CSI node plugin and controller
plugin specs in both the operator config and driver CR generation functions.
This ensures the CSI driver components use the configured pull policy
from CSIParam.ImagePullPolicy.

Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit 67c4203531)
2026-02-24 16:52:57 +00:00
Oded Viner 27a3576407 nvmeof: add default gateway pod spreading
ensure each gw pod runs on a different node when possible.
add default pod anti-affinity and topology spread constraints.
keep user placement overrides by merging defaults with spec placement.
add tests for default spread and user override behavior.

Signed-off-by: Oded Viner <oviner@redhat.com>
(cherry picked from commit bc6e5fbc86)
2026-02-18 18:43:15 +00:00