Commit Graph
1642 Commits
Author SHA1 Message Date
Travis Nielsen d34eeee786 Merge pull request #16669 from YiteGu/use-sync-map-for-monitoringRoutines
core: use sync.Map for cluster.monitoringRoutines
2025-11-06 11:18:35 -07:00
Travis Nielsen a5180adb92 Merge pull request #16617 from Nordix/Sunnatillo/add-networkHost-field-cephMgr
mgr: add hostNetwork field to Manager spec
2025-11-06 09:28:19 -07:00
Yite Gu bf6c295d02 core: use sync.Map for cluster.monitoringRoutines
`cluster.monitoringRoutines` stores three types of
daemon health: "mon", "osd", and "status". These are
used by three goroutines respectively, resulting in:
fatal error: concurrent map reads and map writes.

Signed-off-by: Yite Gu <guyite@bytedance.com>
2025-11-06 19:36:40 +08:00
Sunnatillo b9a4685f65 mgr: add hostNetwork field to Manager spec
Add optinal hostnetwork field for Ceph Mgr to control whether the
mgr pods to run in host network. If unset, this defaults to
cluster level Network.HostNetwork value. This particulary useful
when user wants to restrict ceph mgr pod from exposing metrics
port in all host interfaces.

Signed-off-by: Sunnatillo <sunnat.samadov@est.tech>
2025-11-05 21:52:49 +02:00
Joshua Hoblitt 43f26534ed osd: add CephCluster.spec.storage.OSDMaxUpdatesInParallel
Configures the maximum number of OSDs to update in parallel.

Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
2025-11-03 12:10:28 -07:00
Travis Nielsen e8e013d88b Merge pull request #16619 from sp98/mon-canary-deletion
mon: wait for the canary pods to be terminated
2025-10-23 10:38:19 -06:00
Travis Nielsen 66ca290f98 Merge pull request #16614 from travisn/osd-resources-device-class
osd: Set osd resources for specific device class
2025-10-23 09:55:09 -06:00
Santosh Pillai 3137a857df mon: wait for the canary pods to be terminated
Wait for the mon-canary pods to be terminated before starting the actual
mons. When topology spread constraints are added to evenly spread the
mons, then pod-anti affinity does not allow mons to be created on the
same node where other mons are running, this also includes the
mon-canary pods. We delete the mon-canary pods and then start the actual
mon pods. Sometimes the termination of the canary pods might take more
time, so schedular won't schedular mon pods on that node.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-10-23 21:11:13 +05:30
Travis Nielsen 48b70500fa mon: trap the sigterm in the mon canary
The mon canary should quickly shut down after the pod is
deleted, since it has no purpose to continue running.
Since the sleep does not catch the SIGTERM as pid 1,
use a bash trap to exit quickly.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2025-10-22 15:37:13 -06:00
Travis NielsenandMatthias Osthues d3c0bacd88 osd: set osd resources for specific device class
The OSDs may require different resources based on their device
class. The overall osd resources were only being used, rather
than properly looking up the specific device class resources.
Now the expected resources will take affect per device class.

Co-authored-by: Matthias Osthues <osthues.matthias@gmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2025-10-22 10:21:40 -06:00
Parth Arora 21536e496e Merge pull request #16610 from parth-gr/fix-v2-v1-mix
external: update rook ceph mon endpoint cm to use v2
2025-10-22 15:39:45 +05:30
parth-gr 3ba911e79e external: update rook ceph mon endpoint cm to use v2
if v2 is enabled update the rook ceph mon endpoint cm
with v2 ports

Signed-off-by: parth-gr <partharora1010@gmail.com>
2025-10-21 18:31:55 +05:30
Parth Arora 9277c4f13b Merge pull request #16594 from parth-gr/fix-osd-create-upgrade-loops
osd: add a timeout for osd create and update process
2025-10-15 14:52:20 +05:30
Blaine Gardner 038638e5bc Merge pull request #16603 from BlaineEXE/osd-make-key-pull-best-effort
osd: allow OSD init keyring update to be best-effort
2025-10-14 13:00:24 -06:00
Travis Nielsen 96dda041ee Merge pull request #16595 from parth-gr/fix-status-check
core: fix ceph health check up status check
2025-10-14 10:37:46 -06:00
Blaine Gardner d9d31fe527 osd: allow OSD init keyring update to be best-effort
When OSDs start, init containers attempt to get the latest CephX key for
the OSD to allow OSDs to potentially continue working after key rotation.

However, in Ceph corner cases, mons could be unavailable while OSDs
could still be viable with existing on-disk keys. Since key rotation is
a rare event, it's likely that the on-disk keys are still good.

The worst case scenario is that key updates were needed but didn't
succeed in init containers. In this case, instead of the init container
failing, the OSD startup will fail instead. Thus, this should introduce
no extra risk but merely shift the failure to a different container.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2025-10-14 10:19:21 -06:00
parth-gr 3f5ebe2f4f external: add cluster connected condition in case of external mode
external mode was dependednt on the ceph status to
update the cluster connected condition,
BUt as now the status check will only update the ceph status,
so we will explicitly update the ceph status condition in the
end of the reconcile

Signed-off-by: parth-gr <partharora1010@gmail.com>
2025-10-14 18:08:22 +05:30
parth-gr 454c221c53 core: fix cep health check up status check
currently the health checkup was reporting the
condition and reason to be always connected,
But which might not be the case,
So instead of that dont report the condition and
its phase during the health checkups

Signed-off-by: parth-gr <partharora1010@gmail.com>
2025-10-13 20:22:07 +05:30
parth-gr 4377823675 osd: add a timeout for osd create and update process
if there a bit long to update osds in parallel,
we need to timeout and abort the synchronous loop
and start the cluster reconcile again from the beginning

Signed-off-by: parth-gr <partharora1010@gmail.com>
2025-10-13 20:08:14 +05:30
Oded Viner 11d7eacdcb osd: add detailed debug logs to checktopologyconflicts
added comprehensive debug logs throughout CheckTopologyConflicts

Signed-off-by: Oded Viner <oviner@redhat.com>
2025-10-05 22:10:19 +03:00
Travis Nielsen 270d4d4c65 Merge pull request #16501 from subhamkrai/update-ceph-to-tentacle
core: add support for ceph tentacle
2025-09-19 07:09:40 -06:00
subhamkrai 11fac627b0 core: add support for ceph tentacle
we are now adding tentacle to the supported versions and add the
upgrade test from squid to tentacle. Also, clean some unused code.

Signed-off-by: subhamkrai <srai@redhat.com>
2025-09-19 12:03:45 +05:30
Travis Nielsen 50a4a39e62 test: dependency unit test fix for blocking deletion
The dependency blocking unit test was failing with the latest
library updates due to change in behavior that the garbage
collector is not actually deleting the cluster CR after
the dependencies are cleaned up. Instead, check that the
condition indicates the resource has no dependencies remaining.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2025-09-16 11:56:31 -06:00
Santosh Pillai 1ff7604f82 core: use updated context in prevalidaiton checks
If the CephCluster spec is updated before the prevalidation check, then
Cluster.context still refers to the cancelled context of the operator.
This PR just updates the cluster context to use the latest operator
context.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-09-16 12:57:12 +05:30
Santosh Pillai d399615bdc mon: fix mon health nil pointer exception
shouldFailoverMonImmediately has a nil pointer exception with mons on
PVC. Rook-ceph-mon-endpoint does not store node information of the mons
on PVC. So `mapping.Hostname` throws nill pointer

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-09-15 18:13:46 +05:30
Blaine Gardner f90fa269f0 Merge pull request #16421 from parth-gr/cephx-osd-bootstrap
osd: remove the osd bootstrap keyring
2025-09-04 09:41:42 -06:00
parth-gr 794a232835 osd: remove the osd bootstrap keyring
when the osd is created there is no need for
osd bootstrap keyring, so remove the keyring
and keyring file,

New keyring will be created if
new osd need to be provisoned

Signed-off-by: parth-gr <partharora1010@gmail.com>
2025-09-03 21:41:52 +05:30
Blaine Gardner 982b2fe632 Merge pull request #16372 from sp98/delete-bootstrap-keys
core: delete bootstrap keys
2025-09-02 12:16:59 -06:00
Blaine Gardner ff3ef88c61 Merge pull request #16409 from sp98/rotate-lockbox-keys
osd: rotate lockbox keys for encrypted osd
2025-09-02 12:16:06 -06:00
Santosh Pillai 4cdba0df54 core: delete bootstrap keys
Delete the client.boostrap.* keys. These keys are not created by rook so
it can't rotate them. This PR just delete these keys in the
postMonStartUpAction.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-09-01 08:55:23 +05:30
Santosh Pillai b4e1c1720d osd: rotate lockbox keys for encrypted osd
ceph-volume creates osd-lockbox keys for the encryted osds. This PR
rotates those keys along with the OSD keys.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-09-01 08:44:14 +05:30
Travis Nielsen 12f1673385 Merge pull request #16382 from subhamkrai/add-cm-watcher
csi: add watcher for cm cluster controller
2025-08-29 08:24:17 -06:00
subhamkrai dd6693ff97 csi: add watcher for cm cluster controller
this commit add watcher for cm `rook-ceph-operator-config`
to look for updated value of `ROOK_USE_CSI_OPERATOR` before
adding this watcher we had to restart rook operator pod
so that it reconciles and create cephConnection CR.

Signed-off-by: subhamkrai <srai@redhat.com>
2025-08-29 13:29:34 +05:30
Travis Nielsen 3edbe6c550 Merge pull request #16402 from cuiweixie/reflect.TypeFor
operator: refactor to use reflect.TypeFor
2025-08-28 10:29:28 -06:00
Travis Nielsen 57bdd548e2 Merge pull request #16399 from travisn/update-client-profile
csi: Set the cephfs kernel mount options when network encryption is enabled
2025-08-28 10:27:02 -06:00
cuiweixie 20b3b9deef operator: refactor to use reflect.TypeFor
Signed-off-by: cuiweixie <cuiweixie@gmail.com>
2025-08-28 23:02:19 +08:00
Travis Nielsen 6309cd367e csi: default to secure kernel mount options if network encrypted
If the network is encrypted with msgr2, cephfs requires
the mount options to include ms_mode=secure. Now this
will be set by default if the mount options are not
already set in the cephcluster CR or the operator env
var CSI_CEPHFS_KERNEL_MOUNT_OPTIONS.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2025-08-27 16:30:47 -06:00
Blaine Gardner 7c33186e72 core: admin cephx key rotation
Implement CephX key rotation for Rook's client.admin user.

Admin user rotation is risky, so this has been tested extensively both
in unit tests as well as by manually injecting failures during runtime.
In testing, all failures were able to be recovered by the recovery
routine.

A mutex is also added to help ensure that two simultaneous admin key
rotation processes cannot be running simultaneously for any given
namespace. The mutex is tested in unit tests, and it was verified during
runtime via  manual testing.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2025-08-27 14:47:45 -06:00
Blaine Gardner a952b7511a core: allow cephx rotation for ceph 19.2.3 except mons
Allow Rook to rotate CephX keys for Ceph v19.2.3 and up with the
exception of Ceph mons. While v19.2.3 has the `ceph auth rotate` command
that Rook needs to perform key rotation, there is still ongoing work in
Ceph to change how the `mon.` key is tracked internally. Rook's mon key
rotation requires those currently-unreleased Ceph changes.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2025-08-18 16:50:11 -06:00
Blaine Gardner 2fd9c9a88c core: finalize CephCluster CephX status handling
Cleanup up overall usage of CephX statuses in CephCluster.

Convert all CephX statuses to non-pointer types. There is no specific
need for pointer types, and keeping these as non-pointers means there is
not a risk of nil pointer exceptions in all the various Rook controllers
for upgraded clusters.

Move initialization of CephCluster CephX status items to
`preMonStartupActions()`. This helps resolve 2 issues:

1. 'Uninitialized' state was being set for external CephClusters where
   CephX statuses are not relevant.
2. 'Uninitialized' state was being set for upgraded clusters whose key
   status cannot be determined.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2025-08-18 14:53:08 -06:00
Travis Nielsen 36a47ac01a Merge pull request #16176 from sp98/mon-key-rotation
mon: rotate cephx key
2025-08-15 11:06:31 -06:00
Travis Nielsen d6b91b9409 Merge pull request #16275 from anshuman-agarwala/secret-removal
operator: Removed legacy secret deletion code in rbd
2025-08-14 14:06:34 -06:00
Santosh Pillai fe42fe102f mon: rotate keys
rotate mon daemon keys and update the cephx status for mons in the
cephcluster resource.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-08-14 11:16:39 +05:30
Denis Egorenko 09622a8b8d operator: mount udev to cleanup job and disable udev sync
Mount /run/udev inside cluster cleanup jobs and disable udev sync
with DM_DISABLE_UDEV.

Related-Issue: rook#16292
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
2025-08-12 22:46:27 +04:00
Blaine Gardner 59ca78229d Merge pull request #16075 from sp98/mirror-key-rotation
rbd mirror peer key rotation
2025-08-11 10:54:20 -06:00
Santosh Pillai ee0b137d75 core: rotate rbd mirror peer key
rotate the client.rbd-mirror-peer key based on the cephxconfig.
Also update the cephxStatus in the cephcluster as well as the
mirrored cephblockpool.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-08-11 21:53:14 +05:30
Madhu Rajanna 1ff3f07ac5 csi: use cluster namespace for cephConn
CephConnection CR is created with the
kubernetes namespace name as its the
unique name in the entire cluster.
This PR removes unwanted input
parameters and adjust the namespace
name for the cephConnection CR.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2025-08-11 07:38:27 +02:00
Anshuman 79b24564ef operator: removed legacy secret deletion code in rbd
Signed-off-by: Anshuman <anshuman@ibm.com>
2025-08-08 21:56:06 +05:30
Blaine Gardner 517f519c23 Merge pull request #16227 from sp98/mirror-daemon-key-rotation
core: rotate mirror daemon keys
2025-08-08 09:45:48 -06:00
Santosh Pillai d0fbf6c424 core: rotate mirror daemon keys
rotate RBD mirror daemon keys, add secret resource version to the mirror
deployment and update the mirror resource status after the key is
rotated

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2025-08-08 10:23:49 +05:30