Going forward, admin will manage the csi operator
CR's and rook will only manage Ceph Connection cr
and client Profile cr.
The old csi driver is completely removed from Rook
and can no longer be used starting in Rook v1.20.
The upgrade guide will contain the needed transition steps
for managing the csi operator settings.
Signed-off-by: subhamkrai <srai@redhat.com>
Detect OSD encryption from the dmcrypt block path in `getOSDInfo` as a
fallback when the `encrypted` label is missing on pre-label deployments.
Include `osd.Encrypted` in the label check in `makeDeployment` so the
label reflects the actual OSD state on upgrade.
Fixes a bug where upgrading to a Rook version with the `encrypted` label
would set the label to `false` on already-encrypted OSDs, because the
label didn't exist yet and `getOSDInfo` defaulted to `false`.
Signed-off-by: Santosh <sapillai@redhat.com>
migrate bucket provisioner from aws sdk v1 to v2.
replace awserr.Error type assertions with smithy.APIError
using errors.As for v2-style error handling.
switch setBucketPolicy and setBucketLifecycle to use
the v2 s3 client (ClientV2) directly with context.
replace v1 s3 types (BucketLifecycleConfiguration,
GetBucketLifecycleConfigurationOutput) with v2 equivalents
from s3types and s3v2 packages.
add cmpopts.IgnoreUnexported to cmp.Diff calls to handle
unexported noSmithyDocumentSerde fields in v2 sdk types.
add nil guard on lifecycle output before accessing rules
to prevent nil dereference when get returns an error.
depends on #17414
Signed-off-by: Oded Viner <oviner@redhat.com>
Clean up stale CRUSH rules after the Ceph mgr starts so rules left behind by pool failure-domain or device-class changes do not accumulate indefinitely.
The cleanup is guarded by a package-level RWMutex. Pool create and update paths hold the read lock while creating and assigning CRUSH rules, while cluster-wide cleanup holds the write lock before listing pools and deleting unused rules. This keeps pool reconciles parallel with each other while preventing cleanup from deleting a rule that another reconcile has just created but not yet attached to a pool.
Keep direct pool-delete cleanup for the pool's current CRUSH rule, make the cluster-wide cleanup best-effort across all unused rules, and add an operator-level ROOK_DELETE_UNUSED_CRUSH_RULES setting for clusters that need to leave unused custom rules in place.
Document the operator and Helm settings, regenerate the Helm chart docs, and add a pending release note for the default cleanup behavior.
Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
Strip unnecessary leading and trailing newlines from osdLivenessProbeScript and livenessProbeScript raw string literals in spec.go
Signed-off-by: Santosh <sapillai@redhat.com>
Ceph relaxed the stretch mode CRUSH weight check in ceph/ceph#66580 to
allow weights within 10 percent of each other. This change is available
in Ceph Umbrella and was backported to tentacle v20.2.2 via ceph/ceph
number 67790. The squid backport is still pending.
Signed-off-by: Lumir Sliva <61183145+lumir-sliva@users.noreply.github.com>
The floating mon in a TNF cluster is designed to share a node with a pinned mon. Fixed the pre-cluster start evaluation logic to allow the same when allowMultiplePerNode is set to false.
Signed-off-by: Prabhala Tara Aasrita <taraasrita@ibm.com>
The mon PDB was previously only being adjusted
during mon failover, where the maxunavailable was set to 0
before failover, then reset to 1 after failover. This allowed
for a race condition where one mon had been drained and is
not fully back in quorum before the next mon is allowed to drain.
This is now prevented by disallowing drains until a mon
is fully back in quorum. Anytime a mon falls out of quorum,
the mon pdb will be adjusted to prevent further drains
from causing a mon quorum outage.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Ceph incorrectly raises the MON_NETSPLIT health warning in stretch cluster configurations. Added a call to MuteHealthWarning() in postMonStartupActions() which runs `ceph health mute MON_NETSPLIT --sticky` to suppress the warning during reconcile. All errors are logged as warnings and the reconcile proceeds.
Signed-off-by: Prabhala Tara Aasrita <taraasrita@ibm.com>
Previously provisionerPriorityClassName config is applied to node plugin
but pluginPriorityClassName is applied to controller plugin. This causes
a misconfiguration since the default values for
provisionerPriorityClassName is system-cluster-critical and
pluginPriorityClassName is system-node-critical.
The default priority for node plugin ought to be system-node-critical so
we can leverage this information to kill the pods in this priority last,
allowing other pods that may mount RBD-backed volume to unmount cleanly
first, otherwise the shutdown sequence may hang due to failure
unmounting RBD-backed volume when the node plugin pod is already killed.
Signed-off-by: Sonny Sasaka <sonnysasaka@gmail.com>
This parameter is written to .spec.cephFS.radosNamespace of the generated
ClientProfile.
The parameter is called 'CSIMetadataRadosNamespace' to clarify that it is used
for CSI metadata and unrelated to the actual CephFS (meta)data.
Context:
The ceph-csi CephFS plugin stores additional metadata related to PV(C)s
in RADOS objects in the metadata pool of the CephFS.
By default those objects are stored in the 'csi' RADOS namespace.
For a multi-tenant CephFS setup (that is used by multiple K8s Clusters/
ceph-csi drivers) segregating those additional objects into separate RADOS
namespaces is desireable so the clusters can not modify the metadata of other
clusters (enforced by appropriate ceph client capabilities).
'ceph-csi' implemented this via the `cephFS.radosNamespace` config option for
cluster entries in the `ceph-csi-config` ConfigMap with https://github.com/ceph/ceph-csi/pull/4661.
The `ceph-csi-operator` added support for that config entry by adding
'radosNamespace' to the 'CephFsConfigSpec' of the 'ClientProfile' with
https://github.com/ceph/ceph-csi-operator/pull/165.
Signed-off-by: Alexander Degenhart <degenhart@fim.uni-passau.de>
When osd versions are being detected, log more info
about the versions to understand better if the
require-osd-version release is being set as
expected.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Support using SSE-S3 encryption with RGW using vault Agent auth.
RGW sends requests to the agent instead of directly to Vault, and the agent transparently injects the authentication token. This eliminates using and managing a static token
Signed-off-by: Santosh <sapillai@redhat.com>
This prevents overwriting the user-specified capabilities
and correctly copies the Capabilities into UserCapabilities
after the AdminOpsClient calls.
Signed-off-by: hjk068 <hello.hyunjin@gmail.com>
initialize both aws sdk v1 and v2 clients in s3agent.
update createbucket to use sdk v2 while keeping all
other methods on sdk v1.
Signed-off-by: Oded Viner <oviner@redhat.com>
for tnf, adding initial api changes and implementation for floating mon.
The current pr follow embeded yaml approach for creating floating mon
deploymen instead of configmap approach.
Signed-off-by: subhamkrai <srai@redhat.com>
Allow users to configure custom labels on CephObjectStore RGW services
by adding a Labels field to the RGWServiceSpec, following the existing
pattern used for Annotations. This enables use cases such as service
mesh integration and monitoring discovery that require specific labels
on Kubernetes services.
Fixes: #17235
Signed-off-by: majiayu000 <1835304752@qq.com>
When an OSD is purged and reprovisioned, it comes back without the
correct CRUSH device class. NVMe devices configured with
deviceClass: nvme in the CR end up as ssd.
Root cause: the operator sets ROOK_OSD_CRUSH_DEVICE_CLASS on prepare
jobs but not on OSD deployments. During re-discovery
(GetCephVolumeRawOSDs), the daemon reads this env var, finds it
absent, and falls back to auto-detected deviceType which returns ssd
for NVMe.
Fix: add crushDeviceClassEnvVar(osd.DeviceClass) to the OSD
deployment spec in spec.go, alongside the existing
dataDeviceClassEnvVar. Existing deployments pick up the new env var
on operator reconciliation without manual intervention.
Fixes#17227
Signed-off-by: David Orman <ormandj@corenode.com>
This PR creates the account root user by default when
CephObjectStoreAccount CR is created and skips it if
`spec.rootuser.skipCreate` is true.
Signed-off-by: Santosh <sapillai@redhat.com>
add unit and integration tests to verify that erasure
coded cephblockpool status transitions to ready and
does not remain stuck in progressing.
Signed-off-by: Oded Viner <oviner@redhat.com>
deleting an erasure coded CephBlockPool left the ec profile
orphaned. recreating with different settings failed because
the old profile could not be overridden.
- delete ec profile during CephBlockPool deletion
- disable --format json on ec profile set command
- fix ec profile name mismatch in CephObjectStore deletion
Signed-off-by: Oded Viner <oviner@redhat.com>
ec pools skip the mirroring configuration path where
the status is set to ready, leaving them stuck in
progressing state forever. add status update to ready
in the else branch when mirroring is skipped.
Signed-off-by: Oded Viner <oviner@redhat.com>
Santosh Pillai
subhamkrai
Travis Nielsen
Oded Viner
Rohan Gupta
Joshua Hoblitt
Asish Kumar
Lumir Sliva
Parth Arora
Prabhala Tara Aasrita
Denis Egorenko
subham rai
Sonny Sasaka
Alexander Degenhart
Blaine Gardner
Joshua Hoblitt
hjk068
Tarun Gupta Akirala
majiayu000
David Orman
Niraj Yadav