The rook-ceph-cluster chart can deploy CephObjectStores but offered no
way to create S3 users for them, so CephObjectStoreUser resources had to
be applied out-of-band. Add a cephObjectStoreUsers values array that
renders one CephObjectStoreUser per entry, following the same name plus
spec passthrough pattern as the multisite resources, with helm-unittest
coverage.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The prepare job runs "ceph-volume raw prepare" per device and then trusts
a no-argument "ceph-volume raw list" to report the OSDs it just created.
That listing can come back without a freshly prepared device. On Ceph
v20.2.0 and v20.2.1, get_devices() enumerates through udev data under
/run/udev/data and returns nothing when that data is empty or not yet
populated (https://tracker.ceph.com/issues/77968). On every released
v20.2.x, a block device smaller than 4KiB in the scan, such as the 1KiB
extended-partition node of an MBR-partitioned OS disk, crashes the
batched ceph-bluestore-tool call behind the listing, which ceph-volume
reports as an empty result (https://tracker.ceph.com/issues/76354).
Either way the no-argument list comes back empty or incomplete even
though the OSD exists. Rook then silently reported fewer OSDs than it
prepared, the operator never created the missing OSD deployment, and the
cluster was left with OSDs in the osdmap but no daemons running.
Verify that every device prepared in raw mode is present in the listing.
For any that is missing, list it by name ("ceph-volume raw list
<device>"), which is immune to both failure modes and reports the OSD.
If even the per-device listing is empty, return an error so the prepare
job exits non-zero and Kubernetes restarts it with a freshly re-scanned
environment, rather than silently under-reporting the OSDs.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Updated the required documentation and yamls
where we are adding support for the QoS for
the rbd pvc that uses the krbd mounter.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Removed the leading v from the Helm OCI chart version tag so that helm pull can auto-resolve the latest version without specifying the tag explicitly.
Signed-off-by: Prabhala Tara Aasrita <taraasrita@ibm.com>
when using encryptedDevice:true with host based (non pvc) osds, resizing
the underlying disk and restarting the OSD didn't expand the OSD because
the LUKS encryption layer was not resized.
This PR adds resize capabilities inside the activate container. Full
device stack must be resized: PV → LV → LUKS
Steps:
1. Run pvresize and lvextend to grow the LVM layers.
2. Retrieve the LUKS key from the Ceph config-key store using the
lockbox crednetials already available in the activate container
3. Pass the key explicitly to cryptsetup resize via --key-file.
Signed-off-by: Santosh <sapillai@redhat.com>
So far, the latest supported kubernetes version was documented as 1.35
This change documents 1.36 (I. e. the latest available) as
the latest supported version.
It also updates the minimun supported kubernetes version
from 1.30 to 1.31
Signed-off-by: Michael Adam <obnox@samba.org>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
The ROOK_RECONCILE_CONCURRENT_CLUSTERS feature was implemented
in v1.19. This feature has been stable, with no related issues
reported. The feature is tested in the CI with no known
stability issues. Let's declare this feature as stable.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Going forward, admin will manage the csi operator
CR's and rook will only manage Ceph Connection cr
and client Profile cr.
The old csi driver is completely removed from Rook
and can no longer be used starting in Rook v1.20.
The upgrade guide will contain the needed transition steps
for managing the csi operator settings.
Signed-off-by: subhamkrai <srai@redhat.com>
Clean up stale CRUSH rules after the Ceph mgr starts so rules left behind by pool failure-domain or device-class changes do not accumulate indefinitely.
The cleanup is guarded by a package-level RWMutex. Pool create and update paths hold the read lock while creating and assigning CRUSH rules, while cluster-wide cleanup holds the write lock before listing pools and deleting unused rules. This keeps pool reconciles parallel with each other while preventing cleanup from deleting a rule that another reconcile has just created but not yet attached to a pool.
Keep direct pool-delete cleanup for the pool's current CRUSH rule, make the cluster-wide cleanup best-effort across all unused rules, and add an operator-level ROOK_DELETE_UNUSED_CRUSH_RULES setting for clusters that need to leave unused custom rules in place.
Document the operator and Helm settings, regenerate the Helm chart docs, and add a pending release note for the default cleanup behavior.
Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
Support using SSE-S3 encryption with RGW using vault Agent auth.
RGW sends requests to the agent instead of directly to Vault, and the agent transparently injects the authentication token. This eliminates using and managing a static token
Signed-off-by: Santosh <sapillai@redhat.com>
if admin secret is created stop creating csi secret from
the rook operator
the csi creds and secret will still be created by the python and
the import script
Signed-off-by: parth-gr <partharora1010@gmail.com>
For all of the controllers besides the cluster controller,
the logging now includes the namespaced name of the resource
that is being reconciled. This will help with log troubleshooting
to help analyze logs consistently for the resource being
reconciled.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The activeStandby property was being ignored when deciding
how many mds daemons to start. 2x the active count of mds
daemons were always being created. If the standby is not
desired, the standby can be removed, and it was unexpected
that it was not removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The cephcluster controller allows setting concurrent reconciles
with a setting in the operator configmap.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Now that core feature development is completed for 1.18,
reset the pending release notes such that changes in
master will go out with 1.19
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During mon failover, if a mon is assigned to a node
but the node does not exist, the mon should be failed
over immediately instead of waiting for a 10 or 20
minute timeout. If the node does not exist, the mon pod
will be in a pending state until the failover is finally
triggered, and no point in waiting that long if the
node is gone.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
add a validation check to fail osd deployment when topology labels
like rack or chassis are duplicated across parent domains
(e.g., zones). this prevents invalid crush map hierarchies
(dfbugs-2610).
Signed-off-by: Oded Viner <oviner@redhat.com>
Providing an option for user to specify the clusterID
they want to use in the storageclass while creating
the subvolume group.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Now rook supports the six most recent minor versions of helm along with
their patch updates. I. e. helm 3.13 and newer is supported.
Fixes: #15980
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Signed-off-by: Michael Adam <obnox@samba.org>
The change creates and manages an Endpoints resource
with the current Ceph monitor (mon) IPs, allowing
clients to resolve mon IPs via DNS without relying
on the rook-ceph-mon-endpoints ConfigMap.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
Add a note about the upcoming potentially-breaking change to OBCs to the
v1.17 release notes. This covers usage of
`ROOK_OBC_ALLOW_ADDITIONAL_CONFIG_FIELDS` for OBC fields that some
admins might not want exposed to users.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
For the upcoming 4.17 release, we reset the pending release
notes so we can add new features to the list.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With the release of K8s 1.32, we update the CI and docs
to support this new release, to maintain the most recent
six releases of K8s.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Modify the CR to allow mirroring of an rados namespace
to a differently named namespace on the remote cluster
1) enable rados namesapce mirroring only
if the blockpool mirrroing is enabled
2) disable blockpool mirroing only if
all the namesapce mirroing is disabled
if the rbd mirroring fails and ceph version is not supported
provide a error message with supported version details
and reason of failing
Signed-off-by: parth-gr <partharora1010@gmail.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With the release of K8s v1.31.0, we update the minimum
supported version to v1.26, and add v1.31 to the CI
so we can test the most recent six versions of K8s.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Update Rook and Ceph upgrade docs for upcoming v1.15 release.
Tidy up pending release notes in the working text as well as official
doc texts.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>