One ore more OSDs can be down and PGs can still be active+clean (data
was rebalanced to other available OSDs). This PR sets
maxunavailable=1+downOSDs to allow one healthy to be drained.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
(cherry picked from commit a816b4a971)
the rgw operations for s3 can now be accessible using sidecar
rgw-ops-log availabe in json form that can be further filtered logging
for observability, this will set the rgw_enable_ops_log setting
Signed-off-by: Deepika Upadhyay <deepika.upadhyay@clyso.com>
(cherry picked from commit c5d27467d6)
Implement #15119
Allow users to override RGW configurations by specifying Ceph config
options in the CephObjectStore. For configurations that require RGW to
be restarted when the config is applied, allow configs to be specified
as CLI arguments to the RGW as well.
This is an advanced option and is documented as such. Users should be
careful to understand the values they are setting, as there is no
validation to prevent the object store from breaking when these configs
are used.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit aaa16488de)
This commit adds optional support to specify the MDS metadata pool name. It defaults to `<fsName>-metadata` as current implementation expects but allows customization if needed e.g. if exisiting naming conventions used `<fsName>_metadata`. Additionally an "preservePoolNames" boolean has been added to indicate that no generated pool names should be used.
Signed-off-by: Tobias Wolf <wolf@b1-systems.de>
(cherry picked from commit 675678fad6)
For nodes that are explicitly requested for deploying
OSDs, they will be skipped temporarily if not
schedulable or not ready. A future reconcile is
expected to schedule them. Allow the OSDs to be
scheduled even on these nodes that are temporarily
down, even if it blocks the reconcile from completing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Modify the CR to allow mirroring of an rados namespace
to a differently named namespace on the remote cluster
1) enable rados namesapce mirroring only
if the blockpool mirrroing is enabled
2) disable blockpool mirroing only if
all the namesapce mirroing is disabled
if the rbd mirroring fails and ceph version is not supported
provide a error message with supported version details
and reason of failing
Signed-off-by: parth-gr <partharora1010@gmail.com>
These fields must be optional in order to allow usage of pre-existing
pools managed by CephBlockPool CR(s).
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Allow RGW users to mount arbitrary volumes to RGW pods. This follows the
pattern that was established for NFS to support SSSD and LDAP, and
reuses much of the same code.
This opens the door wider for advanced users to take advantage of some
Ceph RGW features that Rook doesn't have first class support for.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
For the specification see:
<https://github.com/rook/rook/blob/master/design/ceph/object/swift-and-keystone-integration.md>
* extend the API object specs for swift and keystone integration
* adapt rgw to the new go-ceph version
- The parameter lists of the API call have changes, as parameters
ignored by the RGW Admin Ops API are no longer serialized, therefore
the mock has to be adapted.
- There is now validation for the user keys that are passed to the
User get API, therefore things failed when we had empty keys in our
User proxy object.
* expand the reconcile loop for the swift and keystone integration
* fix minor mistakes in design document
* add env var to pass extra args to minikube
Minikube decides CPU cores and memory automatically based on the
available resources on the machine which may be insufficient to
run rook. This commit adds an environment variable to add arbitrary
arguments to the minikube command, so both can be specified if
desired.
* integration tests for swift and keystone
The new integration of swift or s3 and keystone support by rook
does not have any integration tests yet.
This commit introduces integration tests for swift and keystone. The
tests are done against a minimal keystone setup (keystone container
image from Yaook-project (https://yaook.cloud), sqlite as database
backend, cert-manager and trust-manager for test certificate setup).
To prevent hardcoded credentials, passwords are generated
by the tests. The integration tests use the openstack client
(keystone- and swift-functionality) (https://docs.openstack.org/
python-openstackclient/ latest/). This was a concious design decision
to use client tooling as close as possible to the end user instead of
using other go-libraries (such as gophercloud).
* add documentation on swift and keystone
Currently there is no documentation on the use of Swift to access
an object store as well as the use of OpenStack keystone for
authentication.
This commit adds documentation on the use of Swift and OpenStack
keystone, as well as CRD-related documentation and an example setup.
* add integration tests for S3 via keystone
This commit introduces integration tests for s3 and keystone. The
tests are run against the same minimal keystone setup that the tests
for swift and keystone use.
The integration tests use the aws s3 client to use client tooling as
close as possible to the end user instead of using other go-libraries.
Co-authored-by: Jan Klippel <jan.klippel@uhurutec.com>
Co-authored-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Signed-off-by: Jan Klippel <jan.klippel@uhurutec.com>
Signed-off-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
Add CephObjectStore spec.hosting.advertiseEndpoint configuration. This
provides a clear documented default for which endpoint Rook "advertises"
to dependent resources like CephObjectStores, OBCs, and COSI
Buckets/Accesses and allows users to override the default behavior if
desired.
The current default is to round-robin an endpoint from
spec.hosting.dnsNames, which has proven to be troublesome for some
users' object store configurations. This change provides much-needed
disambiguation for users.
This may be a breaking change for some existing spec.hosting.dnsNames
users. This is unexpected but is documented.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Normally the device class of an OSD is determined at provisioning
and is not updated thereafter. In some scenarios the admin may
want to force update the device class to a new value. The device
classes can be updated by first setting allowDeviceClassUpdate
in the storage spec of the cephcluster, then updating the
device class specified on the deviceSets or other OSDs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
initially, we have the limit of 40 char set on
storageclassdeviceset.Name but seems like that is not enough and
upgrade clusters can face issue with this. So, increasing the limit to
50.
Signed-off-by: subhamkrai <srai@redhat.com>
this will be disabled by default but in scenarios where the user want to
update failureDomain, DeviceClass etc, this option can be enabled, to be
noted this can lead to lot of data rebalancing and remapping. Use with
caution
Signed-off-by: Deepika Upadhyay <deepika.upadhyay@clyso.com>
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This PR adds a `settings` option in the `Module` struct
which can help users to futher customize the `balancer`
module with a differet mode.
Signed-off-by: sp98 <sapillai@redhat.com>
LVM-based OSDs on PVCs are legacy and are not well tested.
Therefore, we add status to the cephcluster CR that at least
gives a clue that the OSDs should be replaced.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
there are some cases where storageClassDeviceSets length
are more 63 chars and osd provisioned failed due to this.
with error
```
failed to run osd provisioning job for PVC "ocs-deviceset-*-local-volume-storageclass-0-data-*": Job.batch "rook-ceph-osd-prepare-*" is invalid: [spec.template.spec.volumes[10].name: Invalid value: "ocs-deviceset-*-local-volume-storageclass-0-data-*": must be no more than 63 characters, spec.template.spec.containers[0].volumeMounts[9].name: Not found: "ocs-deviceset-*-local-volume-storageclass-0-data-*",
```
Using x-validator to limit the maxLength to the deviceClassSet.Name to
40 chars and adding code check that overall pvc name should be less than
63 chars.
Signed-off-by: subhamkrai <srai@redhat.com>
The following printcolumns were added
CephBlockPoolRadosNamespace: Phase,
BlockPoolName, Age
CephFilesystemSubVolumeGroup: Phase,
FilesystemName, PinningConfig, Age
Also improved the error messages in
SubVolumeGroupClient
Signed-off-by: NymanRobin <nyman.robin@gmail.com>
For check if cluster PGs are healthy before osd updates
Which helps gainning more confidence in the osd upgrades
Signed-off-by: mmaoyu <wupiaoyu61@gmail.com>
cephfs subvolumegroup supports creating svg
with quota and the datapool, This PR adds the
support for the same.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
make the kubectl output for crds more verbose.
Following changes was done for each crd:
CephBlockPool:
Added print columns:
- Type
- FailureDomain
- Replication
- EC-CodingChunks
- EC-DataChunks
- Age
Extended info field of status to include:
type and failureDomain
CephObjectStore:
Added print columns:
- Endpoint
- SecureEndpoint
- Age
Added Age to print columns of following crds:
CephObjectStoreUser, CephObjectZoneGroup,
CephObjectZone, CephBucketTopic, CephClient,
CephRBDMirror, CephFilesystemMirror,
CephFilesystemSubVolumeGroup
Signed-off-by: NymanRobin <nyman.robin@gmail.com>
Few day's back go 1.22 is release. Let's add that
as valid version to build on. Also, adding 1.22 as
test matric for linux build CI.
https://tip.golang.org/doc/go1.22
Signed-off-by: subhamkrai <srai@redhat.com>
Santosh Pillai
Artem Torubarov
Deepika Upadhyay
Blaine Gardner
Michael Adam
Tobias Wolf
Travis Nielsen
parth-gr
parth-gr
Joshua Hoblitt
Artem Torubarov
subhamkrai
NymanRobin
mmaoyu
Madhu Rajanna
Liang Zheng
subham rai