Normally the device class of an OSD is determined at provisioning
and is not updated thereafter. In some scenarios the admin may
want to force update the device class to a new value. The device
classes can be updated by first setting allowDeviceClassUpdate
in the storage spec of the cephcluster, then updating the
device class specified on the deviceSets or other OSDs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Reset the Multus validation tool debounce time to its intended 30 second
value. It was changed to 5 for testing, and the change was accidentally
committed.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
In order to help users check that they have implemented the newly-added
Multus host configuration prerequisites, add a check to the validation
tool to verify connectivity.
Because users who are already running clusters with Multus enabled, add
a flag that allows users to only check for host configuration
prerequisites. This mode will not start the large number of clients that
would normally be started because those clients could disrupt a running
Rook cluster negatively.
Host checking pods require host network access. Many Kubernetes
distributions have pod security features enabled. In order to allow
non-Vanilla distros to run this tool, allow specifying a service account
that pods will run as, which can be configured by the admin to allow
test pods.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Updating the device class swallowed any error if updated
for the pool. The error was not even logged, so we couldn't
troubleshoot why the new crush rule was not applied.
Log the error for troubleshooting and also fail the pool
reconcile since the desired configuration was not applied.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Pools in stretch clusters must all specify the same
CRUSH rule. No pools can use a different rule. When there
is a change in the device class, we do not even expect to update
the crush rules in a stretch cluster. Different device classes
are not supported in stretch clusters, and it's expected to be
a homogenous environment. Therefore, skip all crush rule updates
in stretch clusters.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
While adding a new encryption key to slot 1
if there exists a key in slot 1 which is not
equal to the one we want to update it with,
we kill the slot and then add the new key to it.
While killing the slot, the existing code uses
the new key, which is not valid in such cases.
This patch modifies the code to use the key in
slot 0 (the one that we know works) to kill the slot.
Signed-off-by: Niraj Yadav <niryadav@redhat.com>
this lead to creation of 2 crush rules because of difference in
deviceClass names:
eg:
creating a new crush rule for changed deviceClass
("default~hdd"-->"hdd") on crush rule "test-crush-bug-az-ab-4"
Signed-off-by: Deepika Upadhyay <deepikaupadhyay01@gmail.com>
this will be disabled by default but in scenarios where the user want to
update failureDomain, DeviceClass etc, this option can be enabled, to be
noted this can lead to lot of data rebalancing and remapping. Use with
caution
Signed-off-by: Deepika Upadhyay <deepika.upadhyay@clyso.com>
Add IPv6 support for multus validation tool. Also test that IPv6 support
works by specifying one of the NetAttachDefs with an IPv6 address range.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The following printcolumns were added
CephBlockPoolRadosNamespace: Phase,
BlockPoolName, Age
CephFilesystemSubVolumeGroup: Phase,
FilesystemName, PinningConfig, Age
Also improved the error messages in
SubVolumeGroupClient
Signed-off-by: NymanRobin <nyman.robin@gmail.com>
The mds standby test was timing out after six seconds on
three different attempts, causing a single unit test to
take almost 20 seconds. Now we reduce the wait time to
a total of three seconds.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
cephfs subvolumegroup supports creating svg
with quota and the datapool, This PR adds the
support for the same.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Cleanup the resources created by subvolumegroup
when its deleted. Following resources will be cleaned up:
- OMAP value
- OMAP keys
- Clones
- Snapshots
- Subvolumes
Signed-off-by: sp98 <sapillai@redhat.com>
For image mode mirroring, if cephBlockPool.Pool.Spec.Mirroring.Enable
is set to false, then remove the peer cluster and disable mirroring on
all the pool if the user has disabled mirroring on all the pool images.
If mirroring is not disabled on all the pool images, then reconcile will
fail asking the users to manually disable mirroring on those images.
Signed-off-by: sp98 <sapillai@redhat.com>
Reduce CRD size by removing some descriptions for things that
(a) are long, (b) are repeated, and (c) are commong kubernetes types.
Remove the `Status` field from VolumeClaimTemplates (a.k.a. PVCs) in
types.go to remove all unnecessary status-related fields.
Use `yq` to go through generated `crds.yaml` to strip out descriptions
for well-known k8s types that are long and repeated where we can't
remove them from types.go. This is mostly placement descriptions and
volumeSource descriptions.
The overall `crds.yaml` file is reduced to less than 60% of its original
size.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Co-authored-by: Blaine Gardner <blaine.gardner@ibm.com>
The crush rule will be updated when the failure domain changes.
This update can be skipped when the expected crush rule is
already configured. Otherwise, the log has a confusing message
that makes it appear that the crush rule was updated
when in fact it was not.
Signed-off-by: travisn <tnielsen@redhat.com>
Rook has been setting the application automatically on all
pools to rbd for CephBlockPools, rook-ceph-rgw for
CephObjectStores, mgr on the built-in .mgr pool,
and nfs on the built-in .nfs pool.
The legacy pool device_health_metrics is long gone
from Pacific which is no longer supported, so we can
remove special handling for that pool in the upgrade
guide and in the code.
The application setting is now available on the pool spec
although it is not expected to commonly need to override
the default applications set by Rook.
The application for CephFilesystem pools is now being
set to cephfs, where it was previously blank.
Signed-off-by: travisn <tnielsen@redhat.com>
Currently, when rook provisions OSDs(in the OSD prepare job), rook effectively run a
c-v command such as the following.
```console
ceph-volume lvm batch --prepare <deviceA> <deviceB> <deviceC> --db-devices <metadataDevice>
```
but c-v lvm batch only supports disk and lvm, instead of disk partitions.
We can resort to `ceph-volume lvm prepare` to implement it.
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
The validation logic of checking the number of devices is
wrong when `metadataDevice` is set and `osdsPerDevice` > 1.
`len(cvReports)` is the expected number of OSDs and is
the number of specified data devices multiplied
by `osdsPerDevice`.
Closes: https://github.com/rook/rook/issues/13637
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
create the ceph config and keyring file in the osd prepare pod
before starting the OSD migration. These files are needed to
run any ceph command.
Signed-off-by: sp98 <sapillai@redhat.com>
Use Quay as the source for the nginx-unprivileged image used for the
Multus validation tool because Quay does not rate limit image pulls,
which are a common complaint for users of the tool.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Loop variables cannot be reliably uses since they will
change with each iteration. Update these loop variable
uses to be safe by indexing the slice rather than
using the loop variable directly.
Also suppress the linter issues for passwords used
in tests.
Signed-off-by: travisn <tnielsen@redhat.com>
This implements the "Ceph Config via Ceph Cluster CRD" design document
as a `cephConfig:` structure on the CRD.
This also fixes the `yq` commands used to manipulate the
`cluster-test.yaml` that caused CI issues for this PR and potentially
unknowingly others.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Originally we create it using this cmd
ceph fs subvolume create <vol_name> <subvol_name>
So we can have 2 variables filesystem and subvolume name,
Currently the CR doesn't allow us to make subvolume-name
as constant as needed to "csi" because of k8s limitations
Signed-off-by: parth-gr <paarora@redhat.com>
Adding callback function in the osd removal method
as in downstream there is requirement of adding extra
check before proceeding with osd removal.
Signed-off-by: subhamkrai <srai@redhat.com>
This commit adds new CSIDriverOptions section in
cephCluster CR. This section contains settings
for read affinity and kernel+fuse Mount options
These settings will be injected directly into
rook-ceph-csi-config cm to be applicable per
ceph cluster.
Signed-off-by: Rakshith R <rar@redhat.com>
parth-gr
Travis Nielsen
sp98
Blaine Gardner
Niraj Yadav
Deepika Upadhyay
Brian Dowling
Satoru Takeuchi
momantech
NymanRobin
clonefetch
Madhu Rajanna
Redouane KachachandBlaine Gardner
Liang Zheng
parth-gr
subhamkrai
Alexander Trost
Rakshith R