When node topology labels with duplicate values are detected, rook silently discards all but one label.
This adds a warning informing administrators of this, allowing for easier identification of labeling issues.
Signed-off-by: Fred Heinecke <fred.heinecke@yahoo.com>
(cherry picked from commit f54ecaaff1)
Add test to canary-integration-test scenarios which removes
osd auth and then recovers it on osd activation from keyring
file.
Resolves: #14825
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
(cherry picked from commit 12613bc2fa)
If host path keyring file exists and created during osd-prepare but not
imported to ceph auth then rook-ceph-osd activate init container will
create new keyring in ceph auth by ceph auth get-or-create command. To
avoid keyrings difference in host path and in ceph auth, consider
importing key from host path keyring file if no keyring found in ceph
auth.
Resolves: #14825
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
(cherry picked from commit 6eb2d1121a)
# Conflicts:
# pkg/operator/ceph/cluster/osd/spec.go
Previously arbiter monitors were allowed to failover in supported versions
of Ceph. This behaviour regressed. Since all supported versions of Ceph now
support this, we no longer need to check this at all.
Signed-off-by: Grant Fleming <gfleming@evroc.com>
(cherry picked from commit cdf692d227)
For nodes that are explicitly requested for deploying
OSDs, they will be skipped temporarily if not
schedulable or not ready. A future reconcile is
expected to schedule them. Allow the OSDs to be
scheduled even on these nodes that are temporarily
down, even if it blocks the reconcile from completing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 7fbe4e7db5)
The /run/udev directory is now mounted in the OSD init container. This change
is necessary because `ceph-volume activate` needs access to /run/udev [1] in order
to properly handle devices. Without this mount, `ceph-volume activate` could
fail to discover devices during OSD initialization.
[1] https://github.com/ceph/ceph/commit/c2e8c295dc46f1bd74861feb163d9d82f8bb5509
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
(cherry picked from commit dbcd0d46dd)
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Resolves CIS benchmark rule 5.7.3, Pods part. SecurityContext
should be explicitly defined in pod level of Pod spec section.
It is sufficient to specify empty securityContext to satisfy
CIS 5.7.3 rule.
5.7.3 Apply Security Context to Your Pods and Containers
When designing your containers and pods, make sure
that you configure the security context for your pods,
containers, and volumes.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
(cherry picked from commit 516eab4d8a)
MustDelete DeleteOption was set to `true` when deleting
osd-repalce-config configMap. This cm won't existing after
all the OSDs get migrated. So deletion of the non-existent
cm will result in error. This PR removes `mustDelete: true` from
the DeletionOption when deleting osd-replace-config configmap
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
(cherry picked from commit 8dd237b460)
The exporter pods were being counted as a ceph pod,
so once an exporter pod was added to a node, it would never
be removed since the controller thinks the exporter pod
itself is a ceph pod for which the exporter must be present.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit a71078b733)
Disabling the RBD and CephFS fencing
in Rook for now as its having bugs where
Rook is blocklisting wrong IP address
due to timing issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 1d1ed5e962)
If the mon failover is in progress, ensure the removal
of an extra mon deployment is skipped since that code
path only has one mon in the list for the mon that was
just newly started. The extra mon was erroneously removing
a random mon in that case, followed immediately by the mon
failover completing and removing the expected failed mon,
and potentially causing mon quroum loss.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit e2cadab6fe)
This adds an operator config setting ROOK_REVISION_HISTORY_LIMIT
defaulting to kubernetes'value for RevisionHistoryLimit.
If configured, the provided value will be used as RevisionHistoryLimit
for all Deployments rook creates.
Fixes: #12722
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit ab8fd90aa6)
The crash collector daemon had missed the ability to
add custom annotations via the cephcluster CR, as already
supported for all the other daemons.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit fa2941d098)
allows to set prio-limit and stats-period options to ceph exporter
Signed-off-by: Artem Torubarov <artem.torubarov@clyso.com>
(cherry picked from commit b2c39ace25)
When removing a mon from quorum, there is a race condition that
can result in mon quorum going being lost at least temporarily.
The mon pod was being deleted first, and then the mon removed
from quorum. If any other mon went down between the time the
pod of the bad mon was deleted and when the mon was removed from
quorum, there may not be sufficient quorum to complete the
action of removing the mon from quorum and the operator would
be stuck.
For example, there could be 4 mons temporarily due to timing
of upgrading K8s nodes where mons may be taken down for some
number of minutes. Say a new mon is started while the down
mon also comes back up. Now the operator sees it can remove
the 4th mon from quorum, so it starts to remove it. Now say
another mon goes down on another node that is being updated
or otherwise drained. Since the 4th mon pod was deleted
and another mon is down, there are only two mons remaining
in quorum, but 3 mons are required in quorum when there
are 4 mons. Therefore, the quorum is stuck until the
third mon comes back up.
The solution is to first remove the extra mon from quorum
before taking down the mon pod.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 8987d26a6d)
This new setting is of Boolean type and defaults to "false".
When set to "true", it changes the behavior of the
rook operator to
nable host network on all pods created by the cephcluster controller
new method to check the setting: opcontroller.EnForceHostNetwork()
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit e378588359)
Previously, Rook would fail to create monitors for a `CephCluster` spec that
required them to be located in distinct failure domains:
```yaml
spec:
mon:
count: 3
failureDomainLabel: topology.kubernetes.io/zone
zones:
- eu-central-1a
- eu-central-1b
- eu-central-1c
```
This was due to the fact that Rook was mistakenly getting the
`failureDomainLabel` from `spec.mon.stretchCluster.failureDomainLabel`.
This commit ensures `spec.mon.failureDomainLabel` is being used instead, which
fixes the cluster deployment for the spec above.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 1c45677f5e)
this commit contains changes that are forced by latest
controller-runtime version v0.19.0.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 2556e14780)
Updating storageClassName for cluster-on-pvc.yaml file from gp2 to gp2-csi.
Signed-off-by: ShravaniVangur <shravanivangur@gmail.com>
(cherry picked from commit 9656d1963a)
As we have new csi-addons v0.9.0
updating the same here as well.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 123025f22c)
# Conflicts:
# go.sum
adding changes to create CSI-operator config CR based on
every radosnamesapce and subvolumegroup.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 667e044013)
adding code changes,rbac changes required for create the new
Ceph-CSI operator CR named cephCluster in api group 'csi.ceph.io'.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 4b0b3a55d9)
If the dashboard cert already exists when ssl is enabled,
we need to not restart the dashboard module. The module
should only be restarted when there is a change to the
dashboard configuration. When ssl is enabled, the dashboard
is essentially being restarted every time there is a reconcile
which is causing issues with the mgr getting blocklisted
unnecessarily by ceph during failover.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Normally the device class of an OSD is determined at provisioning
and is not updated thereafter. In some scenarios the admin may
want to force update the device class to a new value. The device
classes can be updated by first setting allowDeviceClassUpdate
in the storage spec of the cephcluster, then updating the
device class specified on the deviceSets or other OSDs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
currently, it was blocking both the mds ip
instead its should block ip of of mds which is in
the same node which is down.
Signed-off-by: subhamkrai <srai@redhat.com>
With the release of the first squid RC, we add squid
to the supported versions and add tests to run
Rook against the squid release.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Block devices can move between reboots. In corner cases, an OSDs block
device might move to a lower-indexed device while the previous device
does not exist. For example, an OSD on /dev/sde might move to /dev/sdd
on reboot if the original /dev/sdd died. There would be no /dev/sde
after that.
Users report that NVMe drives commonly change names, even when there are
no disk failures.
For these cases, ensure the activate script properly handles cases where
the previous disk is not present on the node and where the OSD is still
available on a different disk.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
For an advanced network configuration, the mon endpoints
expected for the ceph public network may be different from
the public IP address that is set on the K8s node. When
host networking is enabled in rook, the IP address of the
node was only allowed previously. Now, the IP address
used for a mon when scheduled on a node can be determined
by setting an annotation on each node where mons can run:
network.rook.io/mon-ip: <ip-address>
This annotation must be set before the mons are created.
Mon endpoints cannot be changed after they are already
running.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently we allow upgrade of other OSDs while migration
of OSD, due to change in backing store, is pending. This
will not work if the updated OSD ceph version does not support
the currently applied backing store. So rook will skip any
OSD upgrade if the OSD migration is pending.
Signed-off-by: sp98 <sapillai@redhat.com>
This PR adds a `settings` option in the `Module` struct
which can help users to futher customize the `balancer`
module with a differet mode.
Signed-off-by: sp98 <sapillai@redhat.com>
Travis Nielsen
Michael Adam
Fred Heinecke
Peter Razumovsky
Grant Fleming
Guillaume Abrioux
Praveen M
Santosh Pillai
Madhu Rajanna
Artem Torubarov
Benoît Knecht
parth-gr
mergify[bot]
subhamkrai
ShravaniVangur
Blaine Gardner
dependabot[bot]