When node topology labels with duplicate values are detected, rook silently discards all but one label.
This adds a warning informing administrators of this, allowing for easier identification of labeling issues.
Signed-off-by: Fred Heinecke <fred.heinecke@yahoo.com>
If host path keyring file exists and created during osd-prepare but not
imported to ceph auth then rook-ceph-osd activate init container will
create new keyring in ceph auth by ceph auth get-or-create command. To
avoid keyrings difference in host path and in ceph auth, consider
importing key from host path keyring file if no keyring found in ceph
auth.
Resolves: #14825
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
Previously arbiter monitors were allowed to failover in supported versions
of Ceph. This behaviour regressed. Since all supported versions of Ceph now
support this, we no longer need to check this at all.
Signed-off-by: Grant Fleming <gfleming@evroc.com>
Resolves CIS benchmark 5.2.8 rule by adding capabilities with
explicitly defined empty "add" list (where it needed) and with
NET_RAW capability in "drop" list.
5.2.8 Minimize the admission of containers with added capabilities
Containers must drop the `NET_RAW` capability and are not permitted
to add back any capabilities.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
For nodes that are explicitly requested for deploying
OSDs, they will be skipped temporarily if not
schedulable or not ready. A future reconcile is
expected to schedule them. Allow the OSDs to be
scheduled even on these nodes that are temporarily
down, even if it blocks the reconcile from completing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The mon canaries may be created even when the mon daemons
are not created thereafter during the integration tests.
Therefore, the integration tests need to also query a label
specific to the mon daemon so the canaries are not a distraction
to the test.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The /run/udev directory is now mounted in the OSD init container. This change
is necessary because `ceph-volume activate` needs access to /run/udev [1] in order
to properly handle devices. Without this mount, `ceph-volume activate` could
fail to discover devices during OSD initialization.
[1] https://github.com/ceph/ceph/commit/c2e8c295dc46f1bd74861feb163d9d82f8bb5509
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Finish the process of deprecating holder pods by removing Rook's ability
to deploy them. The intent of this change is to make the most
superficial changes possible to accomplish this. There are still
remnants of code in Rook (particularly the CSI controller) that helped
configure or deploy holder pods. Due to the risk of breaking some
features, cleanup work of hose remnants will be deferred for future
work.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
MustDelete DeleteOption was set to `true` when deleting
osd-repalce-config configMap. This cm won't existing after
all the OSDs get migrated. So deletion of the non-existent
cm will result in error. This PR removes `mustDelete: true` from
the DeletionOption when deleting osd-replace-config configmap
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The exporter pods were being counted as a ceph pod,
so once an exporter pod was added to a node, it would never
be removed since the controller thinks the exporter pod
itself is a ceph pod for which the exporter must be present.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The v1beta1 cron jobs have been obsolete since K8s 1.21,
and Rook has not supported that version of K8s
for many moons, so we can remove the obsolete code
for the handling of v1beta1 cron jobs for crash pruning.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Disabling the RBD and CephFS fencing
in Rook for now as its having bugs where
Rook is blocklisting wrong IP address
due to timing issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Resolves CIS benchmark rule 5.7.3, Pods part. SecurityContext
should be explicitly defined in pod level of Pod spec section.
It is sufficient to specify empty securityContext to satisfy
CIS 5.7.3 rule.
5.7.3 Apply Security Context to Your Pods and Containers
When designing your containers and pods, make sure
that you configure the security context for your pods,
containers, and volumes.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
If the mon failover is in progress, ensure the removal
of an extra mon deployment is skipped since that code
path only has one mon in the list for the mon that was
just newly started. The extra mon was erroneously removing
a random mon in that case, followed immediately by the mon
failover completing and removing the expected failed mon,
and potentially causing mon quroum loss.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This adds an operator config setting ROOK_REVISION_HISTORY_LIMIT
defaulting to kubernetes'value for RevisionHistoryLimit.
If configured, the provided value will be used as RevisionHistoryLimit
for all Deployments rook creates.
Fixes: #12722
Signed-off-by: Michael Adam <obnox@samba.org>
The crash collector daemon had missed the ability to
add custom annotations via the cephcluster CR, as already
supported for all the other daemons.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This new setting is of Boolean type and defaults to "false".
When set to "true", it changes the behavior of the
rook operator to
nable host network on all pods created by the cephcluster controller
new method to check the setting: opcontroller.EnForceHostNetwork()
Signed-off-by: Michael Adam <obnox@samba.org>
When removing a mon from quorum, there is a race condition that
can result in mon quorum going being lost at least temporarily.
The mon pod was being deleted first, and then the mon removed
from quorum. If any other mon went down between the time the
pod of the bad mon was deleted and when the mon was removed from
quorum, there may not be sufficient quorum to complete the
action of removing the mon from quorum and the operator would
be stuck.
For example, there could be 4 mons temporarily due to timing
of upgrading K8s nodes where mons may be taken down for some
number of minutes. Say a new mon is started while the down
mon also comes back up. Now the operator sees it can remove
the 4th mon from quorum, so it starts to remove it. Now say
another mon goes down on another node that is being updated
or otherwise drained. Since the 4th mon pod was deleted
and another mon is down, there are only two mons remaining
in quorum, but 3 mons are required in quorum when there
are 4 mons. Therefore, the quorum is stuck until the
third mon comes back up.
The solution is to first remove the extra mon from quorum
before taking down the mon pod.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Previously, Rook would fail to create monitors for a `CephCluster` spec that
required them to be located in distinct failure domains:
```yaml
spec:
mon:
count: 3
failureDomainLabel: topology.kubernetes.io/zone
zones:
- eu-central-1a
- eu-central-1b
- eu-central-1c
```
This was due to the fact that Rook was mistakenly getting the
`failureDomainLabel` from `spec.mon.stretchCluster.failureDomainLabel`.
This commit ensures `spec.mon.failureDomainLabel` is being used instead, which
fixes the cluster deployment for the spec above.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
adding code changes,rbac changes required for create the new
Ceph-CSI operator CR named cephCluster in api group 'csi.ceph.io'.
Signed-off-by: subhamkrai <srai@redhat.com>
If the dashboard cert already exists when ssl is enabled,
we need to not restart the dashboard module. The module
should only be restarted when there is a change to the
dashboard configuration. When ssl is enabled, the dashboard
is essentially being restarted every time there is a reconcile
which is causing issues with the mgr getting blocklisted
unnecessarily by ceph during failover.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Normally the device class of an OSD is determined at provisioning
and is not updated thereafter. In some scenarios the admin may
want to force update the device class to a new value. The device
classes can be updated by first setting allowDeviceClassUpdate
in the storage spec of the cephcluster, then updating the
device class specified on the deviceSets or other OSDs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
currently, it was blocking both the mds ip
instead its should block ip of of mds which is in
the same node which is down.
Signed-off-by: subhamkrai <srai@redhat.com>
With the release of the first squid RC, we add squid
to the supported versions and add tests to run
Rook against the squid release.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Michael Adam
Travis Nielsen
Fred Heinecke
Peter Razumovsky
Grant Fleming
parth-gr
Satoru Takeuchi
Guillaume Abrioux
Blaine Gardner
Praveen M
Santosh Pillai
Madhu Rajanna
Artem Torubarov
Benoît Knecht
parth-gr
subhamkrai
ShravaniVangur