today, csi operator cephConnection and client profile
CR's where not getting created in external mode. So, let's
start creating those.
Signed-off-by: subhamkrai <srai@redhat.com>
Adds the missing exporter container port to enable
proper Prometheus scraping and filtering.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
Implements #14733. Allows to set IDs of external mons to
Cluster CRD. Rook will not remove external mons from quorum
and will add external mon addresses to mon endpoints.
Use-case for external mon is to maintain quorum for 2-AZ
k8s cluster in case of zone outage.
Signed-off-by: Artem Torubarov <artem.torubarov@clyso.com>
There is a race condition in ceph v19 at startup of a
stretch cluster where if the .mgr pool is created by
the mgr pod after stretch mode is entered, the
devicehealth mgr module will crash and cause a
ceph HEALTH_ERR. The error then causes rook reconcile
of all CRs to be skipped because Rook is assuming
admin intervention is needed to continue. The fix
is to wait until the .mgr pool is created by the mgr
pod before entering stretch mode.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator settings loaded from the configmap have proven
inefficient for load time and frequently checking the configmap.
To avoid this ineffenciency, the configmap is only loaded once
each time it is created or updated. The values in the configmap
are applied as environment variables, which then are very efficient
to query throughout the various controllers, without needing
to be concerned about loading the configmap again.
Co-authored-by: Dmitry Mishin <dmitry.mishin@gmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a locally mounted volume is blocked on reads because this OSD is
down, lvs will deadlock trying to read from it. To avoid this, tell LVM
to filter out RBD volumes from its scanning process.
Signed-off-by: Steven Fackler <sfackler@gmail.com>
Some ceph.conf settings need to be unique per node
depending on the hardware. The ceph.conf overrides
can now be customized per-node by creating a node-specific
configmap that will be loaded for all OSDs and OSD prepare
jobs on that node, instead of the default ceph.conf
that is loaded from the rook-config-override configmap.
The node-specific configmaps must have the label
node.config.rook.io/osd, and be named as:
rook-config-override-<hostname>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This PR addresses an issue where the CrashCollector Pruner
CronJob pods were stuck in a Pending state due to missing
tolerations in their PodTemplateSpec.
These tolerations were not being propagated from the
cephClusterSpec.placement.all field.
Added tolerations to the PodTemplateSpec of the
CrashCollector Pruner CronJob.
Signed-off-by: Oded Viner <oviner@redhat.com>
Fix label selector for watch-active mgr container: during update active mgr label,
add missed app=rook-ceph-mgr label, to avoid setting label for mon pod.
Resolves: #15208
Signed-off-by: Denis Egorenko degorenko@mirantis.com
The ci was using a pretty old version og golangci-lint.
This updates to the latest version.
Additionally, it silences some
gosec integer conversion overflow false positves
and fixes some real errors of this category
and string format errors found by golangci-lint, while at it.
Co-authored-by: Blaine Gardner <b.blaine.gardner@gmail.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Signed-off-by: Michael Adam <obnox@samba.org>
Fix the issue when using with external ceph cluster, ceph exporter
secret isn't created which cause ceph-exporter fail to run. By
default this option is false means ceph-exporter will run for
external cluster.
monitoring
metricsDisabled: false
This patch also adds metricsDisabled option to helm values.yaml.
fixes#14275
Signed-off-by: Yaguang Tang <heut20008@gmail.com>
With the v19.2.0 release being out for some time now,
update the default version to be deployed with Rook
as v19.2.0.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
GetOSDResources method was populating
wrong device class to get the osd
resources
This fix will fix the deviceClass label
for the GetOSDResources function
Signed-off-by: parth-gr <partharora1010@gmail.com>
currently device class and device type
labels were clubbed with eachother
create a seperate label, as these two labels
can be seperate
Signed-off-by: parth-gr <partharora1010@gmail.com>
When node topology labels with duplicate values are detected, rook silently discards all but one label.
This adds a warning informing administrators of this, allowing for easier identification of labeling issues.
Signed-off-by: Fred Heinecke <fred.heinecke@yahoo.com>
If host path keyring file exists and created during osd-prepare but not
imported to ceph auth then rook-ceph-osd activate init container will
create new keyring in ceph auth by ceph auth get-or-create command. To
avoid keyrings difference in host path and in ceph auth, consider
importing key from host path keyring file if no keyring found in ceph
auth.
Resolves: #14825
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
Previously arbiter monitors were allowed to failover in supported versions
of Ceph. This behaviour regressed. Since all supported versions of Ceph now
support this, we no longer need to check this at all.
Signed-off-by: Grant Fleming <gfleming@evroc.com>
Resolves CIS benchmark 5.2.8 rule by adding capabilities with
explicitly defined empty "add" list (where it needed) and with
NET_RAW capability in "drop" list.
5.2.8 Minimize the admission of containers with added capabilities
Containers must drop the `NET_RAW` capability and are not permitted
to add back any capabilities.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
For nodes that are explicitly requested for deploying
OSDs, they will be skipped temporarily if not
schedulable or not ready. A future reconcile is
expected to schedule them. Allow the OSDs to be
scheduled even on these nodes that are temporarily
down, even if it blocks the reconcile from completing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The mon canaries may be created even when the mon daemons
are not created thereafter during the integration tests.
Therefore, the integration tests need to also query a label
specific to the mon daemon so the canaries are not a distraction
to the test.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The /run/udev directory is now mounted in the OSD init container. This change
is necessary because `ceph-volume activate` needs access to /run/udev [1] in order
to properly handle devices. Without this mount, `ceph-volume activate` could
fail to discover devices during OSD initialization.
[1] https://github.com/ceph/ceph/commit/c2e8c295dc46f1bd74861feb163d9d82f8bb5509
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Finish the process of deprecating holder pods by removing Rook's ability
to deploy them. The intent of this change is to make the most
superficial changes possible to accomplish this. There are still
remnants of code in Rook (particularly the CSI controller) that helped
configure or deploy holder pods. Due to the risk of breaking some
features, cleanup work of hose remnants will be deferred for future
work.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
MustDelete DeleteOption was set to `true` when deleting
osd-repalce-config configMap. This cm won't existing after
all the OSDs get migrated. So deletion of the non-existent
cm will result in error. This PR removes `mustDelete: true` from
the DeletionOption when deleting osd-replace-config configmap
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The exporter pods were being counted as a ceph pod,
so once an exporter pod was added to a node, it would never
be removed since the controller thinks the exporter pod
itself is a ceph pod for which the exporter must be present.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The v1beta1 cron jobs have been obsolete since K8s 1.21,
and Rook has not supported that version of K8s
for many moons, so we can remove the obsolete code
for the handling of v1beta1 cron jobs for crash pruning.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Disabling the RBD and CephFS fencing
in Rook for now as its having bugs where
Rook is blocklisting wrong IP address
due to timing issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Resolves CIS benchmark rule 5.7.3, Pods part. SecurityContext
should be explicitly defined in pod level of Pod spec section.
It is sufficient to specify empty securityContext to satisfy
CIS 5.7.3 rule.
5.7.3 Apply Security Context to Your Pods and Containers
When designing your containers and pods, make sure
that you configure the security context for your pods,
containers, and volumes.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
If the mon failover is in progress, ensure the removal
of an extra mon deployment is skipped since that code
path only has one mon in the list for the mon that was
just newly started. The extra mon was erroneously removing
a random mon in that case, followed immediately by the mon
failover completing and removing the expected failed mon,
and potentially causing mon quroum loss.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This adds an operator config setting ROOK_REVISION_HISTORY_LIMIT
defaulting to kubernetes'value for RevisionHistoryLimit.
If configured, the provided value will be used as RevisionHistoryLimit
for all Deployments rook creates.
Fixes: #12722
Signed-off-by: Michael Adam <obnox@samba.org>
The crash collector daemon had missed the ability to
add custom annotations via the cephcluster CR, as already
supported for all the other daemons.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>