Named containerPort is a useful feature when generating dynamic scrape
targets with prometheus.
This makes it clear that the containerPort is intended for metrics
scraping.
The name matches that which is on the mgr deployment.
Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
Adds .spec.cephConfigFromSecret to CephCluster for loading
Ceph config parameters from a Kubernetes Secret.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
In case of multicluster service, we need to always delete the mon-canary
pods even if there is any failure while exporting the service, else
multiple mon canary pods will get created.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
When finalizers are added to the CRs, a follow-up reconcile
will be triggered due to the increased generation on the CR.
Therefore, abort the initial reconcile when adding the finalizer,
and allow the follow-up reconcile to complete the configuration.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
All existing controller runtime watches are converted to use "typed"
handlers and predicates instead of operating on `client.Object`. The
intent is to be bug for bug equivalent with the existing logic while
replacing run time type assertions and switch statements with compile
time type constraints and type casts. In several cases, functions using
assertions were split up such that each function only handles a single
Kind at a time. It is hoped that this will improve readability and
maintainability while facilitating future refactoring such as migrating
some watches to using IndexFields.
Of particular note is that the massive switch statement in
`WatchControllerPredicate()` from
`pkg/operator/ceph/controller/predicate.go` has been replaced with
generics, reflection, and splitting the obc logic into its own predicate
function. There are still many helper functions operating on
`client.Object`. These were not updated unless required by the compiler
in order to limit the size of this change. The type safety of these
funcs should be improved as followup work.
It is strongly suggested that going forward, handlers and predicates
only handle a single Kind (generic or not) and that switches / type
assertions are heavily discouraged or forbidden. This PR removed all
but a single switch statement in a predicate, which should be addressed
in future work.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Fixes an ordering issue where spec.cephConfig settings failed to apply
because the manager wasn't yet running. Moved the config logic from
postMonStartupActions() to a new postMgrStartupActions() to ensure
the mgr is ready before applying config, allowing successful reconciliation.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
The change creates and manages an Endpoints resource
with the current Ceph monitor (mon) IPs, allowing
clients to resolve mon IPs via DNS without relying
on the rook-ceph-mon-endpoints ConfigMap.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
One ore more OSDs can be down and PGs can still be active+clean (data
was rebalanced to other available OSDs). This PR sets
maxunavailable=1+downOSDs to allow one healthy to be drained.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
today, csi operator cephConnection and client profile
CR's where not getting created in external mode. So, let's
start creating those.
Signed-off-by: subhamkrai <srai@redhat.com>
Adds the missing exporter container port to enable
proper Prometheus scraping and filtering.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
Implements #14733. Allows to set IDs of external mons to
Cluster CRD. Rook will not remove external mons from quorum
and will add external mon addresses to mon endpoints.
Use-case for external mon is to maintain quorum for 2-AZ
k8s cluster in case of zone outage.
Signed-off-by: Artem Torubarov <artem.torubarov@clyso.com>
There is a race condition in ceph v19 at startup of a
stretch cluster where if the .mgr pool is created by
the mgr pod after stretch mode is entered, the
devicehealth mgr module will crash and cause a
ceph HEALTH_ERR. The error then causes rook reconcile
of all CRs to be skipped because Rook is assuming
admin intervention is needed to continue. The fix
is to wait until the .mgr pool is created by the mgr
pod before entering stretch mode.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator settings loaded from the configmap have proven
inefficient for load time and frequently checking the configmap.
To avoid this ineffenciency, the configmap is only loaded once
each time it is created or updated. The values in the configmap
are applied as environment variables, which then are very efficient
to query throughout the various controllers, without needing
to be concerned about loading the configmap again.
Co-authored-by: Dmitry Mishin <dmitry.mishin@gmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a locally mounted volume is blocked on reads because this OSD is
down, lvs will deadlock trying to read from it. To avoid this, tell LVM
to filter out RBD volumes from its scanning process.
Signed-off-by: Steven Fackler <sfackler@gmail.com>
Some ceph.conf settings need to be unique per node
depending on the hardware. The ceph.conf overrides
can now be customized per-node by creating a node-specific
configmap that will be loaded for all OSDs and OSD prepare
jobs on that node, instead of the default ceph.conf
that is loaded from the rook-config-override configmap.
The node-specific configmaps must have the label
node.config.rook.io/osd, and be named as:
rook-config-override-<hostname>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This PR addresses an issue where the CrashCollector Pruner
CronJob pods were stuck in a Pending state due to missing
tolerations in their PodTemplateSpec.
These tolerations were not being propagated from the
cephClusterSpec.placement.all field.
Added tolerations to the PodTemplateSpec of the
CrashCollector Pruner CronJob.
Signed-off-by: Oded Viner <oviner@redhat.com>
Fix label selector for watch-active mgr container: during update active mgr label,
add missed app=rook-ceph-mgr label, to avoid setting label for mon pod.
Resolves: #15208
Signed-off-by: Denis Egorenko degorenko@mirantis.com
The ci was using a pretty old version og golangci-lint.
This updates to the latest version.
Additionally, it silences some
gosec integer conversion overflow false positves
and fixes some real errors of this category
and string format errors found by golangci-lint, while at it.
Co-authored-by: Blaine Gardner <b.blaine.gardner@gmail.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Signed-off-by: Michael Adam <obnox@samba.org>
Fix the issue when using with external ceph cluster, ceph exporter
secret isn't created which cause ceph-exporter fail to run. By
default this option is false means ceph-exporter will run for
external cluster.
monitoring
metricsDisabled: false
This patch also adds metricsDisabled option to helm values.yaml.
fixes#14275
Signed-off-by: Yaguang Tang <heut20008@gmail.com>
With the v19.2.0 release being out for some time now,
update the default version to be deployed with Rook
as v19.2.0.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
GetOSDResources method was populating
wrong device class to get the osd
resources
This fix will fix the deviceClass label
for the GetOSDResources function
Signed-off-by: parth-gr <partharora1010@gmail.com>
currently device class and device type
labels were clubbed with eachother
create a seperate label, as these two labels
can be seperate
Signed-off-by: parth-gr <partharora1010@gmail.com>
When node topology labels with duplicate values are detected, rook silently discards all but one label.
This adds a warning informing administrators of this, allowing for easier identification of labeling issues.
Signed-off-by: Fred Heinecke <fred.heinecke@yahoo.com>
If host path keyring file exists and created during osd-prepare but not
imported to ceph auth then rook-ceph-osd activate init container will
create new keyring in ceph auth by ceph auth get-or-create command. To
avoid keyrings difference in host path and in ceph auth, consider
importing key from host path keyring file if no keyring found in ceph
auth.
Resolves: #14825
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
Previously arbiter monitors were allowed to failover in supported versions
of Ceph. This behaviour regressed. Since all supported versions of Ceph now
support this, we no longer need to check this at all.
Signed-off-by: Grant Fleming <gfleming@evroc.com>
Resolves CIS benchmark 5.2.8 rule by adding capabilities with
explicitly defined empty "add" list (where it needed) and with
NET_RAW capability in "drop" list.
5.2.8 Minimize the admission of containers with added capabilities
Containers must drop the `NET_RAW` capability and are not permitted
to add back any capabilities.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
For nodes that are explicitly requested for deploying
OSDs, they will be skipped temporarily if not
schedulable or not ready. A future reconcile is
expected to schedule them. Allow the OSDs to be
scheduled even on these nodes that are temporarily
down, even if it blocks the reconcile from completing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The mon canaries may be created even when the mon daemons
are not created thereafter during the integration tests.
Therefore, the integration tests need to also query a label
specific to the mon daemon so the canaries are not a distraction
to the test.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The /run/udev directory is now mounted in the OSD init container. This change
is necessary because `ceph-volume activate` needs access to /run/udev [1] in order
to properly handle devices. Without this mount, `ceph-volume activate` could
fail to discover devices during OSD initialization.
[1] https://github.com/ceph/ceph/commit/c2e8c295dc46f1bd74861feb163d9d82f8bb5509
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Finish the process of deprecating holder pods by removing Rook's ability
to deploy them. The intent of this change is to make the most
superficial changes possible to accomplish this. There are still
remnants of code in Rook (particularly the CSI controller) that helped
configure or deploy holder pods. Due to the risk of breaking some
features, cleanup work of hose remnants will be deferred for future
work.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Blaine Gardner
Justin Cichra
Artem Torubarov
Patryk Rostkowski
Santosh Pillai
Travis Nielsen
Joshua Hoblitt
subham rai
subhamkrai
Artem Torubarov
Steven Fackler
Oded Viner
Denis Egorenko
Yaguang Tang
parth-gr
parth-gr
Fred Heinecke
Peter Razumovsky
Grant Fleming
Satoru Takeuchi
Guillaume Abrioux