This new setting is of Boolean type and defaults to "false".
When set to "true", it changes the behavior of the
rook operator to
nable host network on all pods created by the cephcluster controller
new method to check the setting: opcontroller.EnForceHostNetwork()
Signed-off-by: Michael Adam <obnox@samba.org>
Allow RGW users to mount arbitrary volumes to RGW pods. This follows the
pattern that was established for NFS to support SSSD and LDAP, and
reuses much of the same code.
This opens the door wider for advanced users to take advantage of some
Ceph RGW features that Rook doesn't have first class support for.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
The endpoint property honorLabels: true should be set so that
different rgw instances can show separate prometheus
values instead of being aggregated.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In the primary CSI reconcile, ensure the CSI config map
(`rook-ceph-csi-config`) has the correct owner info.
This corrects any pre-existing config maps that might have incorrect
owner info, which has observed to include references to CephClusters.
The config map should only have a single reference, and it should refer
to the operator deployment.
If any existing Rook clusters have a CSI config map which has a
CephCluster as an owner, this change will ensure that the config map is
not deleted when the CephCluster is deleted. This is especially
important for any environments with multiple CephClusters installed.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Loop variables cannot be reliably uses since they will
change with each iteration. Update these loop variable
uses to be safe by indexing the slice rather than
using the loop variable directly.
Also suppress the linter issues for passwords used
in tests.
Signed-off-by: travisn <tnielsen@redhat.com>
As seen in https://github.com/rook/rook/issues/1988, it's a common
mistake to configure rook nodes with names that don't match Kubernetes's
node label. This PR prints more detailed message to help debugging
problem.
Signed-off-by: Bin Wang <bin.wang@mail.binwang.me>
Add break according to review comment
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Update log message according to review comment
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Change how Rook detects network CIDRs for Multus networks. The IPAM
configuration is only defined as an arbitrary string JSON blob with a
"type" field and nothing more. Rook's detection of CIDRs for whereabouts
had already grown out of date since the initial implementation.
Additionally, Rook did not support DHCP IPAM, which is a reasonable
choice for users. And more, Rook did not support CNI plugin chaining,
which further complicates NADs. Based on the CNI spec, network chaning
can result in any changes to network CIDRs from the first-given plugin.
All these problems make it more and more difficult for Rook to support
Multus by inspecting the NAD itself to predict network CIDRs. Instead,
it is better for Rook to treat the CNI process as a black box. To
preserve legacy functionality of auto-detecting networks and to make
that as robust as possible, change to a canary-style architecture like
that used for Ceph mons, from which Rook will detect the network CIDRs
if possible.
Also allow users to specify overrides for CIDR ranges. This allows Rook
to still support esoteric and unexpected NAD or network configurations
where a CIDR range is not detectable or where the range detected would
be incomplete. Because it may be impossible for Rook to understand the
network CIDRs wholistically while residing only on a portion of the
network, this feature should have been present from Multus's inception.
Improving CIDR auto-detection and allowing users to specify overrides
for auto-detected CIDRs rounds out Rook's Multus support for CephCluster
(core/RADOS) installations. No further architectural changes should be
needed for CephClusters as regards application of public/cluster network
CIDRs for Multus networks.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
ceph-exporter's ServiceMonitor and Service CRD contain different port
name, which results in no metrics being collected by prometheus.
this commit makes GetCephMonitor configurable, which we use to set
consistent port names in ServiceMonitor and Service for ceph-exporter.
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk
Signed-off-by: sp98 <sapillai@redhat.com>
earlier, we're always logging when `GetValue` method is
called to get operator config values which somettime
spamming the logs for example `ROOK_WATCH_FOR_NODE_FAILURE`
keep logging every time 5 min.
Now, we'll store the settings in the map and log only when
map key value is changed.
Signed-off-by: subhamkrai <srai@redhat.com>
If the ceph version does not suppport the exporter, the operator
will attempt to delete the service monitor related to the exporter
to ensure it does not exist for a node. If the expected rbac does
not exist, this will cause unnecessary errors since there would anyway
be no service monitor to delete. So we ignore the error of deleting
the service monitor for the exporter.
The context is also passed to the exporter so a new kubeconfig does
not need to be initiated and cause unnecessary logging about
invalid options for the config.
Signed-off-by: travisn <tnielsen@redhat.com>
Cleanup exporter daemon even if ceph version is not supported along with other resources like
metrics service and service monitor.
Signed-off-by: avanthakkar <avanjohn@gmail.com>
When deploying a cluster with Helm Chart, if the namespace is not the default value `rook-ceph`, and if the monitoring feature is enabled, then the generated ServiceMonitor's `rook_cluster` selector now follows the namespace, not the hard-coded value `rook-ceph`.
Signed-off-by: Ho Kim <ho.kim@ulagbulag.io>
After an update, the mgr pod will be restarted. Since the
readiness probe will indicate the standby mgr pod is never
ready, we need to check simply for the running mgr pod
and not whether or not it is ready.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
few functions got change as they were deprecated
for ex: ioutil.Readfile change to os.Readfile
ioutil.TempFile change to os.CreateTemp
And fixed golang-ci-lint-issues
Signed-off-by: parth-gr <paarora@redhat.com>
This fixes the the MultiClusterDeploySuite CI failure. The operator
is timing out waiting for the mgr deployments to be
ready, according to the WaitForDeploymentToStart() method. After
the reconcile times out after about five minutes, the next
reconcile succeeds since the wait is only done for new mgr
deployments.
Closes: https://github.com/rook/rook/issues/11685
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Ceph will use a random nonce for the msgr daemons as long as
they are running as PID 1 or have the environment variable
CEPH_USE_RANDOM_NONCE set to indicate running in a containerized
environment. While the Ceph daemons in Rook are expected to
run in PID 1, there are some environments where the container
runtime may choose to run something else such as
/usr/bin/pod as PID 1. The side effect then is that ceph uses
the pid as the nonce and causes problems when the daemons
restart with the same PID.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Introduce a new env variable ROOK_CSI_IMAGE_PULL_POLICY in rook operator configmap which should be used to
customize the imagePullPolicy for the csi driver and imagePullPolicy property in cephVersionSpec for ceph pods.
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
The sssd.conf may refer to additional files, like a CA bundle or TLS
certificates. These files need to be made available in the SSSD sidecar.
With the new `sssdGenericFiles` reference to a VolumeSource and a
`MountPath`, a provided `sssd.conf` can use the additional files.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This option open-up volumes and volumemounts
for user customization of the rbd,cephfs,nfs
plugin daemonset.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The tool gofmt in go 1.19 requires certain formatting
in the comments section for better rendering.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Adds support for valid arbitrary NodeAffinity JSON input by the user while
maintaining backward compatibility with the older parsing code.
Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>
The prometheus rules had been previously created if the cephcluster CR
setting monitoring.enabled was set to true. The rules were not customizable
and therefore not flexible enough. Now the rules are installed by the helm
chart. To customize the rules, a post-processor can be applied to the helm
chart.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
adding observedGeneration field in the cephcluster cr
status for having better control on reconciling,
as observedGeneration field will be updated by the controller
Closes: https://github.com/rook/rook/issues/9673
Signed-off-by: parth-gr <paarora@redhat.com>
When a node is being skipped for creation of OSDs, log a message
that indicates whether it was from the node being unschedulable,
the node is not ready, or the node does not meet the placement
criteria specified in the cluster CR.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The merging of OSD resource limits and requests assumed
that only the cpu and memory resources needed to be merged.
It is also possible to set custom resource properties such
as intel.com/sriov_net_in: '1' for use with multus, so
the merging needs to be more general.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Michael Adam
Blaine Gardner
subhamkrai
Travis Nielsen
piao
Redouane Kachach
parth-gr
dependabot[bot]
Bin WangandTravis Nielsen
guoguangwu
Divyansh Kamboj
sp98
Sheetal Pamecha
avanthakkar
Ho Kim
Rakshith R
parth-gr
Avan Thakkar
Niels de Vos
Madhu Rajanna
Eng Zer Jun
Josh Soref
Sébastien Han
Pranshu Srivastava
Blaine Gardner