Exporter service created by rook operator contains the following labels:
`app=rook-ceph-exporter,rook_cluster=<cluster-name>`
Label `ceph_daemon_id=exporter` is not there and therefore the service
monitor selector can not discover the exporter service.
This fix removes the `ceph_daemon_id=exporter` label from the exporter
service monitor selector.
Signed-off-by: Matej Feder <matej.feder@dnation.cloud>
Pick up the latest ceph prometheus rules from the ceph repo
found at https://github.com/ceph/ceph/blob/master/monitoring/ceph-mixin/prometheus_alerts.yml.
The updates include new rules for monitoring of ceph as well as
adjustments related to the Rook ceph deployment.
List of main adjustments:
- Alerts related to cephadm are excluded
- The PrometheusJobMissing alert is adjusted for the rook-ceph-mgr job, and the PrometheusJobExporterMissing alert is added
Signed-off-by: Matej Feder <matej.feder@dnation.cloud>
For an advanced network configuration, the mon endpoints
expected for the ceph public network may be different from
the public IP address that is set on the K8s node. When
host networking is enabled in rook, the IP address of the
node was only allowed previously. Now, the IP address
used for a mon when scheduled on a node can be determined
by setting an annotation on each node where mons can run:
network.rook.io/mon-ip: <ip-address>
This annotation must be set before the mons are created.
Mon endpoints cannot be changed after they are already
running.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
`make test` failed for me on MacOS with this error:
```console
$ make test
...
=== installing go-junit-report
tar: Error opening archive: Unrecognized archive format
make[2]: ***
$
```
So it seems the version v2.0.0 of go-junit-report has a broken source
tarball
Going to the latest version v2.1.0 fixes the problem for me.
Signed-off-by: Michael Adam <obnox@samba.org>
Add IPv6 support for multus validation tool. Also test that IPv6 support
works by specifying one of the NetAttachDefs with an IPv6 address range.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Multus requires that the 'ip' tool be installed in the Rook image.
Today, it is present in the upstream Ceph base image that is used, but
that is expected to change in the future. The Ceph project intends to
switch to CentOS 9 minimal images, and the 'ip' tool will likely be
removed from the base image at that point.
However, Rook CI has occasionally had issues with the current 'dnf
install' command when CentOS repos go down, or when there is otherwise
some problem. Because the package is already installed today, there is
no need to hamstring Rook builds when CentOS is having problems. But we
do want to make sure that Rook builds don't silently succeed in the
eventual future when 'ip' tool is removed from the Ceph image.
For now, replace the 'dnf install' with a check to verify that 'ip' tool
is installed, and add a shorter form of this note as a comment above it
to help Rook maintainers know how to resolve future 'ip' tool removal.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently we allow upgrade of other OSDs while migration
of OSD, due to change in backing store, is pending. This
will not work if the updated OSD ceph version does not support
the currently applied backing store. So rook will skip any
OSD upgrade if the OSD migration is pending.
Signed-off-by: sp98 <sapillai@redhat.com>
Add a note to remove LimitNOFILE setting in containerd service configuration.
This setting causes multiple issues such as slow `ceph status` commands (and failing healthchecks),
various timeouts, and mons failing/timing out.
Upstream containerd already removed the setting. NixOS hasn't incorporated the change.
This note can be removed once https://github.com/NixOS/nixpkgs/pull/313507 makes it to a stable release.
Signed-off-by: Onur Cakmak <occ@occ.me>
This PR adds a `settings` option in the `Module` struct
which can help users to futher customize the `balancer`
module with a differet mode.
Signed-off-by: sp98 <sapillai@redhat.com>
When IPv6-only network mode is enabled and when mons are configured to
listen only on msgrV2 port, Rook has to add the port to the mon
--public-bind-addr in order to force mons to stop listening on the
msgrV1 port. This can cause problems for IPv6 clusters where adding
':<port>' to the end of an IP addr looks like a valid IPv6 address.
In IPv6 mode, Rook needs to set this to '[<ip>]:<port>' with the
brackets to be a non-ambiguous IPv6 address.
In IPv4 mode, Rook needs to keep its existing behavior. IPv4 mode should
also continue to be the defualt assumption if the network config isn't
specified.
If 'dualStack: true' is specified, Rook is informed that the environment
is dual stack, but Rook doesn't know which IP family to expect from the
pod IP. In this scenario, it is safest for Rook not to apply the port,
since assuming either IPv4 or IPv6 format could be incorrect. In this
scenario, mons may still listen on msgrV1 ports, but this is preferable
to an environment where mons perpetually crash.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
currently node predicate watches for resourceVersion and heartbeat
changes, which are not critical things. So with the PR we will add
a early-exit condition for the predicate as an optimization and to
ignore considering for a reconcile of cephcluster if updates are
not crucial
closes: https://github.com/rook/rook/issues/14070
Signed-off-by: parth-gr <partharora1010@gmail.com>
Ceph-CSI support for fscrypt encryption of cephfs.
To achieve this commit add capability of mounting the
required `rook-ceph-csi-kms-config` configmap into
csi-cephfsplugin-provisioner and nodeplugin pods.
Further it modifies the ClusterRoles `cephfs-csi-nodeplugin` and
`cephfs-external-provisioner-runner` to grant privileges
required for reading encryption configuration and fetching
encryption secrets from either kubernetes secrets or
from a Key Management System (KMS).
These privileges are essential for the proper functioning of
ceph-csi-cephfs with fscrypt encryption.
The following privileges have been added:
- `secrets/get`: Allows reading of secrets for encryption.
- `configmaps/get`: Grants access to configuration maps,
this is used to read encryption configuration.
- `serviceaccounts/get`: Enables retrieval of service accounts for
authentication to KMS and for retrieving encryption secrets
stored there.
- `serviceaccounts/token/create`: Allows creation of service account tokens,
which are required for authenticating requests to KMS
when retrieving encryption secrets.
The commit also updated the csi documentation to include cephfs
in the encryption section, with examples updated accordingly.
Signed-off-by: NymanRobin <robin.nyman@est.tech>
SVG cleanup job uses the cephSubvolumeGroup env name
from spec.Name. But sometimes it can be empty. In that
case rook should use the CR name.
Signed-off-by: sp98 <sapillai@redhat.com>
dependabot[bot]
Travis Nielsen
Matej Feder
Blaine Gardner
Michael Adam
parth-gr
Jiffin Tony Thottan
subhamkrai
Santosh Pillai
Onur C. Cakmak
Brian Dowling
subham rai
NymanRobin
parth-gr