The key file deletion process is in the shell script commonly used by
all of encryption-open, encryption-open-metadata, and
encryption-open-wal init containers. The key file is deleted at the
encryption-open init container and encryption-open-metadata and
encryption-open-wal init containers are failed to open the key file.
The key file is in the /etc/ceph folder. Unless that folder is shared,
the key file anyway won't be available in the other init containers
even if it is not deleted by these init containers. And it will
naturally anyway be deleted after the init containers are
completed. So The key file deletion process in shell scripts is
unnecessary.
Fixes: #13737
Signed-off-by: Yuma Ogami <yuma-ogami@cybozu.co.jp>
(cherry picked from commit cdd655ed3f)
the labels listed under the 'monitoring' section are currently only
being applied to the rook-ceph-mgr ServiceMonitor. This change extends
those labels to also include the rook-ceph-exporter ServiceMonitor.
Fixes: https://github.com/rook/rook/issues/13774
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 10e26709c4)
With the release of Reef v18.2.2 we update the default
recommended version of Ceph to fix the prometheus issues
seen with v18.2.1
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 026d745f8a)
cephFs fencing needs more testing and bug fixes to make
it in production. We tried multiple fixes but still it need
more work. So, removing the cephfs networkFence realted code
will rever this commit when we start working on this again.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit bd4250f56e)
earlier, we were only checking the specs of networkFence to see if
unfencing is done before proceeding with deletion but we should also
check the status of networkFence before proceeding with deletion.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 0ace7cf8cf)
If the spec.mon.count is 5 or more, then create
mon pdbs with maxUnavailable count as 2. We can
afford to have more than 1 mons down while
draining the nodes in this case.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit a4866b4b06)
Reduce CRD size by removing some descriptions for things that
(a) are long, (b) are repeated, and (c) are commong kubernetes types.
Remove the `Status` field from VolumeClaimTemplates (a.k.a. PVCs) in
types.go to remove all unnecessary status-related fields.
Use `yq` to go through generated `crds.yaml` to strip out descriptions
for well-known k8s types that are long and repeated where we can't
remove them from types.go. This is mostly placement descriptions and
volumeSource descriptions.
The overall `crds.yaml` file is reduced to less than 60% of its original
size.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Co-authored-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit bd7477ec61)
# Conflicts:
# deploy/charts/rook-ceph/templates/resources.yaml
# deploy/examples/crds.yaml
retry other cephfs/rbd pvc if there
are no active clients found on ceph cluster.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit ca21bd9715)
to keep the code consistent and to propogate
more details about error rook uses error wraping.
updating current code whereever its required to
wrap the details.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit eb97390df2)
The cephfs PVC might exist on the kubernetes
node object but due to some timing issues the
ip might not be visible on the ceph cluster or
the client might already got evicted or disconnected
from ceph cluster. In this case we will not be
able to get IP details for the subvolume and we dont
have any check for empty ip's in the code and rook
tries to create NetworkFence CR with empty Ip's
and the NetworkFence will get moved to the Failed state.
This PR adds the necessary check and logging to prevent
this one.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 0e115c44bf)
since networkFence is a cluster-based resource so that we don't need
the namespace and ownerReferences as it cause garbage-collector errors.
Also, now we create the networkFence with clusteUID label so when doing
cleanup we match the cephCluster uid and networkFence label clusterUID.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 5bff860380)
Mgr sidecar was using a privileged security context because it needed
to create the /var/lib/rook directory, generate the config on it, and
then copy it to the /etc/ceph directory. This change creates empty
directories on /var/lib/rook and /etc/ceph so the mgr sidecar can use
them, removing the need for using a privileged security context.
Fixes: https://github.com/rook/rook/issues/13719
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit db4b3322fa)
since networkFence is a cluster-based resource so that we don't need
the namespace and ownerReferences as it cause garbage-collector errors.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 2ac418a3b2)
this commit updates the name of network fence CR to
node name + driver name, which will allow the creation of
multiple network fence per drivers on the same node, and
modifies the unit test and deletion procedure for the
same.
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
(cherry picked from commit c2e3cf367c)
The exporter service and service monitor should not
be deleted if an individual daemon is being removed
from a node. As long as the exporter is enabled with
v18 and newer, the exporter service and service monitor
should exist. Clusters were seeing their metrics stop
collection when the exporter service and service monitor
would disappear, especially when multiple clusters are
configured.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit a1fc70f2fc)
previous code was calling the following command to create the
dashboard user:
> ceph dashboard ac-user-create <username> -i <password-file>
this works the first time when the user doesn't exist, however subsequent
calls for example when the password has changed or when the secret was
deleted from k8s, will not take place as this command will succeeded
because the user already exists but the password is not updated in
this case. The fix consists in adding an explicit call to the command:
> ceph dashboard ac-user-set-password <username> -i <password-file>
which ensures the password is updated correctly in the access control db
Fixes: https://github.com/rook/rook/issues/13594
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit cb028f3a28)
we dont set removeMgrDaemonConfiguration to true unless all the
delete operations went successfull.
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 39c06d2fa5)
previously, the dashboard parameters supported by Rook were stored in the
daemon configuration section (mgr.X, for example). This differs from
Cephadm-based deployments, where all configurations are stored in the
global mgr configuration section. This variance could result in
configuration mismatches between the active and standby dashboards.
Furthermore, all Ceph dashboard documentation exclusively points to
the global mgr configuration section and makes no use of individual
daemons sections.
Fixes: https://github.com/rook/rook/issues/13577
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit 57e3937d40)
If migration of the OSDs is enabled, operator should wait
for the pgs to be clean after the all the OSDs have reconciled.
This will help add the exponential backoff for the operator
restarts when OSD migration is needed but PGs are still rebalancing the
data after last OSD migration.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 43d3bf147a)
Since v18.2.1 is the default recommendation for the ceph version,
update all the docs and examples to that version.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 56a3370d83)
Allow changing spec.network.hostNetwork settings on a running cluster
by failing over the mons
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 42ea2897cf)
This PR fails over mon when the mon path is changed from hostPath to PVC or vice versa
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 4b6fd89365)
This implements the "Ceph Config via Ceph Cluster CRD" design document
as a `cephConfig:` structure on the CRD.
This also fixes the `yq` commands used to manipulate the
`cluster-test.yaml` that caused CI issues for this PR and potentially
unknowingly others.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
(cherry picked from commit 4ed35d6bd6)
Fixes: #13167
Previously, the mgr did not honor the flag
ContinueUpgradeAfterChecksEvenIfNotHealthy
from the cluster spec. Only osd, mds, and rgw did.
To render the update behavior correct and complete across the daemons, this
change implements the honoring of the flag for the mgr.
Signed-off-by: Michael Adam <obnox@samba.org>
This commit adds new CSIDriverOptions section in
cephCluster CR. This section contains settings
for read affinity and kernel+fuse Mount options
These settings will be injected directly into
rook-ceph-csi-config cm to be applicable per
ceph cluster.
Signed-off-by: Rakshith R <rar@redhat.com>
Restarting the exporter using RollingRelease causes a race condition,
that results in exporter crashing and the ceph health to show a warning.
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
ceph dashboard uses radosgw-admin for certain tasks that
aren't accessible via the rgw REST API. Due to the absence of
a valid ceph.conf file at /etc/ceph/ceph.conf within the mgr pod,
radosgw-admin fails to operate, resulting in 500 errors across
various 'Object Gateway' views on the dashboard. This change
adds CEPH_ARGS environment variable to the mgr pod enabling
its propagation and utilization by the dashboard/radosgw-admin
for executing rgw commands.
closes: https://github.com/rook/rook/issues/13255
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
This patch adds `pgHealthyRegex` field to DisruptionManagementSpec.
`pgHealthyRegex` is a regular expression that is used to determine which
PG states should be considered healthy. The default value of
`pgHealthyRegex` is:
^(active\+clean|active\+clean\+scrubbing|active\+clean\+scrubbing\+deep)$
which is effectively the same as before.
Signed-off-by: Ryotaro Banno <ryotaro.banno@gmail.com>
Similar to the ceph crash collector daemon that generates a keyring with more
restrictive privileges, the exporter should also generate and use a more limited keyring.
Signed-off-by: avanthakkar <avanjohn@gmail.com>
During certain maintenance tasks the admin will own running
operations on the ceph mgr, rgw, mds and rbd-mirror daemons
and the operator should not interfere with those operations.
Co-authored-by: gauravsitlani <gaurav.sitlani@live.com>
Signed-off-by: subhamkrai <srai@redhat.com>
when creating networkFence, rbd command was loading
admin config and hence running rbd command use client.admin
in case of external cluster also. With this commit instead
of client.admin user it will use what is being passed to
config.
Signed-off-by: subhamkrai <srai@redhat.com>
This reverts commit fab23d3407.
The mgr requires rw access for the cron job that collects
the crashes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Yuma Ogami
Travis Nielsen
Redouane Kachach
mergify[bot]
subhamkrai
sp98
Madhu Rajanna
Riya Singhal
Satoru Takeuchi
Alexander Trost
Michael Adam
Rakshith R
Divyansh Kamboj
Ryotaro Banno
avanthakkar
gauravsitlani
parth-gr
Blaine Gardner