If the CephCluster spec is updated before the prevalidation check, then
Cluster.context still refers to the cancelled context of the operator.
This PR just updates the cluster context to use the latest operator
context.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
shouldFailoverMonImmediately has a nil pointer exception with mons on
PVC. Rook-ceph-mon-endpoint does not store node information of the mons
on PVC. So `mapping.Hostname` throws nill pointer
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
currently the extraction ipv6 was not correct
used the net package to extract the host ip from
the ipadress
Signed-off-by: parth-gr <partharora1010@gmail.com>
Enabling crush updates is false by default, and can be enabled
if set to true. The setting needs to be nullable in case
we need to differentiate between the value being set
to false or just not set, to know if the default should be
different. This is necessary for rook consumers who are
refererencing the pool struct and want to change the default,
rather than using rook's default value of false.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently, when there are down OSDs but all PGs are clean we add the
number of downed OSDs ot the main PDB maxUnavailable. For example if
OSDs 1,3,5 are down and all PGs are clean we create the following PDB:
```yaml
maxUnavailable: 4
selector:
matchLabels:
app: rook-ceph-osd
```
This is potentially dangerous as there can be race conditions which can
break the cluster, for example:
1. A new node is added to the cluster with 4 OSDs
2. Those OSDs are down as they have not started up and joined the cluster yet
3. All PGs are healthy as the new OSDs haven't started yet and they are new
4. The rook operator sets maxUnavailable to 5 on the OSD PDB
5. The new OSDs move from down to up as they start
6. Something in the cluster starts a disruption (eviction request)
7. Any 5 OSDs can be disrupted as maxUnavilable is still 5 and the downed OSDs are now up
8. If the 5 disrupted OSDs contain all the replicas of a PG it will cause the PG to go down
To better protect from this scenario, we can instead exclude the downed
OSDs from the main PDB instead of adding to maxUnavailable.
```yaml
maxUnavailable: 1
selector:
matchExpressions:
- key: app
operator: In
values: ["rook-ceph-osd"]
- key: osd
operator: NotIn
values: ["1", "3", "5"]
```
This allows drains to still process when there are downed OSDs and PGs
are clean without allowing the non-downed OSDs to be disrupted.
Signed-off-by: Elias Carter <elias@dropbox.com>
The ceph csi controller plugin did not have the capability to
set host networking, in case the host networking is being enforced
with ROOK_ENFORCE_HOST_NETWORK: true.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Fix an issue where keepPriorKeyCountMax would be counted differently
when rotation was currently in progress, versus when rotation was not in
progress.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
this commit adds multus network annotation to the
csi-operator node plugin and controller plugin pods
when multus is enabled in the ceph cluster.
Signed-off-by: subhamkrai <srai@redhat.com>
when the osd is created there is no need for
osd bootstrap keyring, so remove the keyring
and keyring file,
New keyring will be created if
new osd need to be provisoned
Signed-off-by: parth-gr <partharora1010@gmail.com>
Delete the client.boostrap.* keys. These keys are not created by rook so
it can't rotate them. This PR just delete these keys in the
postMonStartUpAction.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
ceph-volume creates osd-lockbox keys for the encryted osds. This PR
rotates those keys along with the OSD keys.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
this commit deletes csi operator CR's when
user wants to revert back to csi driver for
some reasons. This commits deletes CR's like
`clientProfile, cephConnection, driver,
operatorConfig, and clientProfileMapping`.
Signed-off-by: subhamkrai <srai@redhat.com>
this commit add watcher for cm `rook-ceph-operator-config`
to look for updated value of `ROOK_USE_CSI_OPERATOR` before
adding this watcher we had to restart rook operator pod
so that it reconciles and create cephConnection CR.
Signed-off-by: subhamkrai <srai@redhat.com>
If the ROOK_USE_CSI_OPERATOR setting is not found in the
rook-ceph-operator-config configmap, the default now
is to use the csi operator instead of the previous
csi driver
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the network is encrypted with msgr2, cephfs requires
the mount options to include ms_mode=secure. Now this
will be set by default if the mount options are not
already set in the cephcluster CR or the operator env
var CSI_CEPHFS_KERNEL_MOUNT_OPTIONS.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the cephfs kernel mount options are not specified in the
cephcluster.spec.csi CR settings, fall back to the setting
in the operator env var CSI_CEPHFS_KERNEL_MOUNT_OPTIONS.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The csi client profile CR needed to be updated if
the spec changed. Previously, the client profile was never
being updated because it was calling Update() on the
existing CR retreived from Get(), rather than with
the updated spec.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Implement CephX key rotation for Rook's client.admin user.
Admin user rotation is risky, so this has been tested extensively both
in unit tests as well as by manually injecting failures during runtime.
In testing, all failures were able to be recovered by the recovery
routine.
A mutex is also added to help ensure that two simultaneous admin key
rotation processes cannot be running simultaneously for any given
namespace. The mutex is tested in unit tests, and it was verified during
runtime via manual testing.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
The driver names were only being initialized in certain code paths
where the drivers were being started and stopped. However, another
code path relied on those names being set. Now, we initialize the
driver names every time the CSI params are initialized, instead of
initializing the names separately.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The csi read affinity requires the crush topology labels
to be specified in the cephconnection CR. If they are
not specified by the user, rook can generate the default
topology labels instead of expecting them.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Santosh Pillai
Travis Nielsen
dependabot[bot]
parth-gr
Blaine Gardner
subhamkrai
Elias Carter
cuiweixie
subham rai
Praveen M