Rook uses global or node-local device class configuration if device-level
configurations doesn't exist. However, after introducing device-class
level resource configuration, rook set the default value of device class.
So global or node-level device class configuration has never been used
after that.
Closes: https://github.com/rook/rook/issues/11871
Closes: https://github.com/rook/rook/issues/11826
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This commit adds functionality to be able to rotate
key encryption key of encrypted PVC backed OSDs.
Necessary changes such as adding update functionality
to kms and rbac changes are made as well.
Signed-off-by: Rakshith R <rar@redhat.com>
This commit adds util functions to support encryption key rotation
such as RemoveEncryptionKeySlot, AddEncryptionKey and
ensureEncryptionKey.
Signed-off-by: Rakshith R <rar@redhat.com>
few functions got change as they were deprecated
for ex: ioutil.Readfile change to os.Readfile
ioutil.TempFile change to os.CreateTemp
And fixed golang-ci-lint-issues
Signed-off-by: parth-gr <paarora@redhat.com>
Mons that are configured on msgr1 will be using the v1 port
in the csi driver since only one port is allowed in the configmap.
To allow mons to be started on v2 while keeping quorum with the
v1 mons, the v1 mons must continue running on the v1 port.
So even if msgr2 is required, allow existing v1 mons to continue
running on that port.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
It removes the condition that select the devices name prefixed with /dev/mapper as it is causing the ceph-volume command to fail.
Closes: #11301
Signed-off-by: ausias-armesto <ausiasarmesto@gmail.com>
Multipath devices may have an FSTYPE=mpath_member even though
a valid device for creating an OSD. The device will be allowed
in this case when creating the OSD on a PV since it is a device
specifically provisioned for the OSD.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During osd-prepare-job,
The encrypted device is found to be closed in some cases when
the OSD deployment has been removed manually accompanied
by any of following cases:
- node reboot
- csi managed PVC being unmounted etc
while re-opening the block to re-hydrate the OSDInfo,
the dm device `<pvc-name>-block-dmcrypt` clashes with
the one used by OSD pod which is stale by now.
This commit adds cmd to remove this stale dm device.
Error with output "No such device" is ignored.
Signed-off-by: Rakshith R <rar@redhat.com>
The ceph.conf was being written with 0x444 permissions, which means
the file could not be updated if it already exists. Specifically, when
an osd is purged, if the operation is retried multiple times, only
the first operation was succeeding. The subsequent requests to purge
an osd were failing due to the config already existing. If we make
the config writeable with 0x644 then the operator can always write the file.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit implements this corner case during osd-prepare job.
```
The encrypted block is not opened, this is an extreme corner case
The OSD deployment has been removed manually AND the node rebooted
So we need to re-open the block to re-hydrate the OSDInfo.
Handling this case would mean, writing the encryption key on a
temporary file, then call luksOpen to open the encrypted block and
then call ceph-volume to list against the opened encrypted block.
We don't implement this, yet and return an error.
```
When underlying PVC for osd are CSI provisioned, the encrypted device
is closed when PVC is unmounted due to osd pod being deleted.
Therefore, this may occur more frequently and needs to be handled.
This commit implements the fix for the same.
Signed-off-by: Rakshith R <rar@redhat.com>
We can use LVM logical volumes and loop devices as OSD iff we specify
them in the `devices` field. If we use filters like `deviceFilter`,
these devices are not picked with a log message which let users
to specify the exact device name. It's better to mention which field
users should use in this case.
Closes: https://github.com/rook/rook/issues/11353
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.
This feature is intended to be used for testing purposes only.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
The mons that are out of quorum may cause ceph commands to timeout
or fail unnecessarily trying to connect to a mon that is no longer
online. Now the mon health check will update the mon endpoints configmap
when a mon is detected out of quorum. This also means that if the
operator is restarted during a mon failover, the failed mon will no
longer remain in the ceph.conf, thus allowing the quorum to be more
likely to respond to the mons that are still in quorum.
The update for mons out of quorum only applies if other mons are in
quorum. If quorum is down, the configmap will not keep track of the offline
mons since too many are offline.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the osd is purged, properly log the host name
that is being purged and whether the host was
removed from the crush map.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
currently, osdSafeToDestroy check is done before
marking osd out which prevents ceph from rebalancing.
With this changes, we are moving check for
osdSafeToDestroy after osd is marked out.
Signed-off-by: subhamkrai <srai@redhat.com>
Run the ganesha-rados-grace command in a remote pod when multus
networking is enabled.
Signed-off-by: parth-gr <paarora@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This commit changes cryptographic length for keys created
for kmip kms to 256 from 128, since it is more secure.
Signed-off-by: Rakshith R <rar@redhat.com>
LVM logical volumes are not picked by `deviceFilter`, `devicePathFilter`,
and `useAllDevices`. It's to prevent old version's users from creating
lv-backed osd during upgrading rook by accident.
It's better to clarify lvs are not picked by filters" in docs and
suggest users to use exact device name instead.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
1. Eliminate possible memory leaks of timer.
2. Eliminate duplicated events between udev events and kernel events.
3. Empty struct have the lowest size.
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
OSD on LV with metadata device is not supported. So we once we made
a fix to disallow this configuration. However, it's insuffucient
because we forgot to reset `matched` flag.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The cluster-wide encryption feature now
supports KMIP (Key Management Interoperability Protocol)
kms.
KMIP is an extensible communication protocol that defines
message formats for the manipulation of cryptographic keys
on a key management server.
For more information, refer:
https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol
Signed-off-by: Rakshith R <rar@redhat.com>
The tool gofmt in go 1.19 requires certain formatting
in the comments section for better rendering.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The RGW support server side encryption with help of s3 protocol, till
now the `sse:kms` was support in which keys will be provided by the user
and but it will be saved in external management service like vault. Now
the support for `sse:s3` is added so the entire encryption key
management is performed by RGW itsels.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Travis Nielsen
Miles Simpson
Satoru Takeuchi
parth-gr
Rakshith R
Blaine Gardner
zhucan
ausias-armesto
Shinya Hayashi
subhamkrai
dkeven
Liang Zheng
taki-d
Jiffin Tony Thottan
Eng Zer Jun