We can use LVM logical volumes and loop devices as OSD iff we specify
them in the `devices` field. If we use filters like `deviceFilter`,
these devices are not picked with a log message which let users
to specify the exact device name. It's better to mention which field
users should use in this case.
Closes: https://github.com/rook/rook/issues/11353
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.
This feature is intended to be used for testing purposes only.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
The mons that are out of quorum may cause ceph commands to timeout
or fail unnecessarily trying to connect to a mon that is no longer
online. Now the mon health check will update the mon endpoints configmap
when a mon is detected out of quorum. This also means that if the
operator is restarted during a mon failover, the failed mon will no
longer remain in the ceph.conf, thus allowing the quorum to be more
likely to respond to the mons that are still in quorum.
The update for mons out of quorum only applies if other mons are in
quorum. If quorum is down, the configmap will not keep track of the offline
mons since too many are offline.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the osd is purged, properly log the host name
that is being purged and whether the host was
removed from the crush map.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
currently, osdSafeToDestroy check is done before
marking osd out which prevents ceph from rebalancing.
With this changes, we are moving check for
osdSafeToDestroy after osd is marked out.
Signed-off-by: subhamkrai <srai@redhat.com>
Run the ganesha-rados-grace command in a remote pod when multus
networking is enabled.
Signed-off-by: parth-gr <paarora@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This commit changes cryptographic length for keys created
for kmip kms to 256 from 128, since it is more secure.
Signed-off-by: Rakshith R <rar@redhat.com>
LVM logical volumes are not picked by `deviceFilter`, `devicePathFilter`,
and `useAllDevices`. It's to prevent old version's users from creating
lv-backed osd during upgrading rook by accident.
It's better to clarify lvs are not picked by filters" in docs and
suggest users to use exact device name instead.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
1. Eliminate possible memory leaks of timer.
2. Eliminate duplicated events between udev events and kernel events.
3. Empty struct have the lowest size.
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
OSD on LV with metadata device is not supported. So we once we made
a fix to disallow this configuration. However, it's insuffucient
because we forgot to reset `matched` flag.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The cluster-wide encryption feature now
supports KMIP (Key Management Interoperability Protocol)
kms.
KMIP is an extensible communication protocol that defines
message formats for the manipulation of cryptographic keys
on a key management server.
For more information, refer:
https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol
Signed-off-by: Rakshith R <rar@redhat.com>
The tool gofmt in go 1.19 requires certain formatting
in the comments section for better rendering.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The RGW support server side encryption with help of s3 protocol, till
now the `sse:kms` was support in which keys will be provided by the user
and but it will be saved in external management service like vault. Now
the support for `sse:s3` is added so the entire encryption key
management is performed by RGW itsels.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Rook supports raw mode OSD in host-based cluster. So we can also
support OSD on logical volume in this kind of cluster.
Logical volumes aren't picked by filters (i.e. `useAllDevices: true`
and `device{Path,}Filter` to avoid unwanted LV consumption on upgrade.
Closes: https://github.com/rook/rook/issues/2047
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Let it fail further once we run c-v activate. The error reported by LVM
could potentially ignored. If there is a failure, the c-v command will
fail anyways.
Signed-off-by: Sébastien Han <seb@redhat.com>
Block deletion of CephFilesystems when there are any raw Ceph
subvolumegroups present that have subvolumes in them. Empty
subvolumegroups will not block deletion.
One important subvolume group is "csi" which is the default location
where CSI subvolumes are kept. If this group is empty, it means that
there are no PVCs created based on the CephFilesystem in question. This
also holds true if there are external consumers of the filesystem in
external cluster mode.
Similarly, if there are any subvolumegroups (for example "_nogroup",
which includes subvolumes in the filesystem root) that contain
manually-created subvolumes, Rook will also see this and block deletion.
This comes into play currently with manually-created NFS exports.
A work-in progress aims to create a Ceph-CSI NFS export provisioner
which will likely create subvolumes in the "csi" group as well. This
implementation will catch this case also.
Rook still checks for CephFilesystemSubVolumeGroups explicitly in
addition to the check added here. This is to ensure that even empty
groups will block deletion if they are created via this CR type.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Return the error instead of hiding it, err could be non-nil for a lot of
reasons, eg: a canceled context or a non-existing object.
Signed-off-by: Sébastien Han <seb@redhat.com>
Ceph added a check recently that prevents creation of the
built-in pools that start with a dot, such as .nfs, .mgr,
and .rgw.root. These pools require a --yes-i-really-mean-it
flag to be passed to force the creation now.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
- disk UUID should only be got for "disk" type device.
- It's not necessary to fail prepare job when failing to get disk UUID
- We should consider that `sgdisk` reports UUID even if there is no GPT.
Closes: https://github.com/rook/rook/issues/9948
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Once we are done cleaning up the content of the encrypted osd, let's
close the main LUKS device.
This avoids having dm devices on the system after the cleanup.
Closes: https://github.com/rook/rook/issues/10181
Signed-off-by: Sébastien Han <seb@redhat.com>
The CSI package needs to load clusterInfo, today this code is in the mon
package which makes the call of LoadClusterInfo impossible without
having a circular import.
Signed-off-by: Sébastien Han <seb@redhat.com>
There is a corner case where an env variable populated with the prepare
job could influence and trigger some encryption code. We have an env
variable to discover whether the prepare job will encrypt a drive or not
so let's use that instead of running code as a noop and sometimes
trigger a corner case. In this scenario, the prepare job had an "IBM_"
env variable present in the OS image (IBM OS) so the setKEKinEnv()
function would get called for no reason.
Signed-off-by: Sébastien Han <seb@redhat.com>
This introduces a new CRD to add the ability
to create rados namespace for a given
ceph block pool. Typically the name of the pool
is the name of the blockpool created by rook.
Closes: #7035
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This commits makes Rook usable on NixOS.
NixOS system holds "installed" binaries in `/run/current-system/sw/{bin,sbin}`.
The only binaries residing outside those paths are `/usr/bin/env` and `/bin/sh`.
I got a suggestion on Slack to simply add those NixOS specific paths
Signed-off-by: Krzysztof Nazarewski <3494992+nazarewk@users.noreply.github.com>
Satoru Takeuchi
Travis Nielsen
Shinya Hayashi
subhamkrai
Blaine Gardner
Rakshith R
dkeven
parth-gr
Liang Zheng
taki-d
Jiffin Tony Thottan
Eng Zer Jun
Sébastien Han
Josh Soref
Madhu Rajanna
Krzysztof Nazarewski