Rook will now append internal default versions to custom
csi image variables passed down if version is missing in
them.
This will enable users using their private registries to
just pass different image names while rook takes care
of adding default versions to them.
Resolves: #10529
Signed-off-by: Rakshith R <rar@redhat.com>
we need to set the variable in the template
instead of the global variable for the
template rendering.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Unset encryption configuration and default
rbd map option when the encryption is not
enabled or it disabled after sometime.
Unset compression option also when the compression
is disabled.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The test is getting time out while checking backend version for kv
engines in RGW KMS unit tests.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Based on the latest info from the Ceph RGW team, update all probes. Get
rid of the liveness probe entirely. Update startup and readiness probes
to support return codes for misconfiguration (500) and for server-side
throttling (498 or 503).
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The machine disruption budgets for handling openshift
machines and machinesets are now removed since they
have been unused and unmaintained since implemented.
This feature is expected to be handled with the more
common Pod Disruption Budgets. A workaround is for the
cluster admin to set up their machine sets so they
match the zone topology. See the original design
doc from the feature here:
https://github.com/rook/rook/blob/master/design/ceph/ceph-openshift-fencing-mitigation.md
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
It removes the condition that select the devices name prefixed with /dev/mapper as it is causing the ceph-volume command to fail.
Closes: #11301
Signed-off-by: ausias-armesto <ausiasarmesto@gmail.com>
Environment variables are not recommended for secrets in pods since
they can be easily leaked if the environemnt variables are logged.
By mounting the mon secret as a file, the mgr and osd prepare pods
can read the mon secret from a file for better security.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The crash collector controller is designed for watching nodes where
ceph daemons are running, and ensuring a special daemon is running
on that node to provide additional support for ceph on that node.
The crash collector is the first example of a daemon that should be
running on all the ceph daemon nodes. The next example of such a
node daemon will be the ceph exporter that will listen for the
ceph metrics as described in the design doc.
https://github.com/rook/rook/blob/master/design/ceph/ceph-exporter.md
Now the crash collector controller is renamed to the node daemon controller
so the ceph exporter daemon can also be managed by the same controller.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The volume names of a pod must follow the standard naming convention
which does not allow periods in the name. Since the device set name
is used as the pvc name and then the volume name to attach to the osd
prepare job, we must sanitize the name for the invalid character
and replace it.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Multipath devices may have an FSTYPE=mpath_member even though
a valid device for creating an OSD. The device will be allowed
in this case when creating the OSD on a PV since it is a device
specifically provisioned for the OSD.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During osd-prepare-job,
The encrypted device is found to be closed in some cases when
the OSD deployment has been removed manually accompanied
by any of following cases:
- node reboot
- csi managed PVC being unmounted etc
while re-opening the block to re-hydrate the OSDInfo,
the dm device `<pvc-name>-block-dmcrypt` clashes with
the one used by OSD pod which is stale by now.
This commit adds cmd to remove this stale dm device.
Error with output "No such device" is ignored.
Signed-off-by: Rakshith R <rar@redhat.com>
The ceph.conf was being written with 0x444 permissions, which means
the file could not be updated if it already exists. Specifically, when
an osd is purged, if the operation is retried multiple times, only
the first operation was succeeding. The subsequent requests to purge
an osd were failing due to the config already existing. If we make
the config writeable with 0x644 then the operator can always write the file.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Ceph will use a random nonce for the msgr daemons as long as
they are running as PID 1 or have the environment variable
CEPH_USE_RANDOM_NONCE set to indicate running in a containerized
environment. While the Ceph daemons in Rook are expected to
run in PID 1, there are some environments where the container
runtime may choose to run something else such as
/usr/bin/pod as PID 1. The side effect then is that ceph uses
the pid as the nonce and causes problems when the daemons
restart with the same PID.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit implements this corner case during osd-prepare job.
```
The encrypted block is not opened, this is an extreme corner case
The OSD deployment has been removed manually AND the node rebooted
So we need to re-open the block to re-hydrate the OSDInfo.
Handling this case would mean, writing the encryption key on a
temporary file, then call luksOpen to open the encrypted block and
then call ceph-volume to list against the opened encrypted block.
We don't implement this, yet and return an error.
```
When underlying PVC for osd are CSI provisioned, the encrypted device
is closed when PVC is unmounted due to osd pod being deleted.
Therefore, this may occur more frequently and needs to be handled.
This commit implements the fix for the same.
Signed-off-by: Rakshith R <rar@redhat.com>
We can use LVM logical volumes and loop devices as OSD iff we specify
them in the `devices` field. If we use filters like `deviceFilter`,
these devices are not picked with a log message which let users
to specify the exact device name. It's better to mention which field
users should use in this case.
Closes: https://github.com/rook/rook/issues/11353
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
When mirror logs are enabled, `ceph osd pool get <poolname> all` command returns peer pool info along
with the debug logs. This causes error while parsing the logs for json
data. Using --out-file flag saves the json data in the file which can be
parsed easily while logging will still be written to stdout/stderr.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The default value of maxUnavailable of daemonset is 1.
Sometimes the upgrade can be slow due to this setting.
So I add a parameter to change the value for rbd daemonset.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
Co-authored-by: Toshikuni Fukaya <toshikuni-fukaya@cybozu.co.jp>
Co-authored-by: Yuji Ito <llamerada.jp@gmail.com>
The fsid, username, and user secrets were from legacy and no
longer needed on the osd daemon or mon pods. This removes the
unnecessary env vars from the pod specs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.
This feature is intended to be used for testing purposes only.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
For the upgrade case, Rook can fail to set status information
on CephObjectStores after CRDs are updated but before the operator is
updated. To fix this, merely allow the slices to be null in the
CephObjectStore's status.endpoints.
This PR seems to be aggravating the helm filesystem upgrade test. Allow
30 more seconds for CRDs to be done before bailing. In debugging, the
filesystem often needed only an extra 3 seconds to succeed.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The mons that are out of quorum may cause ceph commands to timeout
or fail unnecessarily trying to connect to a mon that is no longer
online. Now the mon health check will update the mon endpoints configmap
when a mon is detected out of quorum. This also means that if the
operator is restarted during a mon failover, the failed mon will no
longer remain in the ceph.conf, thus allowing the quorum to be more
likely to respond to the mons that are still in quorum.
The update for mons out of quorum only applies if other mons are in
quorum. If quorum is down, the configmap will not keep track of the offline
mons since too many are offline.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Adding two new entries(userID/userKey) in the cephClient secret, so that the csi driver can easily consume
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Rakshith R
Travis Nielsen
Madhu Rajanna
Jiffin Tony Thottan
subhamkrai
Yuma Ogami
parth-gr
Blaine Gardner
ausias-armesto
Satoru Takeuchi
Santosh Pillai
Sebastian Hasler
Avan Thakkar