The default value of maxUnavailable of daemonset is 1.
Sometimes the upgrade can be slow due to this setting.
So I add a parameter to change the value for rbd daemonset.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
Co-authored-by: Toshikuni Fukaya <toshikuni-fukaya@cybozu.co.jp>
Co-authored-by: Yuji Ito <llamerada.jp@gmail.com>
While #4211 added support for the udev blacklist and also added it to
the values file, it omitted actually passing the value as environment
variable.
Signed-off-by: Jens Erat <email@jenserat.de>
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.
This feature is intended to be used for testing purposes only.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
For the upgrade case, Rook can fail to set status information
on CephObjectStores after CRDs are updated but before the operator is
updated. To fix this, merely allow the slices to be null in the
CephObjectStore's status.endpoints.
This PR seems to be aggravating the helm filesystem upgrade test. Allow
30 more seconds for CRDs to be done before bailing. In debugging, the
filesystem often needed only an extra 3 seconds to succeed.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
External CephObjectStores already have endpoints defined by
spec.gateway.externalRgwEndpoints, and if the external store is
configured with TLS (HTTPS), the store's certificates will likely not
accept connections intended for the Service endpoint Rook creates. Some
users might not be able to easily add the service endpoint to their
certificates. Therefore, don't even bother creating a Service for
external clusters.
This does introduce a few issues. The Service seems to have been
initially created to allow multiple external RGW endpoints to be
addressable via a single address in Rook. For all connections to an
external CephObjectStore with multiple endpoints, simply choose an
endpoint at random. Random selection will prevent Rook from failing to
create buckets or users on an external store if one of the external
store's endpoints fails.
The latest OBC library (lib-bucket-provisioner) allows updating the
endpoints on ObjectBuckets after they are created. This allows Rook
users to change endpoints on external CephObjectStores without breaking
all existing OBCs. It requires implementation of the new GetUserID()
library call, requires updating Provision() and Grant() calls to be
idempotent, and it requires removing the Update() call.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
85b079105b introduced some bugs that here I'm fixing.
1. Docs missing `csi.` prefix.
2. Chart was using `cis.csiRBDPluginVolumes` instead of `csi.csiCephFSPluginVolume`.
3. Values had to be a scalar value, not a raw string. Otherwise, the operator can't parse it.
4. Link to `/run/booted-system` instead of `/run/current-system`. In the former, the kernel modules will always match the current kernel; in the latter, it could happen that after a new `nixos-rebuild switch`, plugin containers can't find the kernel modules for the booted kernel version.
5. Values included in the `values.yaml` template were plural. These are not supported, they must all be singular.
About that 3rd point, this was the error being logged:
```
2022-11-04 09:54:47.489851 I | op-k8sutil: CSI_RBD_PLUGIN_VOLUME="|\n - name: lib-modules\n hostPath:\n path: /run/current-system/kernel-modules/lib/modules/\n - name: host-nix\n hostPath:\n path: /nix" (configmap)
2022-11-04 09:54:47.489907 W | ceph-csi: failed to parse "|\n - name: lib-modules\n hostPath:\n path: /run/current-system/kernel-modules/lib/modules/\n - name: host-nix\n hostPath:\n path: /nix" for "CSI_RBD_PLUGIN_VOLUME". json: cannot unmarshal string into Go value of type []v1.Volume
2022-11-04 09:54:47.489916 I | op-k8sutil: CSI_RBD_PLUGIN_VOLUME_MOUNT="|\n - name: host-nix\n mountPath: /nix\n readOnly: true" (configmap)
2022-11-04 09:54:47.489937 W | ceph-csi: failed to parse "|\n - name: host-nix\n mountPath: /nix\n readOnly: true" for "CSI_RBD_PLUGIN_VOLUME_MOUNT". json: cannot unmarshal string into Go value of type []v1.VolumeMount
```
@moduon MT-1503
Signed-off-by: Jairo Llopis <yajo.sk8@gmail.com>
The security scan is failing due to a go dependency
so we update the go modules and all dependencies to
the latest available.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This updates the cephcsi to latest 3.7.2
release and also updates sidecar to the latest
release in some missing files.
Note:- sidecar images are upto date in most of
the places, in some places it was missing, This
PR updates it in missing place.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The flex driver was deprecated and fully purged long ago,
and now finally we are purging the related properties
from the operator deployment that are no longer applied
anywhere.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
enabling logCollector by default will enable the coredump
generation in case a process terminates with a segmentation fault.
Signed-off-by: gauravsitlani <gauravsitlani@riseup.net>
Remove the health checker for CephObjectStore. The liveness and
readiness probes go through the same code paths in RGW as creating
buckets without as much affect on the storage backend.
Full discussion: https://github.com/rook/rook/issues/11031
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
I see the mon_data_avail_warn is set to 500M
but acording to ceph doc this should be in percentage
And by default this percentage is 30%,
So re assigned to 10% for the CI
Closes: https://github.com/rook/rook/issues/11130
Signed-off-by: parth-gr <paarora@redhat.com>
The osd prepare job is not recommended to have memory limits since
it's a one-time operation depending on the size of the disk that must
be allowed to complete successfully.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The OSD creation may need to burst during OSD provisioning depending
on the size of the OSD or similar factors. If the OSD prepare job is
OOM killed it will cause OSD provisioning to fail and have various
side effects that are difficult to troubleshoot to get the OSD
to succeed. So we increase the recommendation significantly to avoid
the OOM kill.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With Kubernetes 1.25 it seems that applying the yaml files that have a
CephBlockPool with spec.annotations fail with errors like this:
CephBlockPool in version "v1" cannot be handled as a CephBlockPool:
strict decoding error: unknown field "spec.annotations"
Updates: ceph/ceph-csi#3362
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Introduce a new env variable ROOK_CSI_IMAGE_PULL_POLICY in rook operator configmap which should be used to
customize the imagePullPolicy for the csi driver and imagePullPolicy property in cephVersionSpec for ceph pods.
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
The sssd.conf may refer to additional files, like a CA bundle or TLS
certificates. These files need to be made available in the SSSD sidecar.
With the new `sssdGenericFiles` reference to a VolumeSource and a
`MountPath`, a provided `sssd.conf` can use the additional files.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This option open-up volumes and volumemounts
for user customization of the rbd,cephfs,nfs
plugin daemonset.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
ceph cluster on ci mostly have health error
mon disk low on size,
so overriding the min aval capacity to 0 from 30% default so this
error can be removed
Closes: rook#10971
Signed-off-by: parth-gr <paarora@redhat.com>
Currently for external clustr multitenancy is only applicable for new
deployments, but adding doc change so multi-tenancy can be achieved for
upgraded cluster also
Closes: https://github.com/rook/rook/issues/10678
Signed-off-by: parth-gr <paarora@redhat.com>
subhamkrai
Zuhair
Jens Erat
Travis Nielsen
parth-gr
Blaine Gardner
Zuhair AlSader
Jairo Llopis
Anthony D'Atri
gauravsitlani
Madhu Rajanna
Humble Chirammal
Niels de Vos
Avan Thakkar