Update to the latest OBC controller library to fix an issue where
ObjectBucket status is not updated when an OB is created.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When mirror logs are enabled, `ceph osd pool get <poolname> all` command returns peer pool info along
with the debug logs. This causes error while parsing the logs for json
data. Using --out-file flag saves the json data in the file which can be
parsed easily while logging will still be written to stdout/stderr.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The default value of maxUnavailable of daemonset is 1.
Sometimes the upgrade can be slow due to this setting.
So I add a parameter to change the value for rbd daemonset.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
Co-authored-by: Toshikuni Fukaya <toshikuni-fukaya@cybozu.co.jp>
Co-authored-by: Yuji Ito <llamerada.jp@gmail.com>
The fsid, username, and user secrets were from legacy and no
longer needed on the osd daemon or mon pods. This removes the
unnecessary env vars from the pod specs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The already exists error message is perhaps the most common
issue with the csi driver. The csi troubleshooting guide
is now clarified to make that error message easier
to find.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rook has for quite some time only been focused on the Ceph
operator. Maintainers have only been focused on Ceph, and
the community has likewise been focused on Ceph. Now the docs
are updated to clarify that Ceph storage is the focus and
there are no plans for the project to support other storage
providers.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
While #4211 added support for the udev blacklist and also added it to
the values file, it omitted actually passing the value as environment
variable.
Signed-off-by: Jens Erat <email@jenserat.de>
The name of the CRD is more clear for finding the CRD specs
instead of a modified form of the name that was intended
to be more human readable. When someone is looking for the
CRD settings, they more naturally expect the CRD name
in the title
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.
This feature is intended to be used for testing purposes only.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
For the upgrade case, Rook can fail to set status information
on CephObjectStores after CRDs are updated but before the operator is
updated. To fix this, merely allow the slices to be null in the
CephObjectStore's status.endpoints.
This PR seems to be aggravating the helm filesystem upgrade test. Allow
30 more seconds for CRDs to be done before bailing. In debugging, the
filesystem often needed only an extra 3 seconds to succeed.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The mons that are out of quorum may cause ceph commands to timeout
or fail unnecessarily trying to connect to a mon that is no longer
online. Now the mon health check will update the mon endpoints configmap
when a mon is detected out of quorum. This also means that if the
operator is restarted during a mon failover, the failed mon will no
longer remain in the ceph.conf, thus allowing the quorum to be more
likely to respond to the mons that are still in quorum.
The update for mons out of quorum only applies if other mons are in
quorum. If quorum is down, the configmap will not keep track of the offline
mons since too many are offline.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Adding two new entries(userID/userKey) in the cephClient secret, so that the csi driver can easily consume
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
External CephObjectStores already have endpoints defined by
spec.gateway.externalRgwEndpoints, and if the external store is
configured with TLS (HTTPS), the store's certificates will likely not
accept connections intended for the Service endpoint Rook creates. Some
users might not be able to easily add the service endpoint to their
certificates. Therefore, don't even bother creating a Service for
external clusters.
This does introduce a few issues. The Service seems to have been
initially created to allow multiple external RGW endpoints to be
addressable via a single address in Rook. For all connections to an
external CephObjectStore with multiple endpoints, simply choose an
endpoint at random. Random selection will prevent Rook from failing to
create buckets or users on an external store if one of the external
store's endpoints fails.
The latest OBC library (lib-bucket-provisioner) allows updating the
endpoints on ObjectBuckets after they are created. This allows Rook
users to change endpoints on external CephObjectStores without breaking
all existing OBCs. It requires implementation of the new GetUserID()
library call, requires updating Provision() and Grant() calls to be
idempotent, and it requires removing the Update() call.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
85b079105b introduced some bugs that here I'm fixing.
1. Docs missing `csi.` prefix.
2. Chart was using `cis.csiRBDPluginVolumes` instead of `csi.csiCephFSPluginVolume`.
3. Values had to be a scalar value, not a raw string. Otherwise, the operator can't parse it.
4. Link to `/run/booted-system` instead of `/run/current-system`. In the former, the kernel modules will always match the current kernel; in the latter, it could happen that after a new `nixos-rebuild switch`, plugin containers can't find the kernel modules for the booted kernel version.
5. Values included in the `values.yaml` template were plural. These are not supported, they must all be singular.
About that 3rd point, this was the error being logged:
```
2022-11-04 09:54:47.489851 I | op-k8sutil: CSI_RBD_PLUGIN_VOLUME="|\n - name: lib-modules\n hostPath:\n path: /run/current-system/kernel-modules/lib/modules/\n - name: host-nix\n hostPath:\n path: /nix" (configmap)
2022-11-04 09:54:47.489907 W | ceph-csi: failed to parse "|\n - name: lib-modules\n hostPath:\n path: /run/current-system/kernel-modules/lib/modules/\n - name: host-nix\n hostPath:\n path: /nix" for "CSI_RBD_PLUGIN_VOLUME". json: cannot unmarshal string into Go value of type []v1.Volume
2022-11-04 09:54:47.489916 I | op-k8sutil: CSI_RBD_PLUGIN_VOLUME_MOUNT="|\n - name: host-nix\n mountPath: /nix\n readOnly: true" (configmap)
2022-11-04 09:54:47.489937 W | ceph-csi: failed to parse "|\n - name: host-nix\n mountPath: /nix\n readOnly: true" for "CSI_RBD_PLUGIN_VOLUME_MOUNT". json: cannot unmarshal string into Go value of type []v1.VolumeMount
```
@moduon MT-1503
Signed-off-by: Jairo Llopis <yajo.sk8@gmail.com>
Satoru Takeuchi
Blaine Gardner
Santosh Pillai
Travis Nielsen
Deepika Upadhyay
Sebastian Hasler
Jens Erat
dependabot[bot]
parth-gr
Avan Thakkar
Zuhair AlSader
Jairo Llopis