From 1b394ac91841327259bc8fafc8fcdc7e772edbea Mon Sep 17 00:00:00 2001 From: rohan47 Date: Wed, 28 Jul 2021 17:04:12 +0530 Subject: [PATCH 1/2] docs: added information about which pods will have multus annotations Added information about which pods will have multus annotations. Signed-off-by: rohan47 --- Documentation/ceph-cluster-crd.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Documentation/ceph-cluster-crd.md b/Documentation/ceph-cluster-crd.md index 662012d1d..7a5ff259e 100755 --- a/Documentation/ceph-cluster-crd.md +++ b/Documentation/ceph-cluster-crd.md @@ -340,6 +340,9 @@ Based on the configuration, the operator will do the following: \* Internal cluster traffic includes OSD heartbeats, data replication, and data recovery +Only OSD pods will have both Public and Cluster networks attached. The rest of the Ceph component pods and CSI pods will only have the Public network attached. +Rook Ceph Operator will not have any networks attached as it proxies the required commands via a [sidecar container](https://github.com/rook/rook/pull/8272) in the mgr pod. + In order to work, each selector value must match a `NetworkAttachmentDefinition` object name in Multus. For `multus` network provider, an already working cluster with Multus networking is required. Network attachment definition that later will be attached to the cluster needs to be created before the Cluster CRD. From d87b6beec2a46e6420e1c27a1a358349f5262cf6 Mon Sep 17 00:00:00 2001 From: rohan47 Date: Tue, 3 Aug 2021 22:34:04 +0530 Subject: [PATCH 2/2] docs: added known cephcsi issue for multus added known cephcsi issue for multus Signed-off-by: rohan47 --- Documentation/ceph-cluster-crd.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Documentation/ceph-cluster-crd.md b/Documentation/ceph-cluster-crd.md index 7a5ff259e..7566dc96a 100755 --- a/Documentation/ceph-cluster-crd.md +++ b/Documentation/ceph-cluster-crd.md @@ -341,7 +341,7 @@ Based on the configuration, the operator will do the following: \* Internal cluster traffic includes OSD heartbeats, data replication, and data recovery Only OSD pods will have both Public and Cluster networks attached. The rest of the Ceph component pods and CSI pods will only have the Public network attached. -Rook Ceph Operator will not have any networks attached as it proxies the required commands via a [sidecar container](https://github.com/rook/rook/pull/8272) in the mgr pod. +Rook Ceph Operator will not have any networks attached as it proxies the required commands via a sidecar container in the mgr pod. In order to work, each selector value must match a `NetworkAttachmentDefinition` object name in Multus. @@ -382,6 +382,10 @@ spec: * This format is required in order to use the NetworkAttachmentDefinition across namespaces. * In Openshift, to use a NetworkAttachmentDefinition (NAD) across namespaces, the NAD must be deployed in the `default` namespace. The NAD is then referenced with the namespace: `default/rook-public-nw` +#### Known issues with multus +When a CephFS/RBD volume is mounted in a Pod using cephcsi and then the CSI CephFS/RBD plugin is restarted or terminated (e.g. by restarting or deleting its DaemonSet), all operations on the volume become blocked, even after restarting the CSI pods. The only workaround is to restart the node where the cephcsi plugin pod was restarted. +This issue is tracked [here](https://github.com/rook/rook/issues/8085). + #### IPFamily Provide single-stack IPv4 or IPv6 protocol to assign corresponding addresses to pods and services. This field is optional. Possible inputs are IPv6 and IPv4. Empty value will be treated as IPv4. Kubernetes version should be at least v1.13 to run IPv6. Dual-stack is supported as of ceph Pacific.