docs: update node loss handling instructions
Expand the Node Loss section in block-storage.md to accurately describe the automatic fencing flow, and add dedicated Network Fencing sections across the CSI documentation to provide configuration examples for both manifest-based and Helm-based deployments. * block-storage.md: introduce the Network Fencing prerequisite paragraph with links to the new example sections, replace the single-line "auto-fencing" description with three bullets describing the client blocklist flow, and update Node Recovery to a single sentence covering workload rescheduling and the 5-minute cool-down behavior that triggers automatic unblocklist. Add a Warning admonition instructing administrators not to remove the out-of-service taint before the node completes a full power cycle. * csi-configuration.md: add a Network Fencing section enumerating the three prerequisites (CSI-Addons controller, CSI-Addons sidecar, enableFencing flag) and showing the RBD Driver CR manifest with deployCsiAddons and enableFencing set together. * csi-drivers-chart.md: add a Network Fencing subsection under Custom settings showing the per-driver Helm values for deployCsiAddons and enableFencing, with a cross-reference to csi-configuration.md. * ceph-csi-drivers.md: replace the legacy CSI_ENABLE_CSIADDONS configmap patch in the Enable CSI-Addons Sidecar section with the modern deployCsiAddons approach on OperatorConfig or Driver, and add a Network Fencing overview section that links out for configuration details and the operational flow. Signed-off-by: N3rdBot <wujiafeng17@gmail.com>
This commit is contained in:
4 files changed
+68
-13
No files matched your search
@@ -209,10 +209,13 @@ The erasure coded pool must be set as the `dataPool` parameter in
|
||||
|
||||
If a node goes down where a pod is running where a volume is mounted, the volume cannot automatically be mounted on another node. The node must be guaranteed to be offline before the volume can be mounted on another node.
|
||||
|
||||
Network fencing is designed to prevent a failed node from accessing the volume before it is remounted elsewhere. The feature is **not enabled by default**. For step-by-step configuration examples to enable automatic network fencing and unfencing, see:
|
||||
|
||||
* The [`Network Fencing`](../Ceph-CSI/csi-configuration.md#network-fencing) section in the CSI configuration documentation for manifest-based examples using the RBD `Driver` CR.
|
||||
* The [`Network Fencing`](../../Helm-Charts/csi-drivers-chart.md#network-fencing) section in the Ceph-CSI driver Helm chart documentation for Helm-based examples.
|
||||
|
||||
### Handling Node Loss
|
||||
|
||||
|
||||
When a node is confirmed to be down, add the following taints to the node:
|
||||
|
||||
```console
|
||||
@@ -220,11 +223,20 @@ kubectl taint nodes <node-name> node.kubernetes.io/out-of-service=nodeshutdown:N
|
||||
kubectl taint nodes <node-name> node.kubernetes.io/out-of-service=nodeshutdown:NoSchedule
|
||||
```
|
||||
|
||||
After the taint is added to the node, the CephCSI driver will automatically handle the network fencing to prevent connections to Ceph from the volume on that node.
|
||||
After the taint is added to the node and the fencing feature is properly configured:
|
||||
|
||||
* The CSI driver will retrieve the stored client address from the RBD image metadata
|
||||
* If the node has the `out-of-service` taint, the driver will add the client address to the Ceph blocklist to prevent connections to Ceph from the volume on that node
|
||||
* The blocklist entry has an extended duration (approximately 3 years) to ensure the protection persists until the node undergoes a complete power cycle.
|
||||
|
||||
!!! warning
|
||||
When a node becomes out of service, its mounts and device mappings will persist until the node undergoes a complete power lifecycle (includes shutdown and startup). To prevent data inconsistency or corruption, administrators **MUST NOT** remove the `node.kubernetes.io/out-of-service` taint until the node has successfully completed a full power cycle. Removing the taint prematurely may leave stale device states, active client sessions, or lingering mounts, which can lead to serious data integrity issues.
|
||||
|
||||
### Node Recovery
|
||||
|
||||
If the node comes back online, the CephCSI driver will automatically handle the network unfencing when the node taints are removed:
|
||||
If the node comes back online and the fencing feature is properly configured, workloads can be rescheduled onto the recovered node once the `out-of-service` taints are removed. If the 5-minute cool-down period has passed, the CSI-Addons controller will automatically remove the client address from the blocklist.
|
||||
|
||||
To manually remove the taints after the node has completed a full power cycle:
|
||||
|
||||
```console
|
||||
kubectl taint nodes <node-name> node.kubernetes.io/out-of-service=nodeshutdown:NoExecute-
|
||||
|
||||
Reference in new issue
Block a user