feedc0de/my-rook-config
1
0 Fork 0
forked from rook/rook
Code Pull Requests Activity

ceph: disable the discovery daemon by default

Browse Source 
The discovery daemon is not needed in most scenarios, therefore we disable it
by default. More and more clusters are moving to the cluster-on-pvc scenario
which certainly does not need the local discovery. Even where clusters are not
running on PVCs, the discovery is not needed since the device discovery is again
performed in the osd prepare job.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit is contained in:
Travis Nielsen committed 2020-11-10 13:16:52 -07:00
1 parent 969225c0c2
commit b9f692a56e
7 files changed
+17 -12
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Documentation/ceph-quickstart.md
+9 -8
View File
@@ -78,13 +78,16 @@ kubectl create -f operator.yaml
kubectl -n rook-ceph get pod
```
**NOTE**
If you are using kubernetes v1.15 or older you need to create CRDs found here `/cluster/examples/kubernetes/ceph` as apiextension v1beta1 version of CustomResourceDefinition
is deprecated in Kubernetes v1.16 and will no longer be supported from k8s v1.22.
You can also deploy the operator with the [Rook Helm Chart](helm-operator.md).
Before you start the operator in production, there are some settings that you may want to consider:
1. If you are using kubernetes v1.15 or older you need to create CRDs found here `/cluster/examples/kubernetes/ceph/pre-k8s-1.16/crd.yaml`.
The apiextension v1beta1 version of CustomResourceDefinition was deprecated in Kubernetes v1.16.
2. Consider if you want to enable certain Rook features that are disabled by default. See the [operator.yaml](https://github.com/rook/rook/blob/{{ branchName }}/cluster/examples/kubernetes/ceph/operator.yaml) for these and other advanced settings.
1. Device discovery: Rook will watch for new devices to configure if the `ROOK_ENABLE_DISCOVERY_DAEMON` setting is enabled, commonly used in bare metal clusters.
2. Flex driver: The flex driver is deprecated in favor of the CSI driver, but can still be enabled with the `ROOK_ENABLE_FLEX_DRIVER` setting.
3. Node affinity and tolerations: The CSI driver by default will run on any node in the cluster. To configure the CSI driver affinity, several settings are available.
## Create a Rook Ceph Cluster
Now that the Rook operator is running we can create the Ceph cluster. For the cluster to survive reboots,
@@ -99,7 +102,7 @@ kubectl create -f cluster.yaml
Use `kubectl` to list pods in the `rook-ceph` namespace. You should be able to see the following pods once they are all running.
The number of osd pods will depend on the number of nodes in the cluster and the number of devices configured.
If you did not modify the `cluster.yaml` above, it is expected that one OSD will be created per node.
The CSI, `rook-ceph-agent`, and `rook-discover` pods are also optional depending on your settings.
The CSI, `rook-ceph-agent` (flex driver), and `rook-discover` pods are also optional depending on your settings.
> If the `rook-ceph-mon`, `rook-ceph-mgr`, or `rook-ceph-osd` pods are not created, please refer to the
> [Ceph common issues](ceph-common-issues.md) for more details and potential solutions.
@@ -113,7 +116,6 @@ csi-cephfsplugin-rthrp 3/3 Running 0
csi-rbdplugin-hbsm7 3/3 Running 0 140s
csi-rbdplugin-provisioner-5b5cd64fd-nvk6c 6/6 Running 0 140s
csi-rbdplugin-provisioner-5b5cd64fd-q7bxl 6/6 Running 0 140s
rook-ceph-agent-4zkg8 1/1 Running 0 140s
rook-ceph-crashcollector-minikube-5b57b7c5d4-hfldl 1/1 Running 0 105s
rook-ceph-mgr-a-64cd7cdf54-j8b5p 1/1 Running 0 77s
rook-ceph-mon-a-694bb7987d-fp9w7 1/1 Running 0 105s
@@ -126,7 +128,6 @@ rook-ceph-osd-2-6cd4b776ff-v4d68 1/1 Running 0
rook-ceph-osd-prepare-node1-vx2rz 0/2 Completed 0 60s
rook-ceph-osd-prepare-node2-ab3fd 0/2 Completed 0 60s
rook-ceph-osd-prepare-node3-w4xyz 0/2 Completed 0 60s
rook-discover-dhkb8 1/1 Running 0 140s
```
To verify that the cluster is in a healthy state, connect to the [Rook toolbox](ceph-toolbox.md) and run the
PendingReleaseNotes.md
+2
View File
@@ -13,6 +13,8 @@ v1.5...
- Ceph mons require an odd number for a healthy quorum. An even number of mons is now disallowed.
- Update deprecated CRD apiextensions.k8s.io/v1beta1 to v1 ([#6424](https://github.com/rook/rook/pull/6424))
- preservePoolsOnDelete is deprecated for CephFilesystem and is no longer allowed because of data-loss concerns. preserveFilesystemOnDelete acts as a replacement and preserves the filesystem when the CephFilesystem CRD is deleted (which implicitly includes all associated pools). See [#6495](https://github.com/rook/rook/pull/6495) for more details.
- The discovery daemon is disabled by default since the discovery is not necessary in most clusters. Enable the discovery daemon if
devices are being added to nodes and you want to automatically configure OSDs on new devices without restarting the operator.
## Features
cluster/charts/rook-ceph/values.yaml
+1 -1
View File
@@ -252,7 +252,7 @@ csi:
#image: k8s.gcr.io/sig-storage/csi-resizer:v1.0.0
enableFlexDriver: false
enableDiscoveryDaemon: true
enableDiscoveryDaemon: false
## if true, run rook operator on the host network
# useOperatorHostNetwork: true
cluster/examples/kubernetes/ceph/operator-openshift.yaml
+1 -1
View File
@@ -460,7 +460,7 @@ spec:
# Whether to start the discovery daemon to watch for raw storage devices on nodes in the cluster.
# This daemon does not need to run if you are only going to create your OSDs based on StorageClassDeviceSets with PVCs.
- name: ROOK_ENABLE_DISCOVERY_DAEMON
value: "true"
value: "false"
# Whether to start machineDisruptionBudget and machineLabel controller to watch for the osd pods and MDBs.
- name: ROOK_ENABLE_MACHINE_DISRUPTION_BUDGET
cluster/examples/kubernetes/ceph/operator.yaml
+1 -1
View File
@@ -411,7 +411,7 @@ spec:
# Whether to start the discovery daemon to watch for raw storage devices on nodes in the cluster.
# This daemon does not need to run if you are only going to create your OSDs based on StorageClassDeviceSets with PVCs.
- name: ROOK_ENABLE_DISCOVERY_DAEMON
value: "true"
value: "false"
# Time to wait until the node controller will move Rook pods to other
# nodes after detecting an unreachable node.
cmd/rook/ceph/operator.go
+1 -1
View File
@@ -48,7 +48,7 @@ func init() {
operatorCmd.Flags().DurationVar(&mon.MonOutTimeout, "mon-out-timeout", mon.MonOutTimeout, "mon out timeout (duration)")
operatorCmd.Flags().BoolVar(&operator.EnableFlexDriver, "enable-flex-driver", true, "enable the rook flex driver")
operatorCmd.Flags().BoolVar(&operator.EnableDiscoveryDaemon, "enable-discovery-daemon", true, "enable the rook discovery daemon")
operatorCmd.Flags().BoolVar(&operator.EnableDiscoveryDaemon, "enable-discovery-daemon", false, "enable the rook discovery daemon")
// csi deployment templates
operatorCmd.Flags().StringVar(&csi.RBDPluginTemplatePath, "csi-rbd-plugin-template-path", csi.DefaultRBDPluginTemplatePath, "path to ceph-csi rbd plugin template")
tests/framework/installer/ceph_manifests.go
+2
View File
@@ -3076,6 +3076,8 @@ spec:
value: "true"
- name: ROOK_CURRENT_NAMESPACE_ONLY
value: "false"` + openshiftEnv + `
- name: ROOK_ENABLE_DISCOVERY_DAEMON
value: "true"
volumeMounts:
- mountPath: /var/lib/rook
name: rook-config