helm: add tolerationKey and add missing properties in docs

Documentation/helm-operator.md

@@ -81,7 +81,13 @@ The following tables lists the configurable parameters of the rook-operator char
 | `image.pullPolicy` | Image pull policy                    | `IfNotPresent`       |
 | `rbacEnable`       | If true, create & use RBAC resources | `true`               |
 | `resources`        | Pod resource requests & limits       | `{}`                 |
-| `logLevel`        | Global log level        | `INFO`                 |
+| `logLevel`         | Global log level        | `INFO`                 |
+| `agent.flexVolumeDirPath` | Path where the Rook agent discovers the flex volume plugins | `/usr/libexec/kubernetes/kubelet-plugins/volume/exec/` |
+| `agent.toleration`        | Toleration for the agent pods | <none> |
+| `agent.tolerationKey`     | The specific key of the taint to tolerate | <none> |
+| `mon.healthCheckInterval` | The frequency for the operator to check the mon health | `45s` |
+| `mon.monOutTimeout`       | The time to wait before failing over an unhealthy mon | `300s` |
+
 
 Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example to disable RBAC,
 

PendingReleaseNotes.md

@@ -5,6 +5,10 @@
 ## Notable Features
 - Monitoring is now done through the Ceph MGR service for Ceph storage.
 
+### Operator Settings
+- `AGENT_TOLERATION`: Toleration can be added to the Rook agent, such as to run on the master node.
+- `FLEXVOLUME_DIR_PATH`: Flex volume directory can be overridden on the Rook agent.
+
 ## Breaking Changes
 - `armhf` build of Rook have been removed. Ceph is not supported or tested on `armhf`. arm64 support continues.
 

cluster/charts/rook/templates/deployment.yaml

@@ -30,6 +30,14 @@ spec:
         - name: AGENT_TOLERATION
           value: {{ .Values.agent.toleration }}
 {{- end }}
+{{- if .Values.agent.tolerationKey }}
+        - name: AGENT_TOLERATION_KEY
+          value: {{ .Values.agent.tolerationKey }}
+{{- end }}
+{{- if .Values.agent.flexVolumeDirPath }}
+        - name: FLEXVOLUME_DIR_PATH
+          value: {{ .Values.agent.flexVolumeDirPath }}
+{{- end }}
 {{- end }}
         - name: ROOK_LOG_LEVEL
           value: {{ .Values.logLevel }}
@@ -45,10 +53,6 @@ spec:
           valueFrom:
             fieldRef:
               fieldPath: metadata.namespace
-{{- if .Values.flexVolumeDirPath }}
-        - name: FLEXVOLUME_DIR_PATH
-          value: {{ .Values.flexVolumeDirPath }}
-{{- end }}
 {{- if .Values.mon }}
 {{- if .Values.mon.healthCheckInterval }}
         - name: ROOK_MON_HEALTHCHECK_INTERVAL

cluster/charts/rook/values.yaml.tmpl

@@ -28,11 +28,10 @@ logLevel: INFO
 rbacEnable: true
 
 ## Rook Agent configuration
-## toleration. Choose between NoSchedule, PreferNoSchedule and NoExecute:
+## toleration: NoSchedule, PreferNoSchedule or NoExecute
+## tolerationKey: Set this to the specific key of the taint to tolerate
+## flexVolumeDirPath: The path where the Rook agent discovers the flex volume plugins
 # agent:
 #   toleration: NoSchedule
-
-## Override FlexVolume plugin directory
-## If unset, the Rook operator tries to discover it or uses a compiled in
-## default.
-#flexVolumeDirPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
+#   tolerationKey: key
+#   flexVolumeDirPath: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/

cluster/examples/kubernetes/rook-operator.yaml

@@ -130,7 +130,10 @@ spec:
         #  value: "NoSchedule"
         # (Optional) Rook Agent toleration key. Set this to the key of the taint you want to tolerate
         # - name: AGENT_TOLERATION_KEY
-        #  value: "<KeyOfTheTaintToTolerate"
+        #  value: "<KeyOfTheTaintToTolerate>"
+        # Set the path where the Rook agent can find the flex volumes
+        # - name: FLEXVOLUME_DIR_PATH
+        #  value: "<PathToFlexVolumes>"
         # The interval to check if every mon is in the quorum.
         - name: ROOK_MON_HEALTHCHECK_INTERVAL
           value: "45s"