Travis Nielsen
fb86955f01
By default, we should set the priority class to one of the built-in priority class names to ensure that pods critical to the storage will be able to remain running when resources are low. Otherwise, critical rook pods could be evicted and affect many other pods that rely on the storage to continue functioning. The options have been available in the CRs, but until now we have just not set the defaults in the examples. Critical rook components are now set to the priority class system-node-critical if they are generally pinned to a node, and system-cluster-critical if they are critical to the storage. Some pods such as the operator and crash collector do not have a default priority class set in the examples since they don't affect the data path. Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
79 lines
3.3 KiB
YAML
79 lines
3.3 KiB
YAML
#################################################################################################################
|
|
# Define the settings for the rook-ceph cluster with common settings for a production cluster.
|
|
# All nodes with available raw devices will be used for the Ceph cluster. At least three nodes are required
|
|
# in this example. See the documentation for more details on storage settings available.
|
|
|
|
# For example, to create the cluster:
|
|
# kubectl create -f crds.yaml -f common.yaml -f operator.yaml
|
|
# kubectl create -f cluster-stretched.yaml
|
|
#################################################################################################################
|
|
|
|
apiVersion: ceph.rook.io/v1
|
|
kind: CephCluster
|
|
metadata:
|
|
name: rook-ceph
|
|
namespace: rook-ceph # namespace:cluster
|
|
spec:
|
|
dataDirHostPath: /var/lib/rook
|
|
mon:
|
|
# Five mons must be created for stretch mode
|
|
count: 5
|
|
allowMultiplePerNode: false
|
|
stretchCluster:
|
|
# The ceph failure domain will be extracted from the label, which by default is the zone. The nodes running OSDs must have
|
|
# this label in order for the OSDs to be configured in the correct topology. For topology labels, see
|
|
# https://rook.github.io/docs/rook/latest/ceph-cluster-crd.html#osd-topology.
|
|
failureDomainLabel: topology.kubernetes.io/zone
|
|
# The sub failure domain is the secondary level at which the data will be placed to maintain data durability and availability.
|
|
# The default is "host", which means that each OSD must be on a different node and you would need at least two nodes per zone.
|
|
# If the subFailureDomain is set to "osd", the OSDs would be allowed anywhere in the same zone including on the same node.
|
|
# If set to "rack" or some other intermediate failure domain, those labels would also need to be set on the nodes where
|
|
# the osds are started.
|
|
subFailureDomain: host
|
|
zones:
|
|
- name: a
|
|
arbiter: true
|
|
- name: b
|
|
- name: c
|
|
mgr:
|
|
count: 2
|
|
cephVersion:
|
|
# Stretch cluster support upstream is only available starting in Ceph Pacific
|
|
image: quay.io/ceph/ceph:v16.2.7
|
|
allowUnsupported: true
|
|
skipUpgradeChecks: false
|
|
continueUpgradeAfterChecksEvenIfNotHealthy: false
|
|
dashboard:
|
|
enabled: true
|
|
ssl: true
|
|
storage:
|
|
useAllNodes: true
|
|
useAllDevices: true
|
|
deviceFilter: ""
|
|
# OSD placement is expected to include the non-arbiter zones
|
|
placement:
|
|
# The arbiter mon can have its own placement settings that will be different from the mons.
|
|
# If the arbiter section is not included in the placement, the arbiter will use the same placement
|
|
# settings as other mons. In this example, the arbiter has a toleration to run on a master node.
|
|
arbiter:
|
|
tolerations:
|
|
- key: node-role.kubernetes.io/master
|
|
operator: Exists
|
|
effect: NoSchedule
|
|
osd:
|
|
nodeAffinity:
|
|
requiredDuringSchedulingIgnoredDuringExecution:
|
|
nodeSelectorTerms:
|
|
- matchExpressions:
|
|
- key: topology.kubernetes.io/zone
|
|
operator: In
|
|
values:
|
|
- b
|
|
- c
|
|
priorityClassNames:
|
|
mon: system-node-critical
|
|
osd: system-node-critical
|
|
mgr: system-cluster-critical
|
|
disruptionManagement:
|
|
managePodBudgets: true
|