Files
my-rook-config/cluster/examples/kubernetes/ceph/filesystem-ec.yaml
T
Juan Miguel Olmo Martínez f62b2fa305 Ceph: MDS pod placement adheres to fault domain topology
With this modification is guaranteed that at least 1 MDS pod is going to be
placed in each of the available zones in the k8s cluster.
This would be improved in the future using:
<Pod Topology Spread Constraints> (still in alpha state since kubernetes V.1.16)

This modification obtain the number of zones in the k8s cluster and appends an
<antiaffinity> term using the topologyKey
<topology.kubernetes.io/zone> in the same number of MDS pods.
In the case that will be more MDS pods than zones, the <antiaffinity> term won't
be added in these extra pods.

Important Note:
The antiaffiniy term only will work with nodes labeled using the NEW label:
 <topology.kubernetes.io/zone> present in k8s V.1.17 clusters.
For previous versions of k8s clusters the label used is:
 <failure-domain.beta.kubernetes.io/zone>
And therefore, in this kind of clusters this modification won't work.

Resolves #4641

[test ceph]

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2020-01-17 19:01:48 +01:00

82 lines
3.1 KiB
YAML

#################################################################################################################
# Create a filesystem with settings for erasure-coding instead of replication. A minimum of three nodes with OSDs
# are required in this example since the default failureDomain is host.
# kubectl create -f filesystem-ec.yaml
#################################################################################################################
apiVersion: ceph.rook.io/v1
kind: CephFilesystem
metadata:
name: myfs-ec
namespace: rook-ceph
spec:
# The metadata pool spec
metadataPool:
replicated:
# You need at least three OSDs on different nodes for this config to work
size: 3
# The list of data pool specs
dataPools:
# You need at least three `bluestore` OSDs on different nodes for this config to work
- erasureCoded:
dataChunks: 2
codingChunks: 1
# Whether to preserve metadata and data pools on filesystem deletion
preservePoolsOnDelete: true
# The metadata service (mds) configuration
metadataServer:
# The number of active MDS instances
activeCount: 1
# Whether each active MDS instance will have an active standby with a warm metadata cache for faster failover.
# If false, standbys will be available, but will not have a warm cache.
activeStandby: true
# The affinity rules to apply to the mds deployment
placement:
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: role
# operator: In
# values:
# - mds-node
# tolerations:
# - key: mds-node
# operator: Exists
# podAffinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- rook-ceph-mds
# topologyKey: kubernetes.io/hostname will place MDS across different hosts
topologyKey: kubernetes.io/hostname
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- rook-ceph-mds
# topologyKey: */zone can be used to spread MDS across different AZ
# Use <topologyKey: failure-domain.beta.kubernetes.io/zone> in k8s cluster if your cluster is v1.16 or lower
# Use <topologyKey: topology.kubernetes.io/zone> in k8s cluster is v1.17 or upper
topologyKey: topology.kubernetes.io/zone
# A key/value list of annotations
annotations:
# key: value
resources:
# The requests and limits set here, allow the filesystem MDS Pod(s) to use half of one CPU core and 1 gigabyte of memory
# limits:
# cpu: "500m"
# memory: "1024Mi"
# requests:
# cpu: "500m"
# memory: "1024Mi"
# priorityClassName: my-priority-class