feedc0de/my-rook-config
1
0
Fork 0
forked from rook/rook
Code Pull Requests Activity
Files
f034f000e1e09f8f92eb0eb660de66a1fa8628a4
my-rook-config/PendingReleaseNotes.md
T
Noah Watkins f034f000e1 ceph: schedule mons across failure domains 
this patch introduces failure-domain aware scheduling for monitor pods.
previously the only policy was to avoid scheduling monitors on the same
nodes. the new policy tries to spread monitors across zones and nodes.

   NOTE: this patch only enforces the well-known k8s "zone"
   failure-domain label (i.e. it does not look for region labels). this
   decision is made for two reasons. first, eliminating explicit
   scheduling is an important goal and will be done as part of upcoming
   work. this will enable all supported failure-domain labels to be
   handled. second, the "zone" domain is likely more appropriate for a
   larger portion of users (as opposed to the geographical distinction
   made with region), and supporting multiple failure domain types would
   add significant complexity for a temporary resolution to failure
   domain scheduling.

at a high-level this patch introduces a data structure `NodeUsage` that
is a pairing of a v1.Node and metadata relevant to scheduling: number of
monitor pods on the node, and its status w.r.t. to being scheduable for
new monitor pods.

each scheduling event computes a unified data structure that organizes
for each node a `NodeUsage` structure into per-failure-zone groups.
subsequent algorithms rely soley on this unified structure to make
scheduling decisions.

there are two entry points to the changes made:

1) during orchestration mon:Cluster:assignMons is invoked to make a
place new monitor pods onto nodes. the core scheduling logic is now
self-contained in mon:scheduleMonitor which implements node and zone
aware scheduling policies.

2) periodic health checks resolve any conflicts that may occur due to
changes to the cluster (e.g. new nodes, configuration changes, etc...).
the two existing checks: mons on invalid nodes, and overloaded nodes are
both handled.

fixes: #2603

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-07-11 14:23:15 -07:00

2.9 KiB
Raw Blame History

Major Themes

Action Required

Notable Features

  • Creation of storage pools through the custom resource definitions (CRDs) now allows users to optionally specify deviceClass property to enable distribution of the data only across the specified device class. See Ceph Block Pool CRD for an example usage
  • Added K8s 1.15 to the test matrix and removed K8s 1.10 from the test matrix.
  • OwnerReferences are created with the fully qualified apiVersion such that the references will work properly on OpenShift.
  • Linear disk device can now be used for Ceph OSDs.

Ceph

  • Rook can now be configured to read "region" and "zone" labels on Kubernetes nodes and use that information as part of the CRUSH location for the OSDs.
  • Rgw pods have liveness probe enabled
  • Rgw is now configured with the Beast backend as of the Nautilus release
  • OSD: newly updated cluster from 0.9 to 1.0.3 and thus Ceph Nautilus will have their OSDs allowing new features for Nautilus
  • Rgw instances have their own key and thus are properly reflected in the Ceph status
  • The Rook Agent pods are now started when the CephCluster is created rather than immediately when the operator is started.
  • Ceph CRUSH tunable are not enforced to "firefly" anymore, Ceph picks the right tunable for its own version, to read more about tunable see the Ceph documentation
  • NodeAffinity can be applied to rook-ceph-agent DaemonSet with AGENT_NODE_AFFINITY environment variable.
  • NodeAffinity can be applied to rook-discover DaemonSet with DISCOVER_AGENT_NODE_AFFINITY environment variable.
  • Rook does not create an initial CRUSH map anymore and let Ceph do it normally
  • Ceph monitor placement will now take failure zones into account see the documentation for more information.
  • The cluster CRD option to allow multiple monitors to be scheduled on the same node---spec.Mon.AllowMultiplePerNode---is now active when a cluster is first created. Previously, it was ignored when a cluster was first installed.

Breaking Changes

Known Issues

Deprecations

Ceph

  • For rgw, when deploying an object store with object.yaml, using allNodes is not supported anymore, a transition path has been implemented in the code though. So if you were using allNodes: true, Rook will replace each daemonset with a deployment (one for one replacement) gradually. This operation will be triggered on an update or when a new version of the operator is deployed. Once complete, it is expected that you edit your object CR with kubectl -n rook-ceph edit cephobjectstore.ceph.rook.io/my-store and set allNodes: false and instances with the current number of rgw instances.

View Git Blame Copy Permalink