forked from rook/rook
Noah Watkins
f034f000e1
this patch introduces failure-domain aware scheduling for monitor pods. previously the only policy was to avoid scheduling monitors on the same nodes. the new policy tries to spread monitors across zones and nodes. NOTE: this patch only enforces the well-known k8s "zone" failure-domain label (i.e. it does not look for region labels). this decision is made for two reasons. first, eliminating explicit scheduling is an important goal and will be done as part of upcoming work. this will enable all supported failure-domain labels to be handled. second, the "zone" domain is likely more appropriate for a larger portion of users (as opposed to the geographical distinction made with region), and supporting multiple failure domain types would add significant complexity for a temporary resolution to failure domain scheduling. at a high-level this patch introduces a data structure `NodeUsage` that is a pairing of a v1.Node and metadata relevant to scheduling: number of monitor pods on the node, and its status w.r.t. to being scheduable for new monitor pods. each scheduling event computes a unified data structure that organizes for each node a `NodeUsage` structure into per-failure-zone groups. subsequent algorithms rely soley on this unified structure to make scheduling decisions. there are two entry points to the changes made: 1) during orchestration mon:Cluster:assignMons is invoked to make a place new monitor pods onto nodes. the core scheduling logic is now self-contained in mon:scheduleMonitor which implements node and zone aware scheduling policies. 2) periodic health checks resolve any conflicts that may occur due to changes to the cluster (e.g. new nodes, configuration changes, etc...). the two existing checks: mons on invalid nodes, and overloaded nodes are both handled. fixes: #2603 Signed-off-by: Noah Watkins <noahwatkins@gmail.com>