Earlier, simultaneous drains could allow the drain detection to switch
disabled PDBs before the Ceph health had fully reflected the effects.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
- Checking whether drain was incorrectly checked precense in map instead
of string length. The actual zero value was an empty string.
- Unsetting drain did not check if the cluster was clean.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
Since I was parsing this file with the previous commits let's add some
cosmetic changes.
%q is more readable since it doubles quotes the variable.
Signed-off-by: Sébastien Han <seb@redhat.com>
WriteConnectionConfig() is called after the controller starts and thus
the 'osd find' command fails to complete because there is no ceph.conf
or key on the operator.
Let's skip this error print if that's the case since it's misleading and
polluting the logs.
Signed-off-by: Sébastien Han <seb@redhat.com>
The node labels were already supported for zones and regions to add to
the CRUSH map. Now all layers of the CRUSH map will be supported
with the new labels such as topology.rook.io/rack.
The labels will be detected at the osd startup time
similar to the zone and region labels already being detected.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
- Implement exportable TolerationSet with the following differences:
- Change map key to a string to ensure it is deterministic.
The old key was a struct that included a pointer.
- Sort the outputted list by this key to ensure the order is deterministic.
- Get the tolerations from the deployment instead of the pod as pod tolerations
are sometimes modified by the system. We only want user provided tolerations
in our deployment.
- Modify tests to try duplicate keys from different structs. The old way tested duplicate
keys using the same structs.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
Change the canaries to use the rook image instead of the busybox image to
run `sleep infinity`. This removes our dependency on an external image that
we do not control.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
The clusterdisruption starts up before the Ceph cluster is created and
during the initilization, it'll display invalid error messages such as:
2019-10-14T04:22:21.243679454Z 2019-10-14 04:22:21.243668 E | clusterdisruption-controller: could not check cluster health: failed to get status: exit status 1
2019-10-14T04:22:22.524222494Z 2019-10-14 04:22:22.524179 I | exec: Running command: ceph status --connect-timeout=15 --cluster=openshift-storage --conf=/var/lib/rook/openshift-storage/openshift-storage.config --keyring=/var/lib/rook/openshift-storage/client.admin.keyring --format json --out-file /tmp/605071670
2019-10-14T04:22:22.617370007Z 2019-10-14 04:22:22.617321 I | exec: Error initializing cluster client: ObjectNotFound('error calling conf_read_file',)
That error message is invalid since the cluster is not up and running
YET.
So let's simply log a message.
Signed-off-by: Sébastien Han <seb@redhat.com>
This change is necessary drain canary pod needs to be schedulable on
the same nodes as the OSDs. Since it affects multiple OSDs, we need to
gather tolerations from multiple OSDs.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
This commit changed mdb controller to make use of cephCluster's namespace instead of cephCluster's name for checking the ceph health as the ceph config is located in the cephCluster's namespace folder which can makes mdb controller fail when rook operator is deployed in any other namespace apart from rook-ceph.
Signed-off-by: Ashish Ranjan <aranjan@redhat.com>
This commit wraps the existing reconcile logic for mdb and machinelabel controller with a secondary reconcile func for better logging as rook's logging mechanism is not compatible with controller-runtime logging interface.
Signed-off-by: Ashish Ranjan <aranjan@redhat.com>
Add support for MachineDisruptionBudget controller and MachineLabel controller required for fencing
in OpenShift. This will ensure that machines are only fenced and OSDs are only stopped
when Ceph is in a healthy state.
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: travisn <tnielsen@redhat.com>
The log messages for orchestrations being skipped due to nodes still
being schedulable or unschedulable is filling the logs and not necessary.
The log message for reconciling is now debug, and also fixed another
formatting log message.
Signed-off-by: travisn <tnielsen@redhat.com>
Rohan CJ
Travis Nielsen
Sébastien Han
Ashish Ranjan