Previously, we were ignoring configuration changes coming from the
operator's pod env variables. We were assuming most users were using the
operator configmap "rook-ceph-operator-config" but most Helm users
don't. Now we will reconcile if a cephcluster is found during a CREATE
event (can be an operator restart or a cephcluster creation) AND no
"rook-ceph-operator-config" is found which means the operator's pod env
var are used.
Also, unit tests have been added (long due) for the predicate!
Closes: #9602, #9487, #9579
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:
* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited
As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.
A second new controller for the operator's general config has been
created, it manages:
* the logging level
* the ceph CLI command timeout
* the discovery daemon
The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.
Signed-off-by: Sébastien Han <seb@redhat.com>