since node fencing is disabled in rook for sometime
and this feature is implemented in csi. So, this
commits remove obsolete code related to node loss
Signed-off-by: subhamkrai <srai@redhat.com>
The logger names are seen on each line of logging. As
long as they are unique, they really don't need to be
long and descriptive, so let's make them a bit more
concise, and independent from controller names.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator settings loaded from the configmap have proven
inefficient for load time and frequently checking the configmap.
To avoid this ineffenciency, the configmap is only loaded once
each time it is created or updated. The values in the configmap
are applied as environment variables, which then are very efficient
to query throughout the various controllers, without needing
to be concerned about loading the configmap again.
Co-authored-by: Dmitry Mishin <dmitry.mishin@gmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For nodes that are explicitly requested for deploying
OSDs, they will be skipped temporarily if not
schedulable or not ready. A future reconcile is
expected to schedule them. Allow the OSDs to be
scheduled even on these nodes that are temporarily
down, even if it blocks the reconcile from completing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Disabling the RBD and CephFS fencing
in Rook for now as its having bugs where
Rook is blocklisting wrong IP address
due to timing issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently, it was blocking both the mds ip
instead its should block ip of of mds which is in
the same node which is down.
Signed-off-by: subhamkrai <srai@redhat.com>
cephFs fencing needs more testing and bug fixes to make
it in production. We tried multiple fixes but still it need
more work. So, removing the cephfs networkFence realted code
will rever this commit when we start working on this again.
Signed-off-by: subhamkrai <srai@redhat.com>
earlier, we were only checking the specs of networkFence to see if
unfencing is done before proceeding with deletion but we should also
check the status of networkFence before proceeding with deletion.
Signed-off-by: subhamkrai <srai@redhat.com>
to keep the code consistent and to propogate
more details about error rook uses error wraping.
updating current code whereever its required to
wrap the details.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The cephfs PVC might exist on the kubernetes
node object but due to some timing issues the
ip might not be visible on the ceph cluster or
the client might already got evicted or disconnected
from ceph cluster. In this case we will not be
able to get IP details for the subvolume and we dont
have any check for empty ip's in the code and rook
tries to create NetworkFence CR with empty Ip's
and the NetworkFence will get moved to the Failed state.
This PR adds the necessary check and logging to prevent
this one.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
since networkFence is a cluster-based resource so that we don't need
the namespace and ownerReferences as it cause garbage-collector errors.
Also, now we create the networkFence with clusteUID label so when doing
cleanup we match the cephCluster uid and networkFence label clusterUID.
Signed-off-by: subhamkrai <srai@redhat.com>
since networkFence is a cluster-based resource so that we don't need
the namespace and ownerReferences as it cause garbage-collector errors.
Signed-off-by: subhamkrai <srai@redhat.com>
this commit updates the name of network fence CR to
node name + driver name, which will allow the creation of
multiple network fence per drivers on the same node, and
modifies the unit test and deletion procedure for the
same.
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
when creating networkFence, rbd command was loading
admin config and hence running rbd command use client.admin
in case of external cluster also. With this commit instead
of client.admin user it will use what is being passed to
config.
Signed-off-by: subhamkrai <srai@redhat.com>
we need to return a valid CIDR IP
to get it fenced properly, the previously
returned one was like `10.63.0.5:0/32` which
is not a valid CIDR, removing `:0` to look like
proper IP CIDR.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
when node fencing is triggered and a negative case where
one deployment pod consuming pv not provisioned by csi
and another deployment pod with rbd rwo are on the same
node and node fencing is triggered.
In this case rook operator panics since it is trigger to get
field `pv.Spec.CSI.Driver` which will be nil.
Also, skip static pv as there is no `imageName` in the spec
rbd status command will fail.
Signed-off-by: subhamkrai <srai@redhat.com>
During faster node loss recovery,
Rook needs to check and blocklist only rbd pvs
that do not support multinode access modes such as
ReadOnlyMany(ROX) and ReadWriteMany(RWX).
This commit adds check for the same.
Signed-off-by: Rakshith R <rar@redhat.com>
in the existing node watcher, we'll check for node update
event and see if there are `out-of-service` taints are applied
and `ROOK_WATCH_FOR_NODE_FAILURE` is enabled in rook-ceph-operator-configmap,
if then we'll create the networkFence cr and delete the cr if nodes come back.
And, added the unit test too.
Signed-off-by: subhamkrai <srai@redhat.com>
When a node is being skipped for creation of OSDs, log a message
that indicates whether it was from the node being unschedulable,
the node is not ready, or the node does not meet the placement
criteria specified in the cluster CR.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The cluster info is important context for the cluster controller to
create the cluster, and all the fields must be properly set.
A test cluster name was being set temporarily, resulting in
mons incorrectly getting the wrong cluster CR name. There is no
known issue from the temporary value, it was just exposed by
https://github.com/rook/rook/pull/8678 setting the value to a label.
Now the functions are more clearly named so only unit and
integration tests should be using the test value for the cluster
name where it is not important.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the op is initializing, the ceph config is not ready and thus we
should reconcile. This avoids a double reconcile.
Signed-off-by: Sébastien Han <seb@redhat.com>
The VolumeSource type was only used internally when loading
the storageClassDeviceSets. The volumeSource element is never
expected to be specified in the CephCluster CR, so it is now
removed from the schema. The VolumeSource type is renamed to
deviceSet for clarity.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the node get added to the cluster the operator always watches for node events and starts a reconcile when a node is ready.
Checking the Node if they already have OSDs present and stopping them from reconcile.
Closes: https://github.com/rook/rook/issues/7534
Signed-off-by: parth-gr <paarora@redhat.com>
OSDs not automatically started when adding nodes to existing cluster.
Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush map before querying for the node in the crush map, and reconcile when new nodes get added.
Closes: https://github.com/rook/rook/issues/7462
Signed-off-by: parth-gr <partharora1010@gmail.com>
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
When a node is not tolerable, also output the node name so it's clear
which not it is which is not tolerable.
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
This change is the first part to improve the watch OSD nodes logic in
the Rook Ceph operator.
This commit changes the "is node valid check" to use the OSD placement
instead of the all placement. The OSD placement is "merged" with the
all placement already.
Second step will be to skip the first "iteration" of the "nodes added"
events and then start the actual watch logic.
Partially resolves#6111.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This is the final conversion to controller-runtime conversion. This time the
CephCluster CRD has been converted to use the controller-runtime
library.
The controller incorporates all the previous watchers too, so the Node
and hot-plug configmap are been watched too.
Only the operator setting configmap is not being watcher since it's not
related to the CephCluster CRD.
Not only the patch converts to controller-runtime but also tries to
re-organize the tree of the repo to actually make the code more
readable and have better functions/methods/tests separations.
Closes: https://github.com/rook/rook/issues/4939
Signed-off-by: Sébastien Han <seb@redhat.com>
subhamkrai
Travis Nielsen
Joshua Hoblitt
Artem Torubarov
Madhu Rajanna
momantech
Riya Singhal
guoguangwu
Rakshith R
parth-grandBlaine Gardner
Josh Soref
Divyansh Kamboj
Sébastien Han
parth-gr
Mateusz Gozdek
Thomas Bechtold
subhamkrai
Alexander Trost