This fixes the resync of the Deployment of the OSD pod that uses
an LV-backed PV.
`getOSDInfo()` reads the configuration of the OSD pod from the existing
Deployment, but it overlooked this case.
Closes: https://github.com/rook/rook/issues/5139
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
unhandled type assertions can lead to runtime panic. using
the boolean 'ok' flag to handle failure to assert type.
Issue: #4566
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
When the CephCluster CR gets updated with a new image version, we need
to notify the child controllers of that upgrade.
For this, we set a 'ceph_version' label on the CR itself which will
effectively trigger a reconcile based on an update event.
This needs to be revisited and see how we can make use of
`EnqueueRequestsFromMapFunc` which might be a better approach using
controller-runtime built-in.
Closes: https://github.com/rook/rook/issues/5153
Signed-off-by: Sébastien Han <seb@redhat.com>
You can now use Rook along with Multus. Multus must be up and running
and the right ressources must exist such as NetworkAttachmentDefinition
CR.
The Cluster CR spec has new fields to work with multus:
network:
provider: multus (or 'host' for hostNetworking)
selectors:
public: NetworkAttachmentDefinition name
cluster: NetworkAttachmentDefinition name
If only a single NetworkAttachmentDefinition is provided Rook will use
both anyway for the Ceph traffic.
Please refer to the doc to learn more.
Closes: https://github.com/rook/rook/issues/4716
Signed-off-by: Sébastien Han <seb@redhat.com>
In case of multiple clusters, we don't want to delete all the mon directories under the dataDirHostPath during cluster cleanup.
This PR deletes the mon directory only if the montior secret key matches.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Thanks to https://github.com/ceph/ceph/pull/33633, when running on
Octopus, we don't need to use `--pid=host`. This means we are not using
the host PID but use the PID namespace of the pod.
Signed-off-by: Sébastien Han <seb@redhat.com>
This type was a string already and was just making us doing string()
calls all the time to it's not worth it.
Signed-off-by: Sébastien Han <seb@redhat.com>
Now Kubernetes will perform liveness checks on mon, mds and osd daemons.
The command will:
* call the socket (check for existence)
* execute a command and check the return code (success if 0)
This handles the case where the daemon is stuck locally and
unresponsive. It's unlikely but not impossible.
These checks bring more robustness to the implementation.
rbd-mirror and nfs have been leftover for the following reason. The
rbd-mirror socket name is different from other daemons (could be fixed
though): /run/ceph/ceph-client.rbd-mirror.a.1.94362516231272.asok also,
the command to call would need to be changed from "status" to "rbd
mirror status" so we can keep this for a later.
The nfs ganesha has no socket only a PID file which doesn't mean much.
No PID means the process does not run so Kubernetes will already handle
this and the pod will crash loop.
Signed-off-by: Sébastien Han <seb@redhat.com>
In order to ensure proper clean up of all the rook-ceph data when the cluster is deleted, we need to clean up the dataDirHostPath (var/lib/rook)
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
In order to perform full automation of cephcluster deletion safely
and protect against catastrophic data loss, the end user must be
able to signify that they intend to irrecoverably delete the data
in their cluster. The cleanupPolicy field of the cluster spec is
intended to communicate this. This patch does not implement
automated deletion, but only creates the spec field and the
safety feature of halting orchestration other than deletion on a
cluster with a set cleanup policy value.
Partially-fixes: #3222
Signed-off-by: Elise Gafford <egafford@redhat.com>
The CrashCollector pod remains in pending state indefinitely after
running on a k8s node that was deleted. The code now deletes the
deployment after the node is deleted.
Signed-off-by: rohan47 <rohgupta@redhat.com>
The CRD watcher has been replaced by the new controller-runtime
framework.
This brings robustness in our operator, meaning that any resources that
are modified will be reconciled into the desired state.
Closes: https://github.com/rook/rook/issues/4941
Signed-off-by: Sébastien Han <seb@redhat.com>
The CRD watcher has been replaced by the new controller-runtime
framework.
This brings robustness in our operator, meaning that any resources that
are modified will be reconciled into the desired state.
Closes: https://github.com/rook/rook/issues/4940
Signed-off-by: Sébastien Han <seb@redhat.com>
All processes created with the exec package will now only have debug logging
for the commands and their output. The osd prepare job relies heavily
on many of these commands so we can enable debug logging for the entire
job.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The helpers for executing a process have long required an actionName
param which is not being used. Now we remove the old param
while also cleaning up various other usages of the exec
package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph commands are now only written to the log in debug mode.
For commands that change the system state we now ensure that
a useful log entry is written. If all the details of the ceph
commands are needed, debug logging should still be enabled.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The CRD watcher has been replaced by the new controller-runtime
framework.
This brings robustness in our operator, meaning that any resources that
are modified will be reconciled into the desired state.
Closes: https://github.com/rook/rook/issues/4937
Signed-off-by: Sébastien Han <seb@redhat.com>
As part of the transition to support Ceph release **as of** Nautilus, we
left over that portion of code.
We don't need it anymore.
Signed-off-by: Sébastien Han <seb@redhat.com>
The OSD needs to talk to the udev socket to get information about a device.
like serial, vendor etc. So that it can populate the output of:
`ceph osd metadata $n` correctly.
Signed-off-by: Sébastien Han <seb@redhat.com>
The OSDs pick up on several topology labels for CRUSH hierarchy.
The GA label topology.kubernetes.io was partially implemented, but
not picked up by the OSDs. Now the OSDs will pick up both the topology
labels from pre-1.17 such as failure-domain.beta.kubernetes.io/zone
and topology.kubernetes.io/zone.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Now, the CephObjectStoreUser CRD is managed with the controller-runtime.
So the watcher is outside of the main controller reconciliation loop of
CephCluster which brings numerous benefit such as:
* having its own reconciliation loop
* won't block anything from the main CephCluster controller loop
* fast than waiting for CephCluster loop to completion
Partially close: https://github.com/rook/rook/issues/1981
Signed-off-by: Sébastien Han <seb@redhat.com>
In the cluster CR the nodes element will be ignored under storage when
useAllNodes is true. The nodes will only be applied when useAllNodes is
false. It is a common usability problem that users can't figure out
why their nodes are not configured. At least this log entry can give
them a clue.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd prepare job is now always working with devices so
we always need to mount /dev and run privileged. There is
no scenario for not mounting /dev since filestore support
was removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The communication between the operator and OSD prepare job
is done through updates to the configmap. If an update fails
to the configmap, we are anyway going to need to check the logs
for failure. So let's just log the error and make a best effort
to continue the orchestration. If the failure was temporary,
the next update to the status would succeed and likely continue
working as expected.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Now, the CephBlockPool CRD is managed with the controller-runtime.
So the watcher is outside of the main controller reconciliation loop of
CephCluster which brings numerous benefit such as:
* having its own reconciliation loop
* won't block anything from the main CephCluster controller loop
* fast than waiting for CephCluster loop to completion
Partially close: https://github.com/rook/rook/issues/1981
Signed-off-by: Sébastien Han <seb@redhat.com>
Prior attempts to delete CSI drivers using owner references to the
Ceph ConfigMap resulted in garbage collection of the driver
resources as described in rook#4590. This patch manually deletes these
resources by name as a stopgap to provide an appropriate user
experience while the team investigates this issue further.
Resolves: rook#4824
Signed-off-by: Elise Gafford <egafford@redhat.com>
This commit will add a phase and also a condition to display a Failed status on the external rook-ceph cluster if its failed to establish a connection.
Signed-off-by: Nizamudeen <nia@redhat.com>
The rook types used across the storage providers moved from the v1alpha2
package to the v1 package. This commit points the packages at their new
location. Implementation is expected to remain unchanged.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit is to handle all those unhandled errors which raises the gosec warning.
Fixed G104: Unhandled Errors are handled now
Signed-off-by: Nizamudeen <nia@redhat.com>
The ceph/ceph:v14.2.7 image is released so we can pick these fixes
up as the recommended version of ceph.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the OSD on PVC is backed by a metadata block PVC, the Ceph CRUSH
device class should be set to something else rather than the rotational
property of the drive.
Closes: https://github.com/rook/rook/issues/4881
Signed-off-by: Sébastien Han <seb@redhat.com>
When the cluster is upgraded, the child controllers are notified with the
callback to the ParentClusterChanged() method. These should be made
in a separate goroutine so that the cluster controller is not blocked
on the child controllers and also so child controllers aren't blocked
on each other for the upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Travis Nielsen
morimoto-cybozu
Umanga Chapagain
Sébastien Han
Santosh Pillai
Elise Gafford
rohan47
Nizamudeen