Previously, the rbd-mirror daemon was integrated into the `CephCluster`
CRD. This wasn't really practical since we would have to wait for the
whole orchestration to be done to actually set it up. The same goes for
any CR update. Let's say you want to change the number of daemons, Rook
would go through mons, mgrs and osds until it get to rbd-mirror.
This triggers an undesired full orchestration.
With its own CRD this component just gains a lot more flexibility.
Closes: https://github.com/rook/rook/issues/5084
Signed-off-by: Sébastien Han <seb@redhat.com>
When a daemon is not safe-to-stop the operator will print the full
stack trace rather than a simple error message that the daemon
cannot be stopped.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSI CephFS provisioner fails to provision a Persistent Volume after upgrading from Rook Ceph Operator 1.1.2 to 1.1.7.
This issue can be fixed by removing givePoolAppTag function whenever there is an appName cephFs.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
In the v1.3 release the target port for the rgw daemons changed to 8080.
The upgrade was skipping the service update anytime the service already existed.
Now the service will be updated with every reconcile to ensure the target port
is correct.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The controllers should ignore events for status updates. The reconcile
only needs to happen when the spec is updated or the resource is marked
for deletion. Otherwise, the controllers may stay in an update loop as
the reconcile events are triggered again every time there is a status
update during the reconcile.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When updating the status on resources sometimes the update fails due to
the resource being an outdated version. This frequently occurs when
the same reconcile loop updates the status multiple times, or the finalizer
is added, or some other update to the resource. Upon the next reconcile
the error would generally go away since the status or finalizer didn't
need to be updated multiple times. But now the error is not expected
since the controller will retrieve the latest version of the resource
immediately before attempting to update it.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This fixes the resync of the Deployment of the OSD pod that uses
an LV-backed PV.
`getOSDInfo()` reads the configuration of the OSD pod from the existing
Deployment, but it overlooked this case.
Closes: https://github.com/rook/rook/issues/5139
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
The old Filesystem's pool is not getting updated when changed with the new File system.
So, we added two helper functions namely generateDataPoolNames and generateMetaDataPoolName that will generate the names of the pools and then change those pool's values using the SetPoolSize method.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
Since e396379 rook create rgw metadata pools with 8 PGs instead of
32. But the pg_autoscaler increase it back to 32 as the pg_num_min parameter
is not set on those pools.
Signed-off-by: n.fraison <n.fraison@criteo.com>
Value returned by ceph config get can contains some trailing space which leads to
issues when running other ceph command.
For ex.:
2020-04-06 15:32:43.543050 I | exec: Running command: ceph osd pool set test-1.rgw.control pg_num_min 8
--connect-timeout=15 --cluster=rook-ceph --conf=/var/lib/rook/rook-ceph/rook-ceph.config --keyring=/var/lib/rook/rook-ceph/client.admin.keyring --format json --out-file /tmp/249563089
2020-04-06 15:32:44.005018 I | exec: Error EINVAL: error parsing int value '8
Signed-off-by: n.fraison <n.fraison@criteo.com>
When the Ceph health is HEALTH_ERR the controllers will skip the
reconcile. With info level logging there needs to be an indication
of this decision to skip the reconcile, otherwise users will
wonder why their resources aren't being created.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
unhandled type assertions can lead to runtime panic. using
the boolean 'ok' flag to handle failure to assert type.
Issue: #4566
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
When the CephCluster CR gets updated with a new image version, we need
to notify the child controllers of that upgrade.
For this, we set a 'ceph_version' label on the CR itself which will
effectively trigger a reconcile based on an update event.
This needs to be revisited and see how we can make use of
`EnqueueRequestsFromMapFunc` which might be a better approach using
controller-runtime built-in.
Closes: https://github.com/rook/rook/issues/5153
Signed-off-by: Sébastien Han <seb@redhat.com>
With the release of the CSI v2.0.1 driver, this should now be the
default driver deployed by Rook. The minimum driver version
supported is still v2.0.0
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
You can now use Rook along with Multus. Multus must be up and running
and the right ressources must exist such as NetworkAttachmentDefinition
CR.
The Cluster CR spec has new fields to work with multus:
network:
provider: multus (or 'host' for hostNetworking)
selectors:
public: NetworkAttachmentDefinition name
cluster: NetworkAttachmentDefinition name
If only a single NetworkAttachmentDefinition is provided Rook will use
both anyway for the Ceph traffic.
Please refer to the doc to learn more.
Closes: https://github.com/rook/rook/issues/4716
Signed-off-by: Sébastien Han <seb@redhat.com>
Previously, delete was returning an error causing the drain-canary
to reconcile even though it no longer existed.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
In case of multiple clusters, we don't want to delete all the mon directories under the dataDirHostPath during cluster cleanup.
This PR deletes the mon directory only if the montior secret key matches.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Thanks to https://github.com/ceph/ceph/pull/33633, when running on
Octopus, we don't need to use `--pid=host`. This means we are not using
the host PID but use the PID namespace of the pod.
Signed-off-by: Sébastien Han <seb@redhat.com>
This type was a string already and was just making us doing string()
calls all the time to it's not worth it.
Signed-off-by: Sébastien Han <seb@redhat.com>
Now Kubernetes will perform liveness checks on mon, mds and osd daemons.
The command will:
* call the socket (check for existence)
* execute a command and check the return code (success if 0)
This handles the case where the daemon is stuck locally and
unresponsive. It's unlikely but not impossible.
These checks bring more robustness to the implementation.
rbd-mirror and nfs have been leftover for the following reason. The
rbd-mirror socket name is different from other daemons (could be fixed
though): /run/ceph/ceph-client.rbd-mirror.a.1.94362516231272.asok also,
the command to call would need to be changed from "status" to "rbd
mirror status" so we can keep this for a later.
The nfs ganesha has no socket only a PID file which doesn't mean much.
No PID means the process does not run so Kubernetes will already handle
this and the pod will crash loop.
Signed-off-by: Sébastien Han <seb@redhat.com>
Removed some extremely verbose debug logging that makes
the logs difficult to read when debug is enabled.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When running on SDN, the container is not privileged and the network
stack is not exposed so running the gw on port 80 will not work (not
enough privileged).
Closes: https://github.com/rook/rook/issues/5106
Signed-off-by: Sébastien Han <seb@redhat.com>
In order to ensure proper clean up of all the rook-ceph data when the cluster is deleted, we need to clean up the dataDirHostPath (var/lib/rook)
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
In order to perform full automation of cephcluster deletion safely
and protect against catastrophic data loss, the end user must be
able to signify that they intend to irrecoverably delete the data
in their cluster. The cleanupPolicy field of the cluster spec is
intended to communicate this. This patch does not implement
automated deletion, but only creates the spec field and the
safety feature of halting orchestration other than deletion on a
cluster with a set cleanup policy value.
Partially-fixes: #3222
Signed-off-by: Elise Gafford <egafford@redhat.com>
The PG count on metadata pools should default to rgw_rados_pool_pg_num_min
instead of the more general default pg count. This means rgw pools
will default to 8 PGs instead of 32 PGs, which means a lot more pools
can be created before hitting the default PG limit.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit adds all the CSI configurations to ConfigMap.
This configMap can be used in combination with Env Vars
to configure Ceph CSI drivers in rook.
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
The CrashCollector pod remains in pending state indefinitely after
running on a k8s node that was deleted. The code now deletes the
deployment after the node is deleted.
Signed-off-by: rohan47 <rohgupta@redhat.com>
The pools had some legacy structs that translated between the
ceph.v1 types used by the CRDs and the internal implementation
of the pools. This simplifies the pool implementation by removing
the intermediate model and leaving us only with the ceph v1
pool types and no unnecessary translation.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
They are cases where we let the exponential backoff retry and some where
we force our own retry. Let's properly log this information.
Signed-off-by: Sébastien Han <seb@redhat.com>
Sébastien Han
Travis Nielsen
RAJAT SINGH
morimoto-cybozu
n.fraison
Umanga Chapagain
Rohan CJ
Santosh Pillai
ShyamsundarR
Elise Gafford
rohan47