There are two issues at odds with the cephfs driver. The
first is that older kernels less than 4.17 do not support
quotas so PVC sizes will not be enforced with the kernel
driver in that case. If the FUSE client is used instead, there
are issues during upgrade because the client mount will be
disrupted and the client pods will have to restart.
To prevent client pods from breaking during upgrade, we enable
the kernel driver by default. If the quotas are critical,
the admin will need to take steps during upgrade to prevent
breaking the applications.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In ceph-csi v1.2.2 we have add a flag `forcecephkernelclient`
to force use Ceph Kernel clients on kernel < 4.17 which
support quotas for Cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
gp2 devices present themselves as SSDs,
but have much more variable performance.
Ceph attempts to adjust some settings based on the drive type,
HDD/SSD
In this case client I/O can be impacted too much by assuming gp2 devices are like other SSDs
(an example is https://bugzilla.redhat.com/show_bug.cgi?id=1765923#c6)
In order to prevent this, rook can set the following for OSDs
if the new CR setting `tuneSlowDeviceClass` is set to `true`:
```
osd_recovery_sleep = 0.1
osd_snap_trim_sleep = 2
osd_delete_sleep = 2
```
Closes: https://github.com/rook/rook/issues/4298 and https://bugzilla.redhat.com/show_bug.cgi?id=1771047
Signed-off-by: Sébastien Han <seb@redhat.com>
An owner reference cannot be set with BlockOwnerDeletion: true
if the operator does not have privs to add a finalizer to the parent.
The discover resources don't need to block deletion based on the
parent so we remove that property from the owner reference.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Removing an OSD unintentionally can result in data loss,
which is to be prevented at all costs. Rook's functionality
to remove OSDs automatically when a node was removed from the
cluster CR has led on several occasions to users having OSDs
unintentionally removed from their cluster.
If the admin decides they really need to remove OSDs they should
do it as a one-time action that they intentionally run
rather than the operator attempting to manage the deletion
with desired state. The risk of misinterpreting the desired
state, even with a typo, is too great.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
It's not 100% safe since before Nautilus osd safe-to-destroy command
didn't have any json output. So added a version check.
Signed-off-by: rohan47 <rohgupta@redhat.com>
Add owner reference to the discover config map so that when the
discover pod goes away we also remove its configmap.
Signed-off-by: Sébastien Han <seb@redhat.com>
Attach owner reference to the discover daemon set so that when the
opetator goes away, the discover pod gets removed too.
Signed-off-by: Sébastien Han <seb@redhat.com>
As of 14.2.5, the crash collector pod will use the client.crash user
instead of the client.admin keyring to post crash dumps.
Closes: https://github.com/rook/rook/issues/4388
Signed-off-by: Sébastien Han <seb@redhat.com>
Both --id and --name do the same thing, so let's use one only: --id
which is shared accross all the daemons already.
Signed-off-by: Sébastien Han <seb@redhat.com>
rgw needs the fronted on its cli line to select the fronted as well the
port. This option is not supported in the mon config store.
Closes: https://github.com/rook/rook/issues/4471
Signed-off-by: Sébastien Han <seb@redhat.com>
As performed with mon, gateways from same ceph object store
must not run on the same host if hostNetwork is set to true
Signed-off-by: n.fraison <n.fraison@criteo.com>
If editing the cluster CR we must update the spec to reflect its new
content. Now the spec is updated so we can update with an unsupported
image as well as setting allowUnsupported: true.
It was not possible before we had to restart the operator, now we can do
it as an update.
Signed-off-by: Sébastien Han <seb@redhat.com>
We must not default the selectors to the labels since labels can be
changed and selector cannot until the deployment is updated.
This way the deployment was failing with:
2019-12-10 10:48:13.414438 E | ceph-crashcollector-controller: node reconcile failed on op "unchanged": Deployment.apps "rook-ceph-crashcollector-minikube" is invalid: spec.template.metadata.labels: Invalid value: map[string]string{"app":"rook-ceph-crashcollector", "ceph-version":"15.0.0", "ceph_daemon_id":"crash", "crashcollector":"crash", "kubernetes.io/hostname":"minikube", "node_name":"minikube", "rook-version":"v1.2.0-3.g8d3b85306.dirty"}: `selector` does not match template `labels`
Signed-off-by: Sébastien Han <seb@redhat.com>
The loop wasn't using seconds for its retry so the crash controller
would simply crash since the version is used later in the pod spec.
Signed-off-by: Sébastien Han <seb@redhat.com>
Just as with the mon, mgr, mds, rgw, and rbd-mirror daemons, do not
generate a ceph.conf in an init container for filestore OSDs.
Instead use the commandline to supply all the needed arguments for
running these OSDs.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
This change adds rook-version and ceph-version tags to crash
collector deployments, in keeping with other Ceph deployments.
Resolves: #4357
Signed-off-by: Elise Gafford <egafford@redhat.com>
ceph client CRD refactoring
Add info about client crd to documentation
Add tests for client controller CRD
Signed-off-by: Mateusz Los <los.mateusz@gmail.com>
When a node has been removed, cordoned, or is otherwise
missing from the filters that rook uses to evaluate deploying
osds on nodes, the orchestration should continue instead
of failing. Otherwise, the orchestration will stay stuck
in a retry loop if any nodes are cordoned or removed in
the middle of an orchestration.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
"OSD on PVC" doesn't work for PV backed by LV. Fixing this problem
by the following changes.
- Rook accepts LVM disk type.
- If a LV-backed device is passed, Rook/Ceph invokes
"ceph-volume lvm prepare" with "--data vg/lv"
instead of "--data /path/to/device".
- If a LV-backed device is passed, Rook/Ceph suppresses
activation/deactivation of VG that owns this LV.
Fixes: https://github.com/rook/rook/issues/4185
Signed-off-by: dulltz <isrgnoe@gmail.com>
Co-authored-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Travis Nielsen
Madhu Rajanna
Sébastien Han
Dmitry Yusupov
Anton Skriptsov
Blaine Gardner
rohan47
Mark Kogan
n.fraison
Elise Gafford
Mateusz Los
Mateusz Gozdek
Sergey Kaydalov
dulltzandSatoru Takeuchi