Commit Graph
101 Commits
Author SHA1 Message Date
Madhu Rajanna 05c75b7084 ceph: add support to disable snapshotter sidecar
In some cases the user dont want to run snapshotter
container either for CephFS or RBD. In that case the
user wont install the required snapshot CRD's due
to that the snapshotter sidecar container produces
lot of noisy logs.
Snapshotter will be enabled by default for both
CephFS and RBD, but with this PR we are providing
an option to disable snapshotter sidecar deployment
either for CephFS or RBD.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-01-21 15:29:55 +05:30
Travis Nielsen 713fc80d03 Merge pull request #6769 from Madhu-1/csi-v3.2.0
ceph: update csi to v3.2.0
2020-12-11 13:45:15 -07:00
Blaine Gardner 2e62b5fb3b ceph: update upgrade docs to allow diff namespaces
Update the upgrade docs to allow users with different cluster/operator
namespaces to use a `sed` command and the new manifest meta-comments for
updates.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-11 11:35:41 -07:00
Madhu Rajanna 5392bc0d07 ceph: update csi to v3.2.0
updating cephcsi to latest release version

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-11 20:22:36 +05:30
Blaine Gardner 26c9512962 ceph: add namespace meta-comments to manifests
Add meta-comments to manifests to allow basic templating via `sed`.

Add the following types of meta-comments:
- # namespace:X
  - A basic namespace
  - replace the field with a namespace for X
- # serviceaccount:namespace:X
  - A service account namespace for SCC
  - e.g., "system:serviceaccount:<ns>:rook-ceph-system"
  - Replace the namespace "<ns>" with with a namespace for X
- # provisioner:namespace:X
  - A provisioner identifier with namespace prefix
  - e.g., "<ns>.cephfs.csi.ceph.com"
  - e.g., "<ns>.ceph.rook.io/bucket"
  - Replace the namespace "<ns>" with a namespace for X

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-09 16:21:31 -07:00
Alexander Trost b04f882372 ceph: fix pod labels set on csi components
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-11-25 14:55:36 +01:00
Alexander Trost 34930020ab ceph: allow pod labels to be added to CSI components
This adds the functionality to add custom pod labels to the CSI
components through the operator configuration way of env vars or config
map.

Resolves #6593

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-11-12 13:06:24 +01:00
Madhu Rajanna 8f5196a559 ceph: add privileged securityContext to cephfs deployment
cephfs csi driver still uses the ceph CLI for all the
ceph fs operations, when cephcsi gets a request from
the kubernetes it will internally starts threads for
each call. in cephcsi we are setting the pids
limit to max, to set this value the container has
to be running as the privileged one.

if we go with default pid limit we will face core dump
due to resource crunch.

without privileged access we will see Failed to set new
PID limit to -1: open /sys/fs/cgroup/pids/kubepods.slice/
kubepods-besteffort.slice/kubepods-besteffort-pod1029ed37_bebc_466d_9e53_79e7323a67ed.slice/crio-9ef6768d027916f414bf30b54ef13eebad8efd0aa4741d97618affa838d7b4c0.scope/pids.max: permission denied

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-06 09:40:18 +05:30
Satoru Takeuchi e22bedbd9b ceph: reduce the number of provisioner pods in small K8s cluster
The default number of provisioner pods are 2. When there is only one node,
for example the CI environment, one of these becomes Pending state.
Although it's harmless, it's better to reduce this value to 1.

Closes: https://github.com/rook/rook/issues/6294

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-15 18:25:27 +00:00
Mudit Agarwal c547b36aa0 ceph: update csi sidecar images
This PR makes the changes in csi templates and
upgrade documentation required for updating
csi sidecar images.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-09-28 18:58:52 +05:30
Madhu Rajanna 0aa0957bf1 ceph: remove preStop from daemonset template
The lifecycle preStop hook fails on container stop / exit
because /bin/sh is not present in the driver registrar container
image.

the driver-registrar will remove the socket file
before stopping. we dont need to have any preStop hook
to remove the socket as it was not working as expected

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-24 19:25:44 +05:30
Humble Chirammal 97a3283099 ceph: add snapshot create/delete and restore doc for cephfs
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-13 22:17:01 +05:30
Humble Chirammal a795ff01e0 ceph: add cephfs examples yamls for restore,clone and snapshotclass
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-10 11:31:10 +05:30
Humble Chirammal dacc013cc9 ceph: add csi-snapshotter sidecar to cephfs deployment
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-10 11:27:05 +05:30
Unai Arrien f6087b280a docs: add provisioner prefix namespace comment in ceph csi examples
Signed-off-by: Unai Arrien <unaittxu@gmail.com>
2020-08-07 15:39:36 +02:00
Madhu Rajanna 1989e0d8c5 ceph: remove csi support for kubernetes 1.13
as the kubernetes 1.13 is EOL and there is no major
functionalities available in 1.13 (resize,snapshot,clone
metrics etc) we are removing the support for kubernetes
for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-05 21:19:59 +05:30
Madhu Rajanna 6aef231896 ceph: add pvc-clone.yaml for rbd
added pvc-clone.yaml for rbd pvc volume
cloning.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-27 13:46:39 +05:30
Madhu Rajanna cc3b4e6906 ceph: remove check for kubernetes version
as we are going to deploy snapshotter by
default we are going to remove the kubernetes
version check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-27 13:45:56 +05:30
Madhu Rajanna 73e1ee41c3 ceph: remove unwanted metadatastore and mountcache flag
metadatastore and mountcache are required for cephcsi v1.0.0

as rook doesnot used ceph-csi v1.+ version
we can remove this flag. Removing these flags doesnot
affect the functionality of anything.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-22 14:36:11 +05:30
Travis Nielsen ba590396e8 docs: remove references to xfs for ceph volumes
When mounting a ceph volume, xfs may cause issues with potential
deadlocks if the volume is on the same node as the OSDs. To avoid
this issue the default recommendation is ext4.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-08 22:31:11 -06:00
Madhu Rajanna 37ddeda7af ceph: remove snapshotter sidecar from provisioner sts
ceph-csi will be deployed as statefulset if the kubernetes
version is less than 1.14, as snapshotter will work
with kube 1.17+, we dont need snapshotter sidecar
deployed as part of statefulset.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 07:20:04 +05:30
Madhu Rajanna 9f596e4c9f ceph: update snapshot yaml as per snapshot beta
in kubernetes 1.17+ the snapshot CRD is changes
we need to update snapshot yaml file as per new CRD

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 07:20:04 +05:30
Madhu Rajanna e01cb7f527 ceph: update snapshotclass as per snapshot beta
few Parameters are changed in snapshotclass in
beta version.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 07:20:04 +05:30
Madhu Rajanna ec3ab55d94 CSI: Set PriorityClassName for CSI provisioner pods
provide an option to set PriorityclassName on the
CSI provisioner pods to avoid pod eviction, as the CSI provisioner
pods a critical in system for storage. the default value
for the CSI provisioner pods is set to system-cluster-critical.

provisioner pods are less priority than plugin pods.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-06 15:45:06 +05:30
Madhu Rajanna cc6ec58891 CSI: Set PriorityClassName for CSI plugin pods
provide an option to set Priorityclassname on the
CSI plugin pods to avoid pod eviction, as the CSI plugin pods are
critical in system for storage. the default value
for the CSI plugin pods is set to system-node-critical.
more info can be found at
https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-06 15:45:01 +05:30
Madhu Rajanna a0270c57ea CSI: rename attacher sidecar name to follow other sidecar names
Renamed the csi attacher sidecar container name
to follow the nameing convention of other
sidecar containers of the CSI

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-29 10:26:56 +05:30
Chu Yong Mark Lee a018d89d9c csi: Update cephcsi driver to v2.0.1
Update cephcsi driver to v2.0.1 to allow nodes with
Ceph persistent volumes to shutdown cleanly. This requires
an additional bind mount (/run/mount) in cephfsplugin
and rbdplugin templates.

Signed-off-by: Chu Yong Mark Lee <markleechuyong@gmail.com>
2020-04-03 11:05:41 -06:00
Sébastien Han dab9233c8b ceph: add CRD setting for pool size 1
As of Octopus, Ceph will prevent you from creating a pool with a
replica size of 1. Allowing such pool could lead to data loss, so enable
the new option: requireSafeReplicaSize: false if you are **ABSOLUTELY**
certain that is what you want.

Closes: https://github.com/rook/rook/issues/4889
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-02-25 16:28:58 +01:00
Travis Nielsen 57fb7cda2e Merge pull request #4714 from Madhu-1/log-level
CSI: Add LogLevel for csi templates
2020-01-31 11:54:32 -07:00
Sébastien Han 720c06cfec Merge pull request #4791 from galexrt/pickup_3689
doc: fix CephFS CSI StorageClass name
2020-01-31 14:18:13 +01:00
Alexander Trost a4f9981d83 doc: fix CephFS CSI StorageClass name
This fixes the StorageClass name example and doc to be `rook-cephfs`
instead of just `csi-cephfs`.

Closes #4130.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-01-31 11:49:35 +01:00
Madhu Rajanna 48a6e493e8 CSI: Add LogLevel for csi templates
Currently there is no option to turn on and off the
log level verbosity in csi containers, This PR adds
the functionality to select logging levels for csi containers.

Fixes: #4690

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-31 15:04:11 +05:30
Madhu Rajanna 5a685fba77 CSI: Add support for rbd erasure coded pool
In ceph-csi v2.0.0 added a support for specifing
the erasure coded pool in storageclass which will be
used to store data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 23:50:23 +05:30
Madhu Rajanna cf16184899 CSI: Add support for CSI volume expansion
This PR makes the necessary changes required
for volume expansion support for both cephfs and
rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 23:50:23 +05:30
Madhu Rajanna 06d2dd2994 CSI: update csi-attacher from v1.2.0 to v2.1.0
external-attacher v2.x is not compatible with v1.x
This PR makes the require changes in csi templates
and upgrade documentation for easily upgrade

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 23:46:24 +05:30
Madhu Rajanna 29f8749ded CSI: Remove containerized flag from rbd daemonset template
In ceph-csi release v2.0.0, deprecated `containerized` flag
is removed.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 12:20:13 +05:30
Madhu Rajanna 47cc1adf62 CSI: run containers as non-privileged in provisioner pod
currently we are running plugin container as privileged
in the provisioner pod for both cephfs and rbd.

we are also running snapshotter container as privileged
in the provisioner pod for rbd.

as we are executing the ceph and rbd commands in provisioner
pod we dont need to be privileged, This PR removed the
privileged capability from the templates.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-13 09:07:21 +05:30
Madhu Rajanna 7a7e65a9b2 CSI: use empty dir to store provisioner socket file
currently we are making use of hostpath
directory to store the provisioner socket, as this
socket is not needed by anyone else other than containers
inside the provisioner pod using empty directory
to store this socket is  the best option.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-10 08:31:18 +05:30
Madhu Rajanna d9dd91d9fb CSI: Add privileged securityContext to all daemonset containers
On systems with SELinux enabled, non-privileged containers
can't access data of privileged containers.
Since the socket is exposed by privileged containers,
all sidecars must be privileged too.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-10 08:31:12 +05:30
Travis Nielsen 92e7f12451 cephcsi: add upgrade documentation for csi driver
If using ceph-fuse or rbd-nbd the application pods will need
to be restarted during upgrade. This adds a description of
the steps necessary to ensure the applications remain healthy
during upgrade.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2019-12-16 23:16:05 -07:00
Madhu Rajanna 8be3cdd10e CSI: cephfs and rbd daemonset upgrade strategy
CSI nodeplugins, specifically when using cephfs FUSE or
rbd-nbd as the mounters, when upgraded, will
cause existing mounts to become stale/not-rechable
(usually connection timeout errors).

This is due to losing the mount processes running within the
CSI nodeplugin pods.

This PR add updated the Daemonset update strategy
based on the ENV variable to take care of above issue
with some manual steps

Moreinfo: https://github.com/ceph/ceph-csi/issues/703

Resolves: https://github.com/rook/rook/issues/4248

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-16 23:15:56 -07:00
Madhu Rajanna 85d622dc2f CSI: Add ENV variable to use ceph kernel client
In ceph-csi v1.2.2 we have add a flag `forcecephkernelclient`
to force use Ceph Kernel clients on kernel < 4.17 which
support quotas for Cephfs

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-16 12:36:31 -07:00
Madhu Rajanna cc418bce3b CSI: Ability to disable snapshotter from ceph-csi rbd
we need to set a ENV `CSI_RBD_ENABLE_SNAPSHOTTER`
variable to "false" in operator template to disable
deployment of snapshotter sidecar in ceph-csi rbd
provisioner.

Fixes: #4401
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-03 13:27:29 +05:30
ShyamsundarR 48a0decc49 CSI: Make attach mandatory in ceph CSIDriver object
A recent PR #4172 added the ability for rook to create
the CSIDriver object on a kubernetes cluster. The change also
made it so that attachRequired value in the ceph CSI drivers
were set to false.

This leads to the entire attach skipping the AD controller
that deals with RWO attachment enforcement. Thus, if a
CephFS volume is marked RWO, 2 pods on different hosts can
end up mounting the same and consuming it at the same time.

Further for RBD, currently the image watchers are the only ones
that prevent a double mount of an image, other than the AD controller
enforcement of single attach for RWO RBD PVs. This has proven
not to be reliable in the face of MON failures, as the watcher
information is lost, and a second mount is allowed.

Thus, we need to bring back the attach requirement to leverage
kubernetes checks and balances for RWO volume type, and
prevent unwanted multi-attach use of the same.

This PR hence undoes some of the changes introduced by the
PR #4172, to bring the attacher side car back, and also declare
the attachRequired for both RBD and CephFS CSI plugins as true.

NOTE: Initially the attachRequired was set to false as this
improved attach times considerably on kubernetes, as the
entire phase was being avoided.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-11-15 16:15:53 -05:00
Madhu Rajanna 9b368ef15c CSI: Create CSIDriver Object
This will be helpful to list the CSI drivers
deployed in the kubernetes cluster. if we have
N number of CSI drivers deployed in the cluster,
the user can query kubectl get csidriver and get
the list of registered CSI drivers and it requires
kube 1.14+

This removes the deployment of Attacher sidecar container,
as we are setting the attacher required to false during
the CSIDriver object creation.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-11-06 11:18:03 +05:30
Sébastien Han 98bc8e4ca0 ceph: revert "CSI: Add support for CSIDriver object creation"
I merged https://github.com/rook/rook/pull/4169 too quickly and didn't
noticed we were supposed to wait for csi 1.2.2.

This reverts commit 520ad44d71.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-24 12:30:11 +02:00
Madhu Rajanna 520ad44d71 CSI: Add support for CSIDriver object creation
This will be helpful to list the CSI drivers deployed
in the kubernetes cluster. if we have N number of
CSI drivers deployed in the cluster, the user can
query kubectl get csidriver and get  the list of
registered CSI drivers and it requires kube 1.14+
and also the functionality is disabled by default

More info here: https://github.com/ceph/ceph-csi/pull/698

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-24 15:08:26 +05:30
Sébastien Han fcc2e21faf ceph: rework csi keys and secrets
We know create 4 secrets containing 4 ceph keys where each of them have
limited permissions to access the cluster.

Closes: https://github.com/rook/rook/issues/4074
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-22 15:14:28 +02:00
awalker60 0345468b6b ceph: fix for csi-snapshotter always pulling
This switches csi-snapshotter to IfNotPresent which works better in production and offline environments.

Signed-off-by: awalker60 <awalker125@users.noreply.github.com>
2019-10-15 10:41:03 +01:00
Madhu Rajanna 0056e58ba6 csi: update cephfs registration directory name
when multiple CSI drivers are running in a cluster,
each should use unique socket for registration.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-10 10:52:27 +05:30