In some cases the user dont want to run snapshotter
container either for CephFS or RBD. In that case the
user wont install the required snapshot CRD's due
to that the snapshotter sidecar container produces
lot of noisy logs.
Snapshotter will be enabled by default for both
CephFS and RBD, but with this PR we are providing
an option to disable snapshotter sidecar deployment
either for CephFS or RBD.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Update the upgrade docs to allow users with different cluster/operator
namespaces to use a `sed` command and the new manifest meta-comments for
updates.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Add meta-comments to manifests to allow basic templating via `sed`.
Add the following types of meta-comments:
- # namespace:X
- A basic namespace
- replace the field with a namespace for X
- # serviceaccount:namespace:X
- A service account namespace for SCC
- e.g., "system:serviceaccount:<ns>:rook-ceph-system"
- Replace the namespace "<ns>" with with a namespace for X
- # provisioner:namespace:X
- A provisioner identifier with namespace prefix
- e.g., "<ns>.cephfs.csi.ceph.com"
- e.g., "<ns>.ceph.rook.io/bucket"
- Replace the namespace "<ns>" with a namespace for X
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This adds the functionality to add custom pod labels to the CSI
components through the operator configuration way of env vars or config
map.
Resolves#6593
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
cephfs csi driver still uses the ceph CLI for all the
ceph fs operations, when cephcsi gets a request from
the kubernetes it will internally starts threads for
each call. in cephcsi we are setting the pids
limit to max, to set this value the container has
to be running as the privileged one.
if we go with default pid limit we will face core dump
due to resource crunch.
without privileged access we will see Failed to set new
PID limit to -1: open /sys/fs/cgroup/pids/kubepods.slice/
kubepods-besteffort.slice/kubepods-besteffort-pod1029ed37_bebc_466d_9e53_79e7323a67ed.slice/crio-9ef6768d027916f414bf30b54ef13eebad8efd0aa4741d97618affa838d7b4c0.scope/pids.max: permission denied
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The default number of provisioner pods are 2. When there is only one node,
for example the CI environment, one of these becomes Pending state.
Although it's harmless, it's better to reduce this value to 1.
Closes: https://github.com/rook/rook/issues/6294
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This PR makes the changes in csi templates and
upgrade documentation required for updating
csi sidecar images.
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
The lifecycle preStop hook fails on container stop / exit
because /bin/sh is not present in the driver registrar container
image.
the driver-registrar will remove the socket file
before stopping. we dont need to have any preStop hook
to remove the socket as it was not working as expected
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as the kubernetes 1.13 is EOL and there is no major
functionalities available in 1.13 (resize,snapshot,clone
metrics etc) we are removing the support for kubernetes
for the same.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as we are going to deploy snapshotter by
default we are going to remove the kubernetes
version check.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
metadatastore and mountcache are required for cephcsi v1.0.0
as rook doesnot used ceph-csi v1.+ version
we can remove this flag. Removing these flags doesnot
affect the functionality of anything.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
When mounting a ceph volume, xfs may cause issues with potential
deadlocks if the volume is on the same node as the OSDs. To avoid
this issue the default recommendation is ext4.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph-csi will be deployed as statefulset if the kubernetes
version is less than 1.14, as snapshotter will work
with kube 1.17+, we dont need snapshotter sidecar
deployed as part of statefulset.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
in kubernetes 1.17+ the snapshot CRD is changes
we need to update snapshot yaml file as per new CRD
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
provide an option to set PriorityclassName on the
CSI provisioner pods to avoid pod eviction, as the CSI provisioner
pods a critical in system for storage. the default value
for the CSI provisioner pods is set to system-cluster-critical.
provisioner pods are less priority than plugin pods.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Renamed the csi attacher sidecar container name
to follow the nameing convention of other
sidecar containers of the CSI
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Update cephcsi driver to v2.0.1 to allow nodes with
Ceph persistent volumes to shutdown cleanly. This requires
an additional bind mount (/run/mount) in cephfsplugin
and rbdplugin templates.
Signed-off-by: Chu Yong Mark Lee <markleechuyong@gmail.com>
As of Octopus, Ceph will prevent you from creating a pool with a
replica size of 1. Allowing such pool could lead to data loss, so enable
the new option: requireSafeReplicaSize: false if you are **ABSOLUTELY**
certain that is what you want.
Closes: https://github.com/rook/rook/issues/4889
Signed-off-by: Sébastien Han <seb@redhat.com>
This fixes the StorageClass name example and doc to be `rook-cephfs`
instead of just `csi-cephfs`.
Closes#4130.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Currently there is no option to turn on and off the
log level verbosity in csi containers, This PR adds
the functionality to select logging levels for csi containers.
Fixes: #4690
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In ceph-csi v2.0.0 added a support for specifing
the erasure coded pool in storageclass which will be
used to store data.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
external-attacher v2.x is not compatible with v1.x
This PR makes the require changes in csi templates
and upgrade documentation for easily upgrade
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently we are running plugin container as privileged
in the provisioner pod for both cephfs and rbd.
we are also running snapshotter container as privileged
in the provisioner pod for rbd.
as we are executing the ceph and rbd commands in provisioner
pod we dont need to be privileged, This PR removed the
privileged capability from the templates.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently we are making use of hostpath
directory to store the provisioner socket, as this
socket is not needed by anyone else other than containers
inside the provisioner pod using empty directory
to store this socket is the best option.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
On systems with SELinux enabled, non-privileged containers
can't access data of privileged containers.
Since the socket is exposed by privileged containers,
all sidecars must be privileged too.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If using ceph-fuse or rbd-nbd the application pods will need
to be restarted during upgrade. This adds a description of
the steps necessary to ensure the applications remain healthy
during upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSI nodeplugins, specifically when using cephfs FUSE or
rbd-nbd as the mounters, when upgraded, will
cause existing mounts to become stale/not-rechable
(usually connection timeout errors).
This is due to losing the mount processes running within the
CSI nodeplugin pods.
This PR add updated the Daemonset update strategy
based on the ENV variable to take care of above issue
with some manual steps
Moreinfo: https://github.com/ceph/ceph-csi/issues/703
Resolves: https://github.com/rook/rook/issues/4248
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In ceph-csi v1.2.2 we have add a flag `forcecephkernelclient`
to force use Ceph Kernel clients on kernel < 4.17 which
support quotas for Cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
we need to set a ENV `CSI_RBD_ENABLE_SNAPSHOTTER`
variable to "false" in operator template to disable
deployment of snapshotter sidecar in ceph-csi rbd
provisioner.
Fixes: #4401
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
A recent PR #4172 added the ability for rook to create
the CSIDriver object on a kubernetes cluster. The change also
made it so that attachRequired value in the ceph CSI drivers
were set to false.
This leads to the entire attach skipping the AD controller
that deals with RWO attachment enforcement. Thus, if a
CephFS volume is marked RWO, 2 pods on different hosts can
end up mounting the same and consuming it at the same time.
Further for RBD, currently the image watchers are the only ones
that prevent a double mount of an image, other than the AD controller
enforcement of single attach for RWO RBD PVs. This has proven
not to be reliable in the face of MON failures, as the watcher
information is lost, and a second mount is allowed.
Thus, we need to bring back the attach requirement to leverage
kubernetes checks and balances for RWO volume type, and
prevent unwanted multi-attach use of the same.
This PR hence undoes some of the changes introduced by the
PR #4172, to bring the attacher side car back, and also declare
the attachRequired for both RBD and CephFS CSI plugins as true.
NOTE: Initially the attachRequired was set to false as this
improved attach times considerably on kubernetes, as the
entire phase was being avoided.
Signed-off-by: ShyamsundarR <srangana@redhat.com>
This will be helpful to list the CSI drivers
deployed in the kubernetes cluster. if we have
N number of CSI drivers deployed in the cluster,
the user can query kubectl get csidriver and get
the list of registered CSI drivers and it requires
kube 1.14+
This removes the deployment of Attacher sidecar container,
as we are setting the attacher required to false during
the CSIDriver object creation.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This will be helpful to list the CSI drivers deployed
in the kubernetes cluster. if we have N number of
CSI drivers deployed in the cluster, the user can
query kubectl get csidriver and get the list of
registered CSI drivers and it requires kube 1.14+
and also the functionality is disabled by default
More info here: https://github.com/ceph/ceph-csi/pull/698
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This switches csi-snapshotter to IfNotPresent which works better in production and offline environments.
Signed-off-by: awalker60 <awalker125@users.noreply.github.com>
when multiple CSI drivers are running in a cluster,
each should use unique socket for registration.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>