As of Octopus, Ceph will prevent you from creating a pool with a
replica size of 1. Allowing such pool could lead to data loss, so enable
the new option: requireSafeReplicaSize: false if you are **ABSOLUTELY**
certain that is what you want.
Closes: https://github.com/rook/rook/issues/4889
Signed-off-by: Sébastien Han <seb@redhat.com>
This fixes the StorageClass name example and doc to be `rook-cephfs`
instead of just `csi-cephfs`.
Closes#4130.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Currently there is no option to turn on and off the
log level verbosity in csi containers, This PR adds
the functionality to select logging levels for csi containers.
Fixes: #4690
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In ceph-csi v2.0.0 added a support for specifing
the erasure coded pool in storageclass which will be
used to store data.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
external-attacher v2.x is not compatible with v1.x
This PR makes the require changes in csi templates
and upgrade documentation for easily upgrade
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently we are running plugin container as privileged
in the provisioner pod for both cephfs and rbd.
we are also running snapshotter container as privileged
in the provisioner pod for rbd.
as we are executing the ceph and rbd commands in provisioner
pod we dont need to be privileged, This PR removed the
privileged capability from the templates.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently we are making use of hostpath
directory to store the provisioner socket, as this
socket is not needed by anyone else other than containers
inside the provisioner pod using empty directory
to store this socket is the best option.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
On systems with SELinux enabled, non-privileged containers
can't access data of privileged containers.
Since the socket is exposed by privileged containers,
all sidecars must be privileged too.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If using ceph-fuse or rbd-nbd the application pods will need
to be restarted during upgrade. This adds a description of
the steps necessary to ensure the applications remain healthy
during upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSI nodeplugins, specifically when using cephfs FUSE or
rbd-nbd as the mounters, when upgraded, will
cause existing mounts to become stale/not-rechable
(usually connection timeout errors).
This is due to losing the mount processes running within the
CSI nodeplugin pods.
This PR add updated the Daemonset update strategy
based on the ENV variable to take care of above issue
with some manual steps
Moreinfo: https://github.com/ceph/ceph-csi/issues/703
Resolves: https://github.com/rook/rook/issues/4248
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In ceph-csi v1.2.2 we have add a flag `forcecephkernelclient`
to force use Ceph Kernel clients on kernel < 4.17 which
support quotas for Cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
we need to set a ENV `CSI_RBD_ENABLE_SNAPSHOTTER`
variable to "false" in operator template to disable
deployment of snapshotter sidecar in ceph-csi rbd
provisioner.
Fixes: #4401
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
A recent PR #4172 added the ability for rook to create
the CSIDriver object on a kubernetes cluster. The change also
made it so that attachRequired value in the ceph CSI drivers
were set to false.
This leads to the entire attach skipping the AD controller
that deals with RWO attachment enforcement. Thus, if a
CephFS volume is marked RWO, 2 pods on different hosts can
end up mounting the same and consuming it at the same time.
Further for RBD, currently the image watchers are the only ones
that prevent a double mount of an image, other than the AD controller
enforcement of single attach for RWO RBD PVs. This has proven
not to be reliable in the face of MON failures, as the watcher
information is lost, and a second mount is allowed.
Thus, we need to bring back the attach requirement to leverage
kubernetes checks and balances for RWO volume type, and
prevent unwanted multi-attach use of the same.
This PR hence undoes some of the changes introduced by the
PR #4172, to bring the attacher side car back, and also declare
the attachRequired for both RBD and CephFS CSI plugins as true.
NOTE: Initially the attachRequired was set to false as this
improved attach times considerably on kubernetes, as the
entire phase was being avoided.
Signed-off-by: ShyamsundarR <srangana@redhat.com>
This will be helpful to list the CSI drivers
deployed in the kubernetes cluster. if we have
N number of CSI drivers deployed in the cluster,
the user can query kubectl get csidriver and get
the list of registered CSI drivers and it requires
kube 1.14+
This removes the deployment of Attacher sidecar container,
as we are setting the attacher required to false during
the CSIDriver object creation.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This will be helpful to list the CSI drivers deployed
in the kubernetes cluster. if we have N number of
CSI drivers deployed in the cluster, the user can
query kubectl get csidriver and get the list of
registered CSI drivers and it requires kube 1.14+
and also the functionality is disabled by default
More info here: https://github.com/ceph/ceph-csi/pull/698
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This switches csi-snapshotter to IfNotPresent which works better in production and offline environments.
Signed-off-by: awalker60 <awalker125@users.noreply.github.com>
when multiple CSI drivers are running in a cluster,
each should use unique socket for registration.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
provided ENV variables to configure the grpc
and liveness metrics port for both cephfs and
rbd CSI drivers.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
`xfs`. However further testing shows that, ext4 should be
the default or preferred fs for RBD devices.
This patch bring that change
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
currently CSI templates have hardcoded kubelet
path which will be used for mounting PVC,this
will not work if kubelet is configured to use
different path, with this PR kubelet path
can be configured for CSI.
Fixes: #3921
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
set containerized flag as false
This is not required as we are setting
containerized=false and we have deprecated
this flag in ceph-csi.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
liveness container is deployed as part of
the provisioner pod so it has to connect to
the provisioner container and check the liveness.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
At present, the request timeout of sidecars
are at 60s and this is a request to increase
this time out value to 150s or higher. The
higher timeout value can help to reduce the
load of our backend ceph cluster and also can
avoid throttling issues at sidecars to an extend.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
grpc metrics is made option in rook
this can be disabled/enabled with ROOK_CSI_ENABLE_GRPC_METRCS
variable in operator.yaml
this will help admin who want to check the grpc metrics
of the ceph-csi plugins
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
earlier 8080 was used by rbd for liveness and 8081
was used by cephfs for liveness. in most of the system
8080 will be sued for some other service, this will
cause the rbd service to fail, choosing 9080 and 9081
as ports for rbd and cephfs to avoid this kind of
suitation.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
pidlimit flag was added only for cephfs
and rbd provisioner earlier, this PR adds
these flags to the rbd and cephfs plugin
to configure the pidlimit to max
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Deployment behaves better when a node gets disconnected
from the rest of the cluster - new provisioner leader
is elected in ~15 seconds, while it may take up to
5 minutes for StatefulSet to start a new replica.
if kube version is 1.13.x deploy provisioner as statefulset.
if kube version is higher than 1.14+ deploy provisioner
as deployment.
Refer: kubernetes-csi/external-provisioner@52d1fbc
Refer: ceph/ceph-csi#497
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This is required if you are running multiple
csi drivers, if multiple node plugins are
listening on the same socket they request
may go to any node plugin. to avoid this we
need to have a unique socket per csi driver.
It is recommended to have socket inside the
driver name folder.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Add a prefix to the csi drivername when ceph csi is deployed by rook.
This aims to prevent trivial conflict between csi instances when two
rook operators are deployed into the same cluster. By default the
prefix is the "<rook-namespace>." but can by customized on the cli
and via environment vars.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
we have seen better performance in device format and
mounting by setting the fstype to xfs
from default ext4.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The CSI driver requires the ceph key for administrating
the cluster. The key is stored in a secret and specified in the
storage class. The operator will now automatically generate
a secret named rook-ceph-csi that can be specified by
default in the storage class instead of requiring the
admin to create it.
Signed-off-by: travisn <tnielsen@redhat.com>