Versions of k8s prior to 1.14 do not support the nullable field in
CRD properties. As k8s 1.13 remains part of our support matrix and
as this field must remain nullable, we must remove CRD validation
of SecurePort. Range validation of this field has been moved into
the logic of rgw.validateStore.
Fixes: #4693
Signed-off-by: Elise Gafford <egafford@redhat.com>
Multiple things:
1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.
2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.
3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.
Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
With this modification is guaranteed that at least 1 MDS pod is going to be
placed in each of the available zones in the k8s cluster.
This would be improved in the future using:
<Pod Topology Spread Constraints> (still in alpha state since kubernetes V.1.16)
This modification obtain the number of zones in the k8s cluster and appends an
<antiaffinity> term using the topologyKey
<topology.kubernetes.io/zone> in the same number of MDS pods.
In the case that will be more MDS pods than zones, the <antiaffinity> term won't
be added in these extra pods.
Important Note:
The antiaffiniy term only will work with nodes labeled using the NEW label:
<topology.kubernetes.io/zone> present in k8s V.1.17 clusters.
For previous versions of k8s clusters the label used is:
<failure-domain.beta.kubernetes.io/zone>
And therefore, in this kind of clusters this modification won't work.
Resolves#4641
[test ceph]
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
A new CR setting has been introduced to disable the crash controller.
To disable it, add `disable: true` on the crashCollector section.
Signed-off-by: Sébastien Han <seb@redhat.com>
The alert was showing wrong POD and hostname in the
alert description. Fixed the query for the same by
adding the right labels from ceph_mon_metadata query
to the original query.
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
When configuring a cephcluster CRD on a different namespace than rook-ceph, like for example when deploying a external cluster, we need to set ROOK_CURRENT_NAMESPACE_ONLY to true so the operator can watch for CRD modifications on other namespaces.
Closes: https://github.com/rook/rook/issues/4667
Signed-off-by: Daniel Parkes <dparkes@redhat.com>
When creating a CephObjectStore object, operator will
use .Gateway.Port or .Gateway.SecurePort to create k8s
Service object, if the port is invalid, the Service
will not be created.
Note .Gateway.Port/.Gateway.SecurePort should in 1~65535,
however .Dashboard.Port is 0~65535 because 0 will use
default setting for .Dashboard.Port and there is no
such mechanism for .Gateway.Port/.Gateway.SecurePort
Closes: https://github.com/rook/rook/issues/4574
Signed-off-by: Yu Chih <secret104278@gmail.com>
currently we are running plugin container as privileged
in the provisioner pod for both cephfs and rbd.
we are also running snapshotter container as privileged
in the provisioner pod for rbd.
as we are executing the ceph and rbd commands in provisioner
pod we dont need to be privileged, This PR removed the
privileged capability from the templates.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently we are making use of hostpath
directory to store the provisioner socket, as this
socket is not needed by anyone else other than containers
inside the provisioner pod using empty directory
to store this socket is the best option.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
On systems with SELinux enabled, non-privileged containers
can't access data of privileged containers.
Since the socket is exposed by privileged containers,
all sidecars must be privileged too.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The minio operator has not had community support nor
any updates since being added to Rook. Support is being
removed from Rook due to this lack of community interest.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
A new CRD option `continueUpgradeAfterChecksEvenIfNotHealthy` is added.
When upgrading, Rook goes OSD by OSD and then waits for PGs to be clean
before proceeding to the next OSD. Currently, Rook waits for 5 hours but
there might be circumstances where PGs need more time to settle.
Thus setting `continueUpgradeAfterChecksEvenIfNotHealthy` to true will
pursue the upgrade process, even if PGs are not 100% active+clean.
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1786029
Signed-off-by: Sébastien Han <seb@redhat.com>
If using ceph-fuse or rbd-nbd the application pods will need
to be restarted during upgrade. This adds a description of
the steps necessary to ensure the applications remain healthy
during upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSI nodeplugins, specifically when using cephfs FUSE or
rbd-nbd as the mounters, when upgraded, will
cause existing mounts to become stale/not-rechable
(usually connection timeout errors).
This is due to losing the mount processes running within the
CSI nodeplugin pods.
This PR add updated the Daemonset update strategy
based on the ENV variable to take care of above issue
with some manual steps
Moreinfo: https://github.com/ceph/ceph-csi/issues/703
Resolves: https://github.com/rook/rook/issues/4248
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Reduce from 5 minutes to less than 50 seconds the time needed to restart a
mgr and osds(PVCs based),rgw,mds,rbd,nfs and toolbox pod that was running
in a k8s "NotReady" Node.
A new Rook Operator setting has been added to allow the user to set the
time used in pod's toleration <node.kubernetes.io/unreachable>:
ROOK_UNREACHABLE_NODE_TOLERATION_SECONDS = <5>
New setting added in release notes
Addressed @travisn,@leseb, and @blaineEXE suggestions
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
There are two issues at odds with the cephfs driver. The
first is that older kernels less than 4.17 do not support
quotas so PVC sizes will not be enforced with the kernel
driver in that case. If the FUSE client is used instead, there
are issues during upgrade because the client mount will be
disrupted and the client pods will have to restart.
To prevent client pods from breaking during upgrade, we enable
the kernel driver by default. If the quotas are critical,
the admin will need to take steps during upgrade to prevent
breaking the applications.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In ceph-csi v1.2.2 we have add a flag `forcecephkernelclient`
to force use Ceph Kernel clients on kernel < 4.17 which
support quotas for Cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Removed some obsolete instructions from the 1.1 upgrade
and made a few minor fixes to make the upgrade to 1.2 smoother.
With the addition of the CephClient CRD, the CRD needs to be
created before the upgrade or else the operator log will be
filled with errors until it is created. The CSI driver also
had an error when registering after the upgrade due to an RBAC
change.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
gp2 devices present themselves as SSDs,
but have much more variable performance.
Ceph attempts to adjust some settings based on the drive type,
HDD/SSD
In this case client I/O can be impacted too much by assuming gp2 devices are like other SSDs
(an example is https://bugzilla.redhat.com/show_bug.cgi?id=1765923#c6)
In order to prevent this, rook can set the following for OSDs
if the new CR setting `tuneSlowDeviceClass` is set to `true`:
```
osd_recovery_sleep = 0.1
osd_snap_trim_sleep = 2
osd_delete_sleep = 2
```
Closes: https://github.com/rook/rook/issues/4298 and https://bugzilla.redhat.com/show_bug.cgi?id=1771047
Signed-off-by: Sébastien Han <seb@redhat.com>
Added default regex values.
Created an environment variable and using that variable the user can give custom regex values
Updated the PendingReleaseNotes.md
Signed-off-by: Nizamudeen <nia@redhat.com>
ceph client CRD refactoring
Add info about client crd to documentation
Add tests for client controller CRD
Signed-off-by: Mateusz Los <los.mateusz@gmail.com>
The deviceFilter property needs to have a value rather
than be completely empty as it was in the settings.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
There is use case when need to run rook operator on host network since
ceph mon is not accessible from kubernetes pod network.
Signed-off-by: ruslanloman <ruslanloman@gmail.com>
we need to set a ENV `CSI_RBD_ENABLE_SNAPSHOTTER`
variable to "false" in operator template to disable
deployment of snapshotter sidecar in ceph-csi rbd
provisioner.
Fixes: #4401
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
cephcsi v1.2.2 release is out and the image is available
for use, updating rook to use latest available release
image
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>