The .metadata.generation field is updated if and only if the value at the .spec subpath changes.
Additionally, if the spec does not change, .metadata.generation is not updated.
This is a must-have for the controller-runtime work, if we don't have
this, the generation field will always be incremented, resulting in a
endless reconcile loop.
Signed-off-by: Sébastien Han <seb@redhat.com>
Checks the version of the configured ceph-csi image while starting the operator. The operator will fail if the image is not supported.
Added an additional parameter to operator to disable the check e.g. to test not yet supported csi images.
Signed-off-by: Stefan Haas <shaas@suse.com>
As of Octopus, Ceph will prevent you from creating a pool with a
replica size of 1. Allowing such pool could lead to data loss, so enable
the new option: requireSafeReplicaSize: false if you are **ABSOLUTELY**
certain that is what you want.
Closes: https://github.com/rook/rook/issues/4889
Signed-off-by: Sébastien Han <seb@redhat.com>
The ceph/ceph:v14.2.7 image is released so we can pick these fixes
up as the recommended version of ceph.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the OSD on PVC is backed by a metadata block PVC, the Ceph CRUSH
device class should be set to something else rather than the rotational
property of the drive.
Closes: https://github.com/rook/rook/issues/4881
Signed-off-by: Sébastien Han <seb@redhat.com>
Creating an EC pool was succeeding, but then the update to the
status was failing because of an incorrect check for changing EC
parameters. Now we correctly check if EC parameters are changing
unexpectedly.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
"rook-ceph-operator-config" can be used to override Env Var
initialized during operator deployment.
This commit makes use of the ConfigMap to determine NodeAffinity
and Tolerations for ceph-csi drivers.
Closes#3239
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
We now support the addition of the PVC that acts as a metadata device
for a given OSD.
For this, you need to create a new `volumeClaimTemplates`, its name must
be "metadata" otherwise, Rook won't pick it up.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block and block.db directly inside the container instead of
running ceph-volume activate. This is much cleaner.
Closes: https://github.com/rook/rook/issues/3852
Signed-off-by: Sébastien Han <seb@redhat.com>
The replication and erasure code settings are mutually exclusive
so must both be treated as optional by the schema validation.
The minimum must be set to 0 to treat them as optional.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This fixes the StorageClass name example and doc to be `rook-cephfs`
instead of just `csi-cephfs`.
Closes#4130.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Currently there is no option to turn on and off the
log level verbosity in csi containers, This PR adds
the functionality to select logging levels for csi containers.
Fixes: #4690
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Fixed conditions getting resetted after the operator restart.
Did the changes which required to implement conditions on the rook ceph cluster
Conditions will eliminate the current status.State and incorporates a type which
provides much more description to the current status of the cluster.
Signed-off-by: Nizamudeen <nia@redhat.com>
In ceph-csi v2.0.0 added a support for specifing
the erasure coded pool in storageclass which will be
used to store data.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
external-attacher v2.x is not compatible with v1.x
This PR makes the require changes in csi templates
and upgrade documentation for easily upgrade
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
CSI image tag ENV required to select the image tag
is missing in operator-openshift.yaml, This commit
adds the missing image tag ENV to operator-openshift.yaml
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Versions of k8s prior to 1.14 do not support the nullable field in
CRD properties. As k8s 1.13 remains part of our support matrix and
as this field must remain nullable, we must remove CRD validation
of SecurePort. Range validation of this field has been moved into
the logic of rgw.validateStore.
Fixes: #4693
Signed-off-by: Elise Gafford <egafford@redhat.com>
Multiple things:
1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.
2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.
3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.
Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
With this modification is guaranteed that at least 1 MDS pod is going to be
placed in each of the available zones in the k8s cluster.
This would be improved in the future using:
<Pod Topology Spread Constraints> (still in alpha state since kubernetes V.1.16)
This modification obtain the number of zones in the k8s cluster and appends an
<antiaffinity> term using the topologyKey
<topology.kubernetes.io/zone> in the same number of MDS pods.
In the case that will be more MDS pods than zones, the <antiaffinity> term won't
be added in these extra pods.
Important Note:
The antiaffiniy term only will work with nodes labeled using the NEW label:
<topology.kubernetes.io/zone> present in k8s V.1.17 clusters.
For previous versions of k8s clusters the label used is:
<failure-domain.beta.kubernetes.io/zone>
And therefore, in this kind of clusters this modification won't work.
Resolves#4641
[test ceph]
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
A new CR setting has been introduced to disable the crash controller.
To disable it, add `disable: true` on the crashCollector section.
Signed-off-by: Sébastien Han <seb@redhat.com>
The alert was showing wrong POD and hostname in the
alert description. Fixed the query for the same by
adding the right labels from ceph_mon_metadata query
to the original query.
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
When configuring a cephcluster CRD on a different namespace than rook-ceph, like for example when deploying a external cluster, we need to set ROOK_CURRENT_NAMESPACE_ONLY to true so the operator can watch for CRD modifications on other namespaces.
Closes: https://github.com/rook/rook/issues/4667
Signed-off-by: Daniel Parkes <dparkes@redhat.com>
When creating a CephObjectStore object, operator will
use .Gateway.Port or .Gateway.SecurePort to create k8s
Service object, if the port is invalid, the Service
will not be created.
Note .Gateway.Port/.Gateway.SecurePort should in 1~65535,
however .Dashboard.Port is 0~65535 because 0 will use
default setting for .Dashboard.Port and there is no
such mechanism for .Gateway.Port/.Gateway.SecurePort
Closes: https://github.com/rook/rook/issues/4574
Signed-off-by: Yu Chih <secret104278@gmail.com>
currently we are running plugin container as privileged
in the provisioner pod for both cephfs and rbd.
we are also running snapshotter container as privileged
in the provisioner pod for rbd.
as we are executing the ceph and rbd commands in provisioner
pod we dont need to be privileged, This PR removed the
privileged capability from the templates.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently we are making use of hostpath
directory to store the provisioner socket, as this
socket is not needed by anyone else other than containers
inside the provisioner pod using empty directory
to store this socket is the best option.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
On systems with SELinux enabled, non-privileged containers
can't access data of privileged containers.
Since the socket is exposed by privileged containers,
all sidecars must be privileged too.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The minio operator has not had community support nor
any updates since being added to Rook. Support is being
removed from Rook due to this lack of community interest.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
A new CRD option `continueUpgradeAfterChecksEvenIfNotHealthy` is added.
When upgrading, Rook goes OSD by OSD and then waits for PGs to be clean
before proceeding to the next OSD. Currently, Rook waits for 5 hours but
there might be circumstances where PGs need more time to settle.
Thus setting `continueUpgradeAfterChecksEvenIfNotHealthy` to true will
pursue the upgrade process, even if PGs are not 100% active+clean.
Closes: https://bugzilla.redhat.com/show_bug.cgi?id=1786029
Signed-off-by: Sébastien Han <seb@redhat.com>
* bumping ceph-operator memory limit from 128Mi to 256Mi in order to
avoid OOMkills of the rook-ceph-operator pod
* updating documentation example for operator to denote a limit of 256Mi
instead of 128Mi
When upgrading from rook v1.1.7 to v1.2.0, the rook-ceph-operator pod
continued to get OOMKilled by the kubernetes scheduler. Manually
bumping this limit to something higher than 128Mi (I tried 256Mi and it
was good) prevented the OOMKills.
Signed-off-by: Jeff Billimek <jeff@billimek.com>
If using ceph-fuse or rbd-nbd the application pods will need
to be restarted during upgrade. This adds a description of
the steps necessary to ensure the applications remain healthy
during upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSI nodeplugins, specifically when using cephfs FUSE or
rbd-nbd as the mounters, when upgraded, will
cause existing mounts to become stale/not-rechable
(usually connection timeout errors).
This is due to losing the mount processes running within the
CSI nodeplugin pods.
This PR add updated the Daemonset update strategy
based on the ENV variable to take care of above issue
with some manual steps
Moreinfo: https://github.com/ceph/ceph-csi/issues/703
Resolves: https://github.com/rook/rook/issues/4248
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Reduce from 5 minutes to less than 50 seconds the time needed to restart a
mgr and osds(PVCs based),rgw,mds,rbd,nfs and toolbox pod that was running
in a k8s "NotReady" Node.
A new Rook Operator setting has been added to allow the user to set the
time used in pod's toleration <node.kubernetes.io/unreachable>:
ROOK_UNREACHABLE_NODE_TOLERATION_SECONDS = <5>
New setting added in release notes
Addressed @travisn,@leseb, and @blaineEXE suggestions
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Sébastien Han
Stefan Haas
Travis Nielsen
Umanga Chapagain
desprert
Maksim Nabokikh
Rohan CJ
Alexander Trost
Madhu Rajanna
Nizamudeen
Elise Gafford
Juan Miguel Olmo Martínez
Anmol Sachan
Daniel Parkes
Yu Chih
Jeff Billimek