After further consideration and working with the feature for a few
weeks, the `configOverride` functionality seems to be more complexity
than is worth it for the expected benefit. Users still have the most
preferred option of setting configs in Ceph's config database using
`ceph cofig set` or the dashboard, and the override ConfigMap still
exists for fallback scenarios. The config database can still be used
by Rook for setting necessary configs, but the user has no way of
telling Rook to do so.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Reduce the duplication of detecting the version of the external cluster
by refactoring to a single method that retrieves and validates the
version
Signed-off-by: travisn <tnielsen@redhat.com>
Add various upgrade tests so that the local cluster does not fail if
an upgrade is available on the external cluster.
Also, add better upgrades handling on update and on add. We now rely as
much as possible on a unique function that works both local and external
cluster 'detectAndValidateCephVersion()'. So the code will act the same
way wether a CR update is triggered or if the operator is restarted.
The health check now reports any topology changes on the external
cluster such as odd number of mons and low number. It will also detect
if the external cluster has been upgraded and will log that information,
helping the admin in his/her decision to trigger an image upgrade.
Finally, we prevent the creation of new CRs if the external cluster
version has a higher major number than the local one.
Closes: https://github.com/rook/rook/issues/3731
Signed-off-by: Sébastien Han <seb@redhat.com>
The log messages for orchestrations being skipped due to nodes still
being schedulable or unschedulable is filling the logs and not necessary.
The log message for reconciling is now debug, and also fixed another
formatting log message.
Signed-off-by: travisn <tnielsen@redhat.com>
grpc metrics is made option in rook
this can be disabled/enabled with ROOK_CSI_ENABLE_GRPC_METRCS
variable in operator.yaml
this will help admin who want to check the grpc metrics
of the ceph-csi plugins
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The init container for copying binaries was being overwritten by the
init container for initializing pvcs. This change appends the init
containers so all expected init containers are preserved.
Signed-off-by: travisn <tnielsen@redhat.com>
Maintenance is only defined when managedDisruptionBudgets are enabled.
Details can be found in the design doc for manged poddisriptionbudgets.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
Add deviceClass as a per OSD config option under
storage/nodes/devices/config for setting a custom crush device class
per OSD.
Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
Adding support for the db-devices flag for ceph versions 14.2.1 and
newer. This flag will allow for explictly setting the db device for
bluestore configurations to the device specified as metadataDevice.
Fixing the databaseSizeMB config parameter to ensure it is used to set
the size of the db-device partition when specified at either the global
or individual device level. (#3652)
Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
Allow users to specify overrides in the CephCluster CRD. The override
ConfigMap still exists for emergency situations and is mounted into
daemon pods directly instead of being merged into a Rook-created config
file.
Rook Ceph no longer uses a config file for managing daemons with the
exception of the OSDs which still generate a config in an init
container.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
OSD prepare job pods don't require node selector when the OSDs are based
on PVC.
The bug was intruduced in https://github.com/rook/rook/pull/3707 which
added node selector to the prepare job pod when portable property of
storageClassDeviceSets is false.
Adding node selector set kubernetes.io/hostname=example-storagedeviceset-xxx, while there is no host example-storagedeviceset-xxx the prepare job failed.
Signed-off-by: rohan47 <rohgupta@redhat.com>
From now on, Rook will only perform check before upgrades when there is
an actual upgrade. So if the Ceph image changed and a new version is
desired Rook will go through all the daemons and update them one by one
and perform checks in between.
Closes: https://github.com/rook/rook/issues/3583
Signed-off-by: Sébastien Han <seb@redhat.com>
this patch adds two new related features. first, it removes the
requirement that all monitor deployments be pinned to nodes using node
selector. this is allowed when not using host networking and when PVC
storage is used. the second feature introduced in this patch is the
elimination of the Rook scheduler for placing monitors that continue to
use the node selector by using canary pods to that are scheduled by k8s,
after which Rook uses the scheduling results to pin the monitors to
nodes.
fixes: #3667
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
creating an objectstore user requires that the objectstore is
created first. This commits adds changes to wait for the object
store to be created before the objectstore user can be created.
It'll retry every 15sec for 5mins.
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
The last image has disabled ephemeral repositories so it's now possible
for images older than 15 days to install packages without having an
error from non-existing repositories.
Closes: https://github.com/rook/rook/issues/3662
Signed-off-by: Sébastien Han <seb@redhat.com>
The decision to start the device discovery daemonset is made
by an env var on the operator pod since in some scenarios
the discovery needs to be started immediately with the operator
instead of being delayed to start with the cluster.
Signed-off-by: travisn <tnielsen@redhat.com>
Deployment behaves better when a node gets disconnected
from the rest of the cluster - new provisioner leader
is elected in ~15 seconds, while it may take up to
5 minutes for StatefulSet to start a new replica.
if kube version is 1.13.x deploy provisioner as statefulset.
if kube version is higher than 1.14+ deploy provisioner
as deployment.
Refer: kubernetes-csi/external-provisioner@52d1fbc
Refer: ceph/ceph-csi#497
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
ceph: Use the new NetworkSpec api
ceph: Change all tests to use the NetworkSpec
ceph: Remove TypeMeta field from NetworkSpec
ceph: Change IsHost network func signature
Signed-off-by: giovanism <giovanism@outlook.co.id>
Blaine Gardner
Sébastien Han
travisn
Madhu Rajanna
Rohan CJ
Michael Vollman
Dmitry Yusupov
rohan47
Anton Skriptsov
rohan47
Noah Watkins
Sameer Kulkarni
Umanga Chapagain
giovanism