The jobs to detect the ceph and csi versions now can have
custom annotationso and labels added to them. They are
very short-lived jobs, but they may need custom annotations
in some environments to run.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Normally the device class of an OSD is determined at provisioning
and is not updated thereafter. In some scenarios the admin may
want to force update the device class to a new value. The device
classes can be updated by first setting allowDeviceClassUpdate
in the storage spec of the cephcluster, then updating the
device class specified on the deviceSets or other OSDs.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For check if cluster PGs are healthy before osd updates
Which helps gainning more confidence in the osd upgrades
Signed-off-by: mmaoyu <wupiaoyu61@gmail.com>
previously, we had kept the Rook orchestrator manager module disabled
so far because it was causing a bunch of issues and errors on the
dashboard. But with the recent changes made in the cephv v18.2.1
release, we've fixed those issues and made some overall improvements
to the dashboard user experience when Rook is enabled. With all that
in mind, it's time to switch on the Rook orchestrator by default.
Fixes: https://github.com/rook/rook/issues/13760
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
With the release of Reef v18.2.2 we update the default
recommended version of Ceph to fix the prometheus issues
seen with v18.2.1
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When CPU requests and limits are assigned to a pod,
the pod will be guaranteed the requests, up to the limits.
Even if there are spare CPU cycles, the pod cannot use
them. Thus, pods can be unnecessarily denied compute
when they need to burst if the limits are set.
Therefore, it is not recommended to set CPU limits
since the CPU requests are already guaranteeing that
no pod will be starved at least for its requests.
Signed-off-by: travisn <tnielsen@redhat.com>
The pg_autoscaler mgr module is no longer possible to disable.
Ceph requires the module to always be enabled. Now the
autoscaling configuration is done with a per-pool setting.
Now the rook module is shown as an example to enable.
Signed-off-by: travisn <tnielsen@redhat.com>
Since v18.2.1 is the default recommendation for the ceph version,
update all the docs and examples to that version.
Signed-off-by: travisn <tnielsen@redhat.com>
Update Rook operator base image and example manifests to the latest Ceph
version released late last week v18.2.1.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Update sample manifests to use Ceph v18 or v18.2.0 as appropriate now
that Reef is found to be stable.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
This commit adds new CSIDriverOptions section in
cephCluster CR. This section contains settings
for read affinity and kernel+fuse Mount options
These settings will be injected directly into
rook-ceph-csi-config cm to be applicable per
ceph cluster.
Signed-off-by: Rakshith R <rar@redhat.com>
Using the new configuration parameters prometheusEndpoint and
prometheusEndpointSSLVerify users can configure dashboard to
point to their prometheus instance.
closes: #12876
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
When OSDs flap, ceph stops the OSD daemon if its marked down greater than
5 times in 600 seconds. But OSD pod restarts and marks the OSD `up` again.
This causes the PGs mapped to these OSDs to peer. While the PGs are peering,
IO to these PGs are blocked.
So we need to ensure that if ceph is marking OSD `down` due to flapping, OSD pod
should not restart to mark the OSDs `up` again.
This PR adds a sleep to the OSD pod if the container returned with a 0 exit code
Default behavior is to sleep for 6 hrs. But user can configure it from the
ceph cluster spec.
Signed-off-by: sp98 <sapillai@redhat.com>
This adds the new `exporter:` resource key to the Ceph cluster example
yaml. It also set some sane resource requests and limits for it in the
CephCluster section of the rook-ceph Helm chart.
Closes#11914
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The prometheus mgr module and ceph exporter can now be optionally
disabled by the monitoring.metricsDisabled setting in the
CephCluster CR. These will not be disabled by default, rather
than the mgr module being disabled by default from v1.11.4.
Signed-off-by: travisn <tnielsen@redhat.com>
The journal size was only applicable to the filestore OSD
format which has not been supported by rook since v1.2.
Remove the remaining obsolete setting from the examples and
code.
Signed-off-by: travisn <tnielsen@redhat.com>
With the release of ceph v17.2.6, the examples and the base
image for the operator are updated to pick up the latest
and greatest.
Signed-off-by: travisn <tnielsen@redhat.com>
With the new mgr HA implementation (based on readiness probe) we
don't need the mgr sidecar (live-watch) anymore. This commit is
intended to remove all the related code.
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Enabling msgr v2 and disabling msgr v1 currently requires enabling
either encryption on the wire or compression on the wire.
As more clients are running on the latest kernel, allow
the clients to run on v2 even when encryption and compression
are not enabled. Clusters that are fully running on v2
will more easily be able to change configuration between
enabling or disabling msgr v2 features.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The machine disruption budgets for handling openshift
machines and machinesets are now removed since they
have been unused and unmaintained since implemented.
This feature is expected to be handled with the more
common Pod Disruption Budgets. A workaround is for the
cluster admin to set up their machine sets so they
match the zone topology. See the original design
doc from the feature here:
https://github.com/rook/rook/blob/master/design/ceph/ceph-openshift-fencing-mitigation.md
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
enabling logCollector by default will enable the coredump
generation in case a process terminates with a segmentation fault.
Signed-off-by: gauravsitlani <gauravsitlani@riseup.net>
For the Rook v1.10 release we want the latest version of Quincy to
be the default deployment. Updating to v17.2.3 with the latest
patch release.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commits add new field `MaxLogSize` inside `LogCollectorSpec` of
cephCluster cr which will take max size of log after which we want to
rotate the log.
Signed-off-by: subhamkrai <srai@redhat.com>
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add prepareosd entry to cluster.yaml docs.
Add prepareosd as an example of placement (commented out)
Signed-off-by: Kim Minjong <make.dirty.code@gmail.com>
By default, we should set the priority class to one of the built-in
priority class names to ensure that pods critical to the storage
will be able to remain running when resources are low. Otherwise,
critical rook pods could be evicted and affect many other pods
that rely on the storage to continue functioning. The options have
been available in the CRs, but until now we have just not set the
defaults in the examples. Critical rook components are now set to
the priority class system-node-critical if they are generally pinned
to a node, and system-cluster-critical if they are critical to the storage.
Some pods such as the operator and crash collector do not have a
default priority class set in the examples since they don't affect
the data path.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The prometheus rules had been previously created if the cephcluster CR
setting monitoring.enabled was set to true. The rules were not customizable
and therefore not flexible enough. Now the rules are installed by the helm
chart. To customize the rules, a post-processor can be applied to the helm
chart.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
msgr2 allows for encryption and/or compression across the wire.
Settings for enabling the encryption and compression are now
available in the cluster CR to that ceph will be automatically
configured with these settings when desired.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
To ensure the mgr is not the single point of failure, the mgr
daemon count is now set to 2 by default in the cluster examples.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The latest ceph image should be used for the base of
the rook image to pick up the latest security releases.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Artem Torubarov
Blaine Gardner
parth-gr
Travis Nielsen
mmaoyu
Redouane Kachach
Rakshith R
sp98
Micah Nagel
Alexander Trost
gauravsitlani
subhamkrai
Josh Soref
parth-gr
Kim Minjong
Blaine Gardner