Commit Graph
31 Commits
Author SHA1 Message Date
Anmol Sachan fdb88edb6a ceph: updated thresholds of OSD capacity alerts and the device_class
This commit updates the OSD capacity thresholds to match the cluster capacity alert thresholds.
Also updated device_class info to provide info about which type of storage needs to be added.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-12-07 20:11:01 +05:30
Alexander Trost be16bc455f docs: operator use rbac.authorization.k8s.io/v1
This replaces any uses of `rbac.authorization.k8s.io/v1beta1` with
`rbac.authorization.k8s.io/v1` affecting RBAC objects like ClusterRole
and ClusterRoleBindings, etc. to make it consistent.
This is done as some RBAC objects used the `v1beta1` and others already
using `v1`.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-08-20 12:08:32 +02:00
Anmol Sachan c35f31bcf2 ceph: removed the storage provisioners usage rules
In the interest of having the metrics avaliable generically for all the storage provisioners and just not ceph, the superseding metrics will be added to Openshift directly. Thus these metrics wont be required in rook.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-08-10 16:48:24 +05:30
Travis Nielsen 99d2131209 ceph: symlink prometheus manifests in same dir
The prometheus manifests should be symlinked in the same directory
instead of from the root of the repo for building across all
platforms

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-08-05 15:55:16 -06:00
Anmol Sachan a42616967d ceph: updated the PV count query for telemeter
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-07-27 20:18:13 +05:30
Anmol Sachan f3eea6ebb7 ceph: added rules for PVC requested and used capacity for telemeter
This commit adds recording rules to monitor aggregated (requested and used) capacity of the PVCs for block and file type PVCs, via the telemeter.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-07-27 20:16:12 +05:30
Sébastien Han 6a643e9522 ceph: add monitoring rules for octopus
Add monitoring rules for octopus clusters.

Closes: https://github.com/rook/rook/issues/5897
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-27 10:16:49 +02:00
Anmol Sachan 6bf3aa9a9a ceph: added prometheus rule file for alerting on external cluster
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-07-17 11:29:51 +02:00
Anmol Sachan ff1a8ad31e ceph: updated PVC utilizaztion alerts
Updated warning and critical alerts for all the PVC's created by rook-ceph provisioners. Added the recommendation to expand the PVCs.

Closes: BZ1849717
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-06-25 13:03:01 +05:30
Anmol Sachan 624ca40817 ceph: added PVC utilization alerts
Added warning and critical alerts for all the PVC's created by rook-ceph provisioners. This is required to notify a admin timely to check on PVC utilization and take a required action to prevent workloads from failing.

Closes: BZ1849723
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-06-23 21:06:59 +05:30
Anmol Sachan 7cebf15a7d ceph: added CephClusterReadOnly alert and updated utilization alerts
Added a new critical alert CephClusterReadOnly which notifies when the cluster becomes read-only
Updated the warning and critical alert to accomodate the new alert

Closes: BZ1809248
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-06-19 14:07:51 +05:30
Satoru Takeuchi a5ea2858c9 manifests: remove unnecessary namespace fields
There are many namespace fields in ClusterRole{,Binding}. However,
ClusterRole{,Binding} are not namespaced. So we can remove these.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-03-18 15:58:31 +00:00
Anmol Sachan caf79f971f ceph: Fixed CephMonHighNumberOfLeaderChanges alert
The alert was showing wrong POD and hostname in the
alert description. Fixed the query for the same by
adding the right labels from ceph_mon_metadata query
to the original query.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2020-01-14 13:32:03 +05:30
Sébastien Han a888ed5d7c Merge pull request #4485 from anmolsachan/osd_alerts
ceph: Added OSD utilization alerts
2019-12-12 18:24:27 +01:00
Anmol Sachan 99e961f72a ceph: Added OSD utilization alerts
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2019-12-12 22:40:26 +05:30
Anmol Sachan 353621452b ceph: Improve storage utilization alert description and message.
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2019-12-12 14:51:27 +05:30
Anmol Sachan 09af832085 ceph: Updated storage utilization thresholds and timings
This PR mainly takes a proactive approach to
alert the user to take action when the cluster
utilizartion is getting close to the maximum capacity.
Since ceph stops I/O at 95% utilization, the user should
be warned before it.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2019-11-10 23:44:53 +05:30
Umanga Chapagain ddb006152b Changed description for CephDataRecoveryTakingTooLong alert
Previous alert description was confusing. This makes it more
generic and user understandable.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-09-13 13:58:51 +05:30
Umanga Chapagain 332c9993b1 Fixed HighMonLeaderChanges Alert
rounded the value of MonLeaderChange Rate to 2 decimal points (e.g. 2.44)
previously, we had 16 decimal points (e.g. 2.4406779661016946).
This makes the alert notification more sensible and readable.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-09-09 10:34:00 +05:30
Madhu Rajanna a8589af337 Implement grpc metrics for cephcsi
grpc metrics is made option in rook
this can be disabled/enabled with ROOK_CSI_ENABLE_GRPC_METRCS
variable in operator.yaml
this will help admin who want to check the grpc metrics
of the ceph-csi plugins

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-30 15:01:45 +05:30
Madhu Rajanna 5587169fe9 Fix namespace selector in csi-livenees-monitoring
csi services are created in rook-ceph namespace

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-30 13:16:55 +05:30
Madhu Rajanna 53e33d4e08 Fix invalid port name in service monitoring
http-metrics1 port was not the service port name
defined for liveness

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-30 13:16:55 +05:30
Umanga Chapagain d92a04e8ed fixes: alerting rules and recording rules
- updated query for latency
- updated severity level for mgr down alert

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-08-27 13:50:51 +05:30
Daniel-Pivonka 665172a9ea ceph: deploy liveness sidecar with csi
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2019-08-19 14:26:47 -04:00
Umanga Chapagain d30c06860d fixes permissions and servicemonitor configurations
this adds the access for rook to create monitoring resources
across namespaces and provides servicemonitor config to
monitor the current service being created

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-07-26 10:53:18 +05:30
Umanga Chapagain 732f3030d0 integrate prometheus based monitoring with rook-ceph
> add new field to cluster spec
> deploy servicemonitor
> deploy prometheusRule

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-07-08 11:40:17 +05:30
Shubhendu 577728bb03 Added latest prometheus alerting rules for ceph
New rules added:
- CephMgrIsAbsent
- CephMgrIsMissingReplicas
- CephMdsMissingReplicas
- CephMonHighNumberOfLeaderChanges
- CephOSDVersionMismatch
- CephMonVersionMismatch

Signed-off-by: Shubhendu <shtripat@redhat.com>
2019-06-06 10:04:45 +05:30
Travis Nielsen 46fadc792b Merge pull request #2996 from umangachapagain/ceph-mixins
Prometheus Rule for Ceph
2019-04-18 09:52:54 -06:00
Umanga Chapagain 361508ca7f Adding Prometheus based alerting for Ceph
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-04-18 19:37:00 +05:30
Kaushal M 2e274b306a ceph: Use aggregated ClusterRoles
The ClusterRoles in the manifests (and helm chart) for the ceph provider
have been updated to use [aggregated ClusterRoles][1].

All ClusterRoles have been split into 2 ClusterRoles.
An aggregated ClusterRole without any rules, named the same, but with an
aggregationRule that matches labels in the format
'rbac.ceph.rook.io/aggregate-to-<ClusterRole name>: "true"'.
A second ClusterRole which contains the existing rules named
'<ClusterRole name>-rules', with label
'rbac.ceph.rook.io/aggregate-to-<ClusterRole name>: "true"'.

These two ClusterRoles give the same behaviour as the previous
ClusterRole.

Additional rules to a ClusterRole can now be provided by creating a new
ClusterRole with the correct label. This should help make updates
simpler.

Fixes #2634.

[1]: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#aggregated-clusterroles

Signed-off-by: Kaushal M <kshlmster@gmail.com>
2019-04-17 13:00:21 +05:30
Alexander Trost 097f640bbb doc: rename monitoring doc for Ceph
doc: move Ceph mon example files to ceph example subfolder

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2018-12-07 19:33:27 +01:00