Commit Graph
1041 Commits
Author SHA1 Message Date
Sébastien Han 128dc4cb79 Merge pull request #7325 from subhamkrai/cert-manager
ceph: use cert-manager for admission webhooks
2021-03-09 10:38:27 +01:00
subhamkrai d594a6c904 ceph: remove config-admission-controller.sh
this commit removes file `config-admission-controller.sh`
as we don't need this anymore and update the doc to point
`tests/scripts/deploy_admission_controller.sh` to start
admission controller.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-05 16:39:52 +05:30
Anmol Sachan 1532070a3e ceph: add osd flapping alert
This PR adds an alert that notifies the users if an OSD restarts
for 5 times under 5 minutes. Each restart corresponds to 2 changes
in the value of ceph_osd_up metric.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2021-03-05 16:22:16 +05:30
Travis Nielsen ec5691ee3a Merge pull request #7330 from thotz/updatecrdsforhealthcheck
ceph: update CRDs for healthcheck in crds.yaml
2021-03-03 08:20:59 -07:00
Sébastien Han e335851b96 Merge pull request #7273 from anmolsachan/BZ1904302
ceph: fixed prometheus query to avoid many-to-many match error
2021-03-03 11:30:13 +01:00
Santosh Pillai 1159455345 ceph: update CRDs for healthcheck in crds.yaml
Even though CRs for bucket health check was added, CRDs were missing from
crds.yaml file. So the health check always enabled for RGW.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-03-03 10:42:35 +05:30
Sébastien Han 8dfa99e01d ceph: rework multiple filesystem support
The upcoming Rook release 1.6 will support Ceph Pacific which introduces
stable support for multiple Ceph Filesystems in the same cluster. So we
now allow the creation of multiple Filesystem if the release is Pacific.

Also, we remove the Operator config flag which was not necessary.

Here I have 2 filesystems:

```
[root@rook-ceph-tools-6b4889fdfd-vncck /]# ceph fs ls
name: myfs, metadata pool: myfs-metadata, data pools: [myfs-data0 ]
name: myfs2, metadata pool: myfs2-metadata, data pools: [myfs2-data0 ]
```

```
  cluster:
    id:     cc747950-589a-4862-bad2-ff7de1217b1a
    health: HEALTH_WARN
            mons a,b,c are low on available space
            6 pool(s) have no replicas configured

  services:
    mon: 3 daemons, quorum a,b,c (age 2d)
    mgr: a(active, since 2d)
    mds: myfs:1 myfs2:1 {myfs2:0=myfs2-b=up:active,myfs:0=myfs-b=up:active} 2 up:standby-replay
    osd: 3 osds: 3 up (since 2m), 3 in (since 6d)

  data:
    pools:   6 pools, 717 pgs
    objects: 44 objects, 4.2 KiB
    usage:   46 MiB used, 90 GiB / 90 GiB avail
    pgs:     717 active+clean

  io:
    client:   1.8 KiB/s rd, 4 op/s rd, 0 op/s wr

  progress:
    Global Recovery Event (45s)
      [===========================.]
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-02 18:27:04 +01:00
Sébastien Han a134d1df6e Merge pull request #7336 from leseb/revert-vault-backend
ceph: revert kms VAULT_KV_VERS to VAULT_BACKEND
2021-03-02 10:44:43 +01:00
Sébastien Han e4c3a4cef7 ceph: revert kms VAULT_KV_VERS to VAULT_BACKEND
The vault env variable VAULT_BACKEND is read by the secret library so we
must not change it. This is reverting
ee6a03dfb8 with a few extra fixes for the
rgw code.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-02 10:08:39 +01:00
Anmol Sachan 136231a71f ceph: fixed prometheus query to avoid many-to-many match error
Ceph_daemon label includes references to a replaced OSD that cause a prometheus
ruleset to fail due many-to-many match error. The changes to this PR filters the
extra matching ceph_daemon time-series.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2021-03-01 17:26:05 +05:30
Blaine Gardner efa7d66d45 ceph: remove driveGroup support
Remove features and design that supports adding OSDs to Ceph clusters
via `spec:driveGroups`. Update the Ceph upgrade doc that informs users
who currently use Drive Groups (we believe there are none of these
users) how to migrate to using the `spec:storage` config.

Resolves https://github.com/rook/rook/issues/7275

Revert "ceph: fix drive group deployment failure"
This reverts commit 76f1d9944e.

Revert "ceph: osd: add drive groups spec to cluster CR"
This reverts commit 7117fc12b7.

Revert "design: ceph orchestrator module add/remove OSDs"
This reverts commit 178187d035.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-02-25 17:08:47 -07:00
Travis Nielsen ade67eb7ce Merge pull request #7302 from BlaineEXE/update-to-ceph-15.2.9
ceph: update build and examples to use v15.2.9
2021-02-25 13:56:27 -07:00
Blaine Gardner 818342cbfb ceph: update build and examples to use v15.2.9
There are still references to v15.2.7 in integration tests to allow
testing OSDs on partitions.

The newest dated Ceph image is ceph/ceph:v15.2.9-20210224

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-02-24 15:15:26 -07:00
Arun Kumar Mohan bc4dd24306 ceph: making changes to fix CI test failures
Changed the random port creation's max value from 255 to 254, as the IP
with value 255 fails.
Re-raise/propagate the 'ExecutionFailureException' exception so that we
don't end-up in ZERO exit value.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-02-24 23:54:42 +05:30
Arun Kumar Mohan 5ce07b9559 ceph: fixed tests for standby mgr changes
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-02-24 23:54:42 +05:30
Arun Kumar Mohan f68529f31f ceph: enable external resource script to collect active and standby mgrs
Now 'create-external-cluster-resources.py' script will add standby mgrs
along with the active mgr while providing the output.
'MonitoringEndpoint' data-key will contain a comma separated list of
active followed by standby mgrs' IP addresses.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-02-24 23:54:42 +05:30
Travis Nielsen 7404de7b38 Merge pull request #7264 from fritchie/pool-quotas
ceph: add ability to set pool quotas
2021-02-19 12:53:41 -07:00
Frank Ritchie a0488dac73 ceph: add ability to set pool quotas
This will allow setting ceph pool quotas in bytes and/or objects.

Ceph documentation: https://docs.ceph.com/en/latest/rados/operations/pools/#set-pool-quotas

Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
2021-02-19 13:49:25 -05:00
Sébastien Han 00e85bc53c Merge pull request #7271 from hookak/patch-2
docs: Delete 'rootPath' in cephfs storageclass yaml
2021-02-19 11:46:31 +01:00
jinhong.kim f519323524 docs: delete 'rootPath' in cephfs storageclass yaml
The pre-provisioned volume or static-PV support in cephfs-csi has changed.
Delete 'rootPath' field which is no longer used

Signed-off-by: Jinhong Kim <hookak25@gmail.com>
2021-02-19 07:13:34 +00:00
Taeuk Kim c0c4d41b47 ceph: correct fs yaml fields
Some MDS fields such as placement, resource belong to mirroring currently.
It corrects the fields location.

Signed-off-by: Taeuk Kim <taeuk_kim@tmax.co.kr>
2021-02-19 01:28:03 +00:00
Sébastien Han 4d4a5d2017 Merge pull request #7239 from leseb/bz-1894412
ceph: revert fail if mgr prometheus is not default
2021-02-17 10:29:50 +01:00
subhamkrai 93309bc492 ceph: helm update variable enableCephFSSnapshotter
this commit removes the casing error in helm.
`enableCephFSSnapshotter` in `values.yaml` does not
match the casing `enableCephfsSnapshotter` in `deployment.yaml`.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-02-17 08:30:27 +05:30
Sébastien Han e99c122038 ceph: revert fail if mgr prometheus is not default
This reverts commit 1ed307cb09. In
8aaff235bb we have introduced the ability
to set a specific monitoring port so the block that fails if the port is
not the default 9283 is not needed anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-16 16:55:37 +01:00
Tom Hellier 2e505d7439 ceph: minor bugfix with multiple filesystems
The multiple filesystems parameter needed to be quoted for kubernetes to accept the parameter, without quotes the environment variable is rejected.

Signed-off-by: Tom Hellier <me@tomhellier.com>
2021-02-15 10:42:54 +00:00
Sébastien Han fd0716a026 Merge pull request #7173 from leseb/fix-7168
ceph: add cephfilesystemmirrors CRD to 1.15
2021-02-09 16:31:25 +01:00
Sébastien Han df683a0c57 ceph: add cephfilesystemmirrors CRD to 1.15
Add the cephfilesystemmirrors definition for CI runs where the
Kubernetes version is 1.15 and extension v1 extensions are not
supported.

Closes: https://github.com/rook/rook/issues/7168
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-09 09:50:16 +01:00
Travis Nielsen 043f947321 build: remove yugabytedb operator
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-06 00:03:41 -07:00
ᗪєνιη ᗷυнʟ 64a0fbc91c ceph: set helm resource policy keep annotation on CRDs in Helm chart
When uninstalling the helm chart do not remove the CRDs, instead
document the way to remove the CRDs after the user uninstalls
the chart.

Closes: https://github.com/rook/rook/issues/6617
Signed-off-by: Devin Buhl <devin.kray@gmail.com>
2021-02-04 12:26:46 -05:00
Madhu Rajanna 81248f1dce ceph: add configmap get RBAC for rbd provisioner
rbd provisioner need to access the configmap
in different namespaces.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-03 21:28:00 +05:30
Madhu Rajanna 3cf9f7147e ceph: pass extra-create-metadata to csi-provisioner
when extra-create-metadata is passed to csi-provisioner
it passes few information like namespace name,pvc
name to the cephcsi driver which is required for
rbd encryption.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-03 21:27:37 +05:30
Travis Nielsen 81b398f6b6 ceph: add deviceClass to the object store schema
The deviceClass was missing from the object store CRD schema,
disallowing the device class from being specified on
the object store pools.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-02 12:13:23 -07:00
Varsha Rao 59084fbe06 ceph: update nfs ganesha pool recommendation
Add following don'ts note for storing ganesha recovery object:
* Don't use fs pool, it can cause this dashboard issue https://tracker.ceph.com/issues/46176
* Don't use erasure coded pool, it will not work due to ganesha's omap requirements

Signed-off-by: Varsha Rao <varao@redhat.com>
2021-02-02 17:04:18 +05:30
Blaine Gardner deec85cd1d Merge pull request #7092 from subhamkrai/route
ceph: create a route for ceph-rgw
2021-01-29 13:14:10 -07:00
subhamkrai beff403c89 ceph: create a route for ceph-rgw
this commit ceph: create a route as
a part of example in `object-openshift.yaml`.
Route is a feature of OpenShift so, changes
are only for object-openshift.yaml.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-29 14:09:31 +05:30
Sébastien Han c0123cf182 ceph: add cephfs mirroring support
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-28 19:21:18 +01:00
Travis Nielsen 7472cfd9fa ceph: enable pod disruption budgets by default
The PDBs have been stabilizing for a good length of time now since
they were created in v1.1, and later redesigned in v1.5. The time
has come to enable the feature by default so everyone can enjoy
the stable Rook storage even while draining K8s nodes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-28 10:25:33 -07:00
Blaine Gardner 2caeb8bfe3 Merge pull request #6833 from sp98/configure-osd-upgrade-timeout
ceph: configure health check timeout for OSDs when upgrading
2021-01-28 09:46:12 -07:00
Travis Nielsen ac6bbe5902 docs: remove obsolete doc for handling rebooting nodes
The approach to handle node restarts has been much improved since the
creation of this doc, with the new approach to drain nodes and
for the application to control the restarts with the pod disruption
budgets. We now remove this obsolete documentation and recommend
the PDBs to be configured.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-27 14:47:14 -07:00
Travis Nielsen 475260de35 Merge pull request #6875 from Madhu-1/fix-6830
ceph: add option to set FSGroupPolicy for CSI PVC
2021-01-27 08:34:45 -07:00
Santosh Pillai 4237aa7ecc ceph: configure health check timeout for OSDs when upgrading
added waitTimeoutForHealthyOSD in the cluster crd that defines the time (in minutes) the operator would wait before an OSD can be stopped for
upgrade. This PR also removes the ok-to-continue logic for osds as its already handled by ok-to-stop. The default value is 10 minutes.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-01-27 11:13:05 +05:30
Travis Nielsen 4b43e7afa5 ceph: allow removal of arbitrary osds on pvcs and simplify pvc names
OSDs on PVCs previously were always replaced with PVCs of a given name.
For on-prem scenarios where the OSDs might need to be removed instead
of replaced, the operator now allows holes to exist in the PVC index
names as long as there are a sufficient number of PVCs to meet the
criteria for the deviceSet.count.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-26 14:38:12 -07:00
Travis Nielsen 53ed11f15b build: remove the edgefs operator from rook
The EdgeFS operator has been deprecated for some time in Rook.
If the replacement is added back to Rook it can be completed
according to the new guidelines in the documentation.
https://rook.io/docs/rook/master/storage-providers.html

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-25 17:51:24 -07:00
Sébastien Han 720380e3d3 Merge pull request #7024 from travisn/remove-cockroachdb
build: Remove the CockroachDB operator from Rook
2021-01-25 10:04:44 +01:00
Travis Nielsen fe7ed021c5 Merge pull request #7050 from subhamkrai/disable-admission-controller
ceph: disable admission controller for k8s older than v1.16.0
2021-01-22 09:15:36 -07:00
Sébastien Han c82ae69ad9 Merge pull request #6641 from thotz/vault-kms-integration
ceph: vault kms configuration for ceph object store
2021-01-22 14:21:34 +01:00
Jiffin Tony Thottan 01649a8cd2 docs: update vault kms with RGW details
The KMS encryption via HashiCorp Vault can be consumed for RGW, adding those details
in the doc.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-01-22 16:04:28 +05:30
Sébastien Han c84d42871b Merge pull request #7052 from leseb/fix-olm-dup
ceph: remove the append flag when overriding
2021-01-22 11:28:20 +01:00
Sébastien HanandUmanga Chapagain chapagainumanga@gmail.com 330d0e90a5 ceph: remove the append flag when overriding
It looks like combining --append with --overwrite will cause duplicate
entries of our CRDs in the CSV definition. Removing the flag fixes the
duplicates.

Diff before and after the fix:

```
1118c1118
<     createdAt: 2021-01-22T11-07-27Z
---
>     createdAt: 2021-01-22T11-06-24Z
1126a1127,1159
>       - kind: CephBlockPool
>         name: cephblockpools.ceph.rook.io
>         version: v1
>       - kind: CephClient
>         name: cephclients.ceph.rook.io
>         version: v1
>       - kind: CephCluster
>         name: cephclusters.ceph.rook.io
>         version: v1
>       - kind: CephFilesystem
>         name: cephfilesystems.ceph.rook.io
>         version: v1
>       - kind: CephNFS
>         name: cephnfses.ceph.rook.io
>         version: v1
>       - kind: CephObjectRealm
>         name: cephobjectrealms.ceph.rook.io
>         version: v1
>       - kind: CephObjectStore
>         name: cephobjectstores.ceph.rook.io
>         version: v1
>       - kind: CephObjectStoreUser
>         name: cephobjectstoreusers.ceph.rook.io
>         version: v1
>       - kind: CephObjectZoneGroup
>         name: cephobjectzonegroups.ceph.rook.io
>         version: v1
>       - kind: CephObjectZone
>         name: cephobjectzones.ceph.rook.io
>         version: v1
>       - kind: CephRBDMirror
>         name: cephrbdmirrors.ceph.rook.io
>         version: v1
```

Co-authored-by: Umanga Chapagain chapagainumanga@gmail.com
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-22 11:12:36 +01:00
Madhu Rajanna e305810bf0 ceph: add option to set FSGroupPolicy for CSI PVC
in kubernetes 1.19 it added a support to set FSGroupPolicy
in the csidriver object, same supported is added to both
CephFS and RBD. where user will have an option to set
the FSGroupPolicy for CephFS and RBD separately.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-01-22 14:26:56 +05:30