Commit Graph
782 Commits
Author SHA1 Message Date
Travis Nielsen 53ed11f15b build: remove the edgefs operator from rook
The EdgeFS operator has been deprecated for some time in Rook.
If the replacement is added back to Rook it can be completed
according to the new guidelines in the documentation.
https://rook.io/docs/rook/master/storage-providers.html

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-25 17:51:24 -07:00
Sébastien Han 720380e3d3 Merge pull request #7024 from travisn/remove-cockroachdb
build: Remove the CockroachDB operator from Rook
2021-01-25 10:04:44 +01:00
Travis Nielsen fe7ed021c5 Merge pull request #7050 from subhamkrai/disable-admission-controller
ceph: disable admission controller for k8s older than v1.16.0
2021-01-22 09:15:36 -07:00
subhamkrai f7d7c2979a ceph: disable admission controller for k8s older than v1.16.0
v1 version of admission controller require minimum v1.16.0
of k8s. So, this commits disable admission controller when
k8s version older than v1.16.0.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-22 14:21:03 +05:30
Jiffin Tony Thottan 968b002a6f test: validation test for RGW vault authentication
Extending existing deploy-validate-vault.sh to include RGW as well

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-01-22 10:44:17 +05:30
subhamkrai f17f908c31 ceph: disable admission controller for CephUpgradeSuite
disabling admission controller for the upgrade test.
Upgrade test was using older controller runtime version
and api v1 need latest version controller runtime(v0.7).

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-21 22:26:02 +05:30
subhamkrai 9898e00ac5 ceph: update admission controller to v1 from v1beta1
this commit update admission controller to v1 from
v1beta1. v1beta1 is deprecated in v1.16+ and unavailable
in v1.22+.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-21 22:26:02 +05:30
Travis Nielsen 2294851ca3 build: remove the cockroachdb operator from rook
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-20 17:28:58 -07:00
Travis Nielsen 1e111897a9 ceph: allow progressing status during integration test cleanup
The integration tests check after every test suite that the cluster is
either ready or connected. Sometimes the operator is in progressing state
because the reconcile may be triggered by some event that is unpredictable
at the end of the test. For test stability we allow the progressing status.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-13 15:58:35 -07:00
Travis Nielsen 08c227dbdd ceph: remove unexpected array type in crd schema for tests
Clean up an unexpected array type in the test schema for pre-k8s-1.16
clusters

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-13 11:20:53 -07:00
subhamkrai fcd1529b23 ci: add option to download artifacts in canary test
this commit add option to download artifacts in
canary integration test for better debug.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-12 21:06:30 +05:30
Travis Nielsen 1705c64ced ceph: add devices to schema at overall storage level
With the 1.5 schema added to the CRDs, the devices at the root
level of the storage element were missed. Now the ability to
specify devices at the root storage level to apply to all
nodes is restored.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-11 11:46:27 -07:00
Sébastien Han 6be647b03d Merge pull request #6915 from BlaineEXE/add-obc-upgrade-test
ceph: update smoke & upgrade tests to verify obc is bound
2021-01-11 11:00:21 +01:00
Blaine Gardner 9b0ba6ae8b ceph: add obc to upgrade test
Add object bucket claim to Ceph upgrade test.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-01-08 09:56:00 -07:00
Satoru Takeuchi d1fc283404 docs: extend timeout in readme
1800s is too short to complete all suites. Let's make it the same
value as CI.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-01-08 12:16:54 +00:00
Blaine Gardner 40fd80cf14 ceph: update smoke test to verify obc is bound
When verifying OBC creation, validating that secret and configmap exist
is good, but the definitive validation is to check that the OBC's phase
is "Bound".

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-01-07 13:58:22 -07:00
Sébastien Han 05fe296436 Merge pull request #6825 from binoue/rename-webhooks
ceph: rename webhooks
2021-01-07 09:08:14 +01:00
subhamkrai f2ea2daf07 ceph: remove extra space
this commit removes extra space from
func test_csi.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-05 14:58:35 +05:30
subhamkrai 6ea06713b6 ceph: remove leftover line
this commit removes the leftover line from PR #6800.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-05 10:46:51 +05:30
binoue 70ed282cc4 ceph: rename webhooks
Currently, ValidatingWebhookConfiguration has two same named webhooks.
This causes kube-apiserver to create error logs

Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
2020-12-25 17:21:26 +09:00
Blaine Gardner b8dc2a3214 ceph: update lib bucket provisioner
Update to the latest lib bucket provisioner code.
Fixes issue 6650

Modifies CRD for objectbucketclaims to fix an additional bug where an
ObjectBucket's 'ClaimRef' is lost due to the CRD validation being
specified incorrectly.

Changes OBC deletion/cleanup to delete the bucket before the user. A
user cannot be deleted without an unsafe purge option if the user has
buckets associated to it.

Does not reintroduce bug 6767 from previous fix for 6650

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-18 12:48:54 -07:00
Travis Nielsen ba0d7288d8 ceph: delete object store in test in a defer method
Always attempt to delete the object store in case some test
fails and aborts the remainder of the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-17 18:21:16 -07:00
Travis Nielsen 114a949cfb ceph: test latest ceph on raw devices and previous ceph on partitions
The latest ceph-volume does not support creating OSDs on partitions.
The github actions only have a partition available, so we will run
the github tests on v14.2.12 and v15.2.7 that still support partitions,
while the Jenkins environment has a raw device available where we
can run the latest versions of Ceph in the tests.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-17 18:21:16 -07:00
Travis Nielsen 2f59a96bcf Merge pull request #6840 from travisn/snapshot-test-version
ceph: use the correct snapshot controller version in the tests
2020-12-16 11:40:17 -07:00
Travis Nielsen c15499ed7e ceph: use the correct snapshot controller version in the tests
The snapshot controller is specifying to use the canary image instead
of the expected version. Update the tests to set the correct version.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-16 10:25:00 -07:00
Travis Nielsen 55203f5069 build: replace hostpath provisioner with static pvs
The hostpath provisioner has never been supported and neither is it
working on K8s 1.20. For the tests we will simply use static
local PVs so we can avoid the whole problem of an unsupported
hostpath provisioner.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-15 22:13:25 -07:00
Travis Nielsen f3f06da322 Merge pull request #6809 from BlaineEXE/add-namespace-comments-to-manifests
ceph: add namespace meta-comments to manifests
2020-12-11 13:43:42 -07:00
subhamkrai 24c3002307 ci: minor update to validate script
canary test failing because the validate
script not updating the argument passed.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-12-10 13:12:39 +05:30
Blaine Gardner 26c9512962 ceph: add namespace meta-comments to manifests
Add meta-comments to manifests to allow basic templating via `sed`.

Add the following types of meta-comments:
- # namespace:X
  - A basic namespace
  - replace the field with a namespace for X
- # serviceaccount:namespace:X
  - A service account namespace for SCC
  - e.g., "system:serviceaccount:<ns>:rook-ceph-system"
  - Replace the namespace "<ns>" with with a namespace for X
- # provisioner:namespace:X
  - A provisioner identifier with namespace prefix
  - e.g., "<ns>.cephfs.csi.ceph.com"
  - e.g., "<ns>.ceph.rook.io/bucket"
  - Replace the namespace "<ns>" with a namespace for X

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-09 16:21:31 -07:00
Sébastien Han 8aaff235bb ceph: ability to set external services ports
When the cluster is external we want to expose manager service port
along with the endpoints.
This allows us to connect but Rook will keep on using 9283 as a facing
port for Prometheus and more.

```
monitoring:
  enabled: true
  ...
  ...
  externalMgrPrometheusPort: 9283
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-12-04 09:52:09 +01:00
subhamkrai c22590e2cb ci: helm integration test is failing on pre-k8s-1.16
testCephHelmSuite is failing in pre-k8s v1.16
due to invalid schema. v1beta1 doesn't support
null value so returning quotes instead of an empty
string.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-12-04 10:52:32 +05:30
Blaine Gardner 3b4ee6c8e3 ceph: update object bucket provisioner library
The library for object bucket provisioning is updated to fix errors
during provisioning.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-03 09:43:12 -07:00
Blaine Gardner 87ee8dce29 Merge pull request #6679 from leseb/add-log-collector
ceph: add log collector
2020-12-01 12:08:50 -07:00
Sébastien Han c6a87203ca ceph: add log collector
We can now collect logs directly into a side-car container.
A new CRD spec has been added:

spec:
  logCollector:
    enabled: true
    periodicity: 24h

Every 24h we will rotate log files for each Ceph daemon.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-12-01 16:30:17 +01:00
Hiroyuki Kaneko 862edaf371 test: fixed a wrong test suite name
A test suite name is not existing in the integration test suites.
Replace the wrong name "TestRookClusterInstallation_SmokeTest" with "TestARookClusterInstallation_SmokeTest"

Signed-off-by: Hiroyuki Kaneko <hkaneko@redhat.com>
2020-11-27 23:25:15 +09:00
Satoru Takeuchi a17cec45a6 Merge pull request #6709 from leseb/fix-6708
ceph: fix RBAC for cron cash pruner
2020-11-27 15:18:36 +09:00
Shachar Sharon 0786b72752 test: avoid minikube deprecated option
Minikube deprecates '--vm-driver' option. When user starts new minkube
run with '--driver=xxx', the test will fail and cause him to use the wrong
default value of 'virtualbox'.

Tested with minikube=v1.15.1, driver=kvm2 on Fedora32.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2020-11-26 18:16:49 +02:00
Shachar Sharon 72791c7151 test: use proper shell variable in minikube script
Users may prefer to use docker alternatives, by setting DOCKERCMD
variable, which is set by 'common.sh'. Use it in 'copy_image_to_cluster'
instead of plain 'docker'.

Tested with DOCKERCMD=podman (podman-2.1.1).

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2020-11-26 18:16:42 +02:00
Sébastien Han 13da4a1521 ceph: fix RBAC for cron cash pruner
The rook-ceph-system service account was lacking delete permission and
the operator was complaining.

Closes: https://github.com/rook/rook/issues/6708
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-26 11:29:47 +01:00
Renan Campos ad9dcb0c05 ceph: periodically prune crash entries older than user-provided days
Rook's crashcollector pod posts entries to the ceph cluster when a crash occurs.
Over time the number cluster may hold crash entries needlessly.
To clean up old crash entries, this PR adds a field to the ceph cluster CR for the user to specify the number of days a crash entry should be kept for.
Providing a value for the field keepXDays creates a cronjob that runs every day at midnight, calling "ceph crash prune <keepXDays>".

Closes: https://github.com/rook/rook/issues/6332
Signed-off-by: Renan Campos <rcampos@redhat.com>
2020-11-24 17:06:18 -05:00
Sébastien Han f86f8300f9 Merge pull request #6568 from aruniiird/upgrade-dependencies-for-operator-sdk-1.x
Bump Controller Runtime version to 0.6
2020-11-19 17:45:18 +01:00
Travis Nielsen 54560250e9 Merge pull request #6497 from sp98/osd-pdb-reconciler
ceph: OSD PDB reconciler changes
2020-11-18 17:30:24 -07:00
Arun Kumar Mohan 65d16bfc94 ceph: manual changes needed for kubernetes api updates
Fetched latest lib-bucket-provisioner changes as well.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2020-11-18 21:14:01 +05:30
Sébastien Han 4545ddc11f Merge pull request #6666 from leseb/fix-6580
ci: fix device intermittent failure
2020-11-18 15:39:15 +01:00
Sébastien Han a08e08104b Merge pull request #6553 from leseb/mirroring-snap-sched
ceph: add snapshot scheduling for mirrored pools
2020-11-18 15:34:55 +01:00
Santosh Pillai 8602b9c116 ceph: osd pdb reconciler changes
-creates a single PDB (max-unavailable=1) for all OSDs.  This PDB allows one OSD to go down at a given time.
-When a drain is detected, blocking PDBs (max-unavailable=0) will be created for each failure domain that is not being drained and the main PDB (max-unavilable=1) will be deleted. This will allow all the OSDs in the currently drained failure domain to be removed while blocking the deletion  of OSDs in other failure domains.
-Once the PGs are healthy again, the blocking PDBs will be deleted and the main PDB will be restored.
-Add PG healthcheck timeout
-Delete any legacy node drain pods and blocking OSD PDBs

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2020-11-18 18:25:07 +05:30
Sébastien Han 38e475601b ci: probe the device and wait for udev
When we are done creating the partitions it's good to give the kernel
some time to reprobe the device and for udev to finish syncing up.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-18 12:16:14 +01:00
Sébastien Han dcd84e63ac ci: use sudo test if the device is a block
Permissions on the disk might changed due to the partitions being
created. So the CI user is not able to read the device correctly.

Closes: https://github.com/rook/rook/issues/6580
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-18 12:15:57 +01:00
Sébastien Han afc7ecff31 ceph: add snapshot scheduling for mirrored pools
Now, we can schedule snapshots on pools from the CephBlockPool CR when
the pool is mirrored.
It can be enabled like this:

```
mirroring:
  enabled: true
  mode: pool
  snapshotSchedules:
    - interval: 24h # daily snapshots
      startTime: 14:00:00-05:00
```

Multiple schedules are supported since snapshotSchedules is a list.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-18 09:50:09 +01:00
subhamkrai 49a3511532 ci: manifests changes to run gh action
to run the integration test there needs
to be a few changes in manifests like using
`deviceFilter` and other related changes.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-11-18 09:31:07 +05:30