Commit Graph
791 Commits
Author SHA1 Message Date
Travis Nielsen 043f947321 build: remove yugabytedb operator
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-06 00:03:41 -07:00
subhamkrai 9be70baf5f ci: intermittent integration test failures due to admission controller
integration test was failing frequently due to some
intermittent failures.
```
Error from server (InternalError): error when creating
"STDIN": Internal error occurred: failed calling webhook.
```
(This error is not coming from us most probably)
This is kind of workaround so we get green build
as we still don't know the root cause.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-02-04 09:42:29 +05:30
Madhu Rajanna 81248f1dce ceph: add configmap get RBAC for rbd provisioner
rbd provisioner need to access the configmap
in different namespaces.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-03 21:28:00 +05:30
Travis Nielsen 81b398f6b6 ceph: add deviceClass to the object store schema
The deviceClass was missing from the object store CRD schema,
disallowing the device class from being specified on
the object store pools.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-02 12:13:23 -07:00
Sébastien Han 685e0a7a55 ceph: fail of any codegen files are found
If any files are found after running `make codegen` we fail.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-01 15:13:37 +01:00
Travis Nielsen bc5be15bb3 Merge pull request #7100 from subhamkrai/disable-adm-controller
ceph: disable admission controller for helm suite
2021-01-29 09:22:33 -07:00
subhamkrai 75030316ca ceph: disable admission controller for helm suite
disable admission controller for helm suite.
v1 version of admissionregistration.k8s.io/v1
require minimum v1.16.0 of k8s. All other suite
was disabled earlier this was left.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-29 12:53:03 +05:30
Sébastien Han c0123cf182 ceph: add cephfs mirroring support
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-28 19:21:18 +01:00
Travis Nielsen 7472cfd9fa ceph: enable pod disruption budgets by default
The PDBs have been stabilizing for a good length of time now since
they were created in v1.1, and later redesigned in v1.5. The time
has come to enable the feature by default so everyone can enjoy
the stable Rook storage even while draining K8s nodes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-28 10:25:33 -07:00
Travis Nielsen 53ed11f15b build: remove the edgefs operator from rook
The EdgeFS operator has been deprecated for some time in Rook.
If the replacement is added back to Rook it can be completed
according to the new guidelines in the documentation.
https://rook.io/docs/rook/master/storage-providers.html

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-25 17:51:24 -07:00
Sébastien Han 720380e3d3 Merge pull request #7024 from travisn/remove-cockroachdb
build: Remove the CockroachDB operator from Rook
2021-01-25 10:04:44 +01:00
Travis Nielsen fe7ed021c5 Merge pull request #7050 from subhamkrai/disable-admission-controller
ceph: disable admission controller for k8s older than v1.16.0
2021-01-22 09:15:36 -07:00
subhamkrai f7d7c2979a ceph: disable admission controller for k8s older than v1.16.0
v1 version of admission controller require minimum v1.16.0
of k8s. So, this commits disable admission controller when
k8s version older than v1.16.0.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-22 14:21:03 +05:30
Jiffin Tony Thottan 968b002a6f test: validation test for RGW vault authentication
Extending existing deploy-validate-vault.sh to include RGW as well

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-01-22 10:44:17 +05:30
subhamkrai f17f908c31 ceph: disable admission controller for CephUpgradeSuite
disabling admission controller for the upgrade test.
Upgrade test was using older controller runtime version
and api v1 need latest version controller runtime(v0.7).

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-21 22:26:02 +05:30
subhamkrai 9898e00ac5 ceph: update admission controller to v1 from v1beta1
this commit update admission controller to v1 from
v1beta1. v1beta1 is deprecated in v1.16+ and unavailable
in v1.22+.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-21 22:26:02 +05:30
Travis Nielsen 2294851ca3 build: remove the cockroachdb operator from rook
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-20 17:28:58 -07:00
Travis Nielsen 1e111897a9 ceph: allow progressing status during integration test cleanup
The integration tests check after every test suite that the cluster is
either ready or connected. Sometimes the operator is in progressing state
because the reconcile may be triggered by some event that is unpredictable
at the end of the test. For test stability we allow the progressing status.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-13 15:58:35 -07:00
Travis Nielsen 08c227dbdd ceph: remove unexpected array type in crd schema for tests
Clean up an unexpected array type in the test schema for pre-k8s-1.16
clusters

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-13 11:20:53 -07:00
subhamkrai fcd1529b23 ci: add option to download artifacts in canary test
this commit add option to download artifacts in
canary integration test for better debug.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-12 21:06:30 +05:30
Travis Nielsen 1705c64ced ceph: add devices to schema at overall storage level
With the 1.5 schema added to the CRDs, the devices at the root
level of the storage element were missed. Now the ability to
specify devices at the root storage level to apply to all
nodes is restored.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-11 11:46:27 -07:00
Sébastien Han 6be647b03d Merge pull request #6915 from BlaineEXE/add-obc-upgrade-test
ceph: update smoke & upgrade tests to verify obc is bound
2021-01-11 11:00:21 +01:00
Blaine Gardner 9b0ba6ae8b ceph: add obc to upgrade test
Add object bucket claim to Ceph upgrade test.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-01-08 09:56:00 -07:00
Satoru Takeuchi d1fc283404 docs: extend timeout in readme
1800s is too short to complete all suites. Let's make it the same
value as CI.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-01-08 12:16:54 +00:00
Blaine Gardner 40fd80cf14 ceph: update smoke test to verify obc is bound
When verifying OBC creation, validating that secret and configmap exist
is good, but the definitive validation is to check that the OBC's phase
is "Bound".

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-01-07 13:58:22 -07:00
Sébastien Han 05fe296436 Merge pull request #6825 from binoue/rename-webhooks
ceph: rename webhooks
2021-01-07 09:08:14 +01:00
subhamkrai f2ea2daf07 ceph: remove extra space
this commit removes extra space from
func test_csi.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-05 14:58:35 +05:30
subhamkrai 6ea06713b6 ceph: remove leftover line
this commit removes the leftover line from PR #6800.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-05 10:46:51 +05:30
binoue 70ed282cc4 ceph: rename webhooks
Currently, ValidatingWebhookConfiguration has two same named webhooks.
This causes kube-apiserver to create error logs

Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
2020-12-25 17:21:26 +09:00
Blaine Gardner b8dc2a3214 ceph: update lib bucket provisioner
Update to the latest lib bucket provisioner code.
Fixes issue 6650

Modifies CRD for objectbucketclaims to fix an additional bug where an
ObjectBucket's 'ClaimRef' is lost due to the CRD validation being
specified incorrectly.

Changes OBC deletion/cleanup to delete the bucket before the user. A
user cannot be deleted without an unsafe purge option if the user has
buckets associated to it.

Does not reintroduce bug 6767 from previous fix for 6650

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-18 12:48:54 -07:00
Travis Nielsen ba0d7288d8 ceph: delete object store in test in a defer method
Always attempt to delete the object store in case some test
fails and aborts the remainder of the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-17 18:21:16 -07:00
Travis Nielsen 114a949cfb ceph: test latest ceph on raw devices and previous ceph on partitions
The latest ceph-volume does not support creating OSDs on partitions.
The github actions only have a partition available, so we will run
the github tests on v14.2.12 and v15.2.7 that still support partitions,
while the Jenkins environment has a raw device available where we
can run the latest versions of Ceph in the tests.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-17 18:21:16 -07:00
Travis Nielsen 2f59a96bcf Merge pull request #6840 from travisn/snapshot-test-version
ceph: use the correct snapshot controller version in the tests
2020-12-16 11:40:17 -07:00
Travis Nielsen c15499ed7e ceph: use the correct snapshot controller version in the tests
The snapshot controller is specifying to use the canary image instead
of the expected version. Update the tests to set the correct version.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-16 10:25:00 -07:00
Travis Nielsen 55203f5069 build: replace hostpath provisioner with static pvs
The hostpath provisioner has never been supported and neither is it
working on K8s 1.20. For the tests we will simply use static
local PVs so we can avoid the whole problem of an unsupported
hostpath provisioner.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-15 22:13:25 -07:00
Travis Nielsen f3f06da322 Merge pull request #6809 from BlaineEXE/add-namespace-comments-to-manifests
ceph: add namespace meta-comments to manifests
2020-12-11 13:43:42 -07:00
subhamkrai 24c3002307 ci: minor update to validate script
canary test failing because the validate
script not updating the argument passed.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-12-10 13:12:39 +05:30
Blaine Gardner 26c9512962 ceph: add namespace meta-comments to manifests
Add meta-comments to manifests to allow basic templating via `sed`.

Add the following types of meta-comments:
- # namespace:X
  - A basic namespace
  - replace the field with a namespace for X
- # serviceaccount:namespace:X
  - A service account namespace for SCC
  - e.g., "system:serviceaccount:<ns>:rook-ceph-system"
  - Replace the namespace "<ns>" with with a namespace for X
- # provisioner:namespace:X
  - A provisioner identifier with namespace prefix
  - e.g., "<ns>.cephfs.csi.ceph.com"
  - e.g., "<ns>.ceph.rook.io/bucket"
  - Replace the namespace "<ns>" with a namespace for X

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-09 16:21:31 -07:00
Sébastien Han 8aaff235bb ceph: ability to set external services ports
When the cluster is external we want to expose manager service port
along with the endpoints.
This allows us to connect but Rook will keep on using 9283 as a facing
port for Prometheus and more.

```
monitoring:
  enabled: true
  ...
  ...
  externalMgrPrometheusPort: 9283
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-12-04 09:52:09 +01:00
subhamkrai c22590e2cb ci: helm integration test is failing on pre-k8s-1.16
testCephHelmSuite is failing in pre-k8s v1.16
due to invalid schema. v1beta1 doesn't support
null value so returning quotes instead of an empty
string.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-12-04 10:52:32 +05:30
Blaine Gardner 3b4ee6c8e3 ceph: update object bucket provisioner library
The library for object bucket provisioning is updated to fix errors
during provisioning.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-12-03 09:43:12 -07:00
Blaine Gardner 87ee8dce29 Merge pull request #6679 from leseb/add-log-collector
ceph: add log collector
2020-12-01 12:08:50 -07:00
Sébastien Han c6a87203ca ceph: add log collector
We can now collect logs directly into a side-car container.
A new CRD spec has been added:

spec:
  logCollector:
    enabled: true
    periodicity: 24h

Every 24h we will rotate log files for each Ceph daemon.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-12-01 16:30:17 +01:00
Hiroyuki Kaneko 862edaf371 test: fixed a wrong test suite name
A test suite name is not existing in the integration test suites.
Replace the wrong name "TestRookClusterInstallation_SmokeTest" with "TestARookClusterInstallation_SmokeTest"

Signed-off-by: Hiroyuki Kaneko <hkaneko@redhat.com>
2020-11-27 23:25:15 +09:00
Satoru Takeuchi a17cec45a6 Merge pull request #6709 from leseb/fix-6708
ceph: fix RBAC for cron cash pruner
2020-11-27 15:18:36 +09:00
Shachar Sharon 0786b72752 test: avoid minikube deprecated option
Minikube deprecates '--vm-driver' option. When user starts new minkube
run with '--driver=xxx', the test will fail and cause him to use the wrong
default value of 'virtualbox'.

Tested with minikube=v1.15.1, driver=kvm2 on Fedora32.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2020-11-26 18:16:49 +02:00
Shachar Sharon 72791c7151 test: use proper shell variable in minikube script
Users may prefer to use docker alternatives, by setting DOCKERCMD
variable, which is set by 'common.sh'. Use it in 'copy_image_to_cluster'
instead of plain 'docker'.

Tested with DOCKERCMD=podman (podman-2.1.1).

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2020-11-26 18:16:42 +02:00
Sébastien Han 13da4a1521 ceph: fix RBAC for cron cash pruner
The rook-ceph-system service account was lacking delete permission and
the operator was complaining.

Closes: https://github.com/rook/rook/issues/6708
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-26 11:29:47 +01:00
Renan Campos ad9dcb0c05 ceph: periodically prune crash entries older than user-provided days
Rook's crashcollector pod posts entries to the ceph cluster when a crash occurs.
Over time the number cluster may hold crash entries needlessly.
To clean up old crash entries, this PR adds a field to the ceph cluster CR for the user to specify the number of days a crash entry should be kept for.
Providing a value for the field keepXDays creates a cronjob that runs every day at midnight, calling "ceph crash prune <keepXDays>".

Closes: https://github.com/rook/rook/issues/6332
Signed-off-by: Renan Campos <rcampos@redhat.com>
2020-11-24 17:06:18 -05:00
Sébastien Han f86f8300f9 Merge pull request #6568 from aruniiird/upgrade-dependencies-for-operator-sdk-1.x
Bump Controller Runtime version to 0.6
2020-11-19 17:45:18 +01:00