Commit Graph
885 Commits
Author SHA1 Message Date
Andy Bursavich 11f0add706 build: update golangci-lint version and nolint comment format
Signed-off-by: Andy Bursavich <abursavich@gmail.com>
2021-06-21 08:50:04 -07:00
Andy Bursavich 7ab1baa794 test: extract utils dependency on k8s.io/kubernetes
Signed-off-by: Andy Bursavich <abursavich@gmail.com>
2021-06-21 08:36:08 -07:00
subhamkrai 0ff452fdd1 ceph: remove unnecessary file
this commit removes `mysql_helper.go`,
unnecessary file.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-06-15 15:28:13 +05:30
Travis Nielsen 44d31f148e ceph: update the toolbox in the upgrade test
The upgrade test needs to also update the toolbox to ensure that it will not
be denied access to the cluster when the insecure connections are disabled
by the operator.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-10 10:05:30 -06:00
Sébastien Han fccbadd837 ci: only run ceph-csi volume replication on k8s >= 1.16
Prior to 1.16 we do not injec the volumereplication CRDs so the
csi-rbdplugin-provisioner will fail to fetch the CRD.

Closes: https://github.com/rook/rook/issues/8039
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-10 15:13:05 +02:00
Travis Nielsen 054004ba76 Merge pull request #7946 from leseb/ci-arm64-run
ci: add arm64 runner
2021-06-02 10:13:31 -06:00
Sébastien Han ba8940800f Merge pull request #8005 from leseb/luks-label
ceph: add ceph cluster fsid to LUKS header
2021-06-02 14:52:56 +02:00
Sébastien Han b178c8ae2c ci: add arm64 hosted runner
Thanks to @xin3liang we now have 2 hosted runners on ARM64. So let's add
a small integration test for it.
The action will run every day at midnight.

Successful run: https://github.com/rook/rook/pull/7946/checks?check_run_id=2726403458

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-02 11:54:43 +02:00
Sébastien Han 0004869f66 ceph: add ceph cluster fsid to LUKS header
When configuring encrypted on-pvc clusters we now set the cluster fsid
in the LUKS header. If needed we can then determine if the encrypted
block is part of our cluster or not. We also attach the pvc_name in case
it might become useful.

Closes: https://github.com/rook/rook/issues/7991
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-31 15:01:52 +02:00
Sébastien Han 7c42b61f3c ci: add more debug logs to failed job
Gather more logs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-31 14:16:46 +02:00
subhamkrai 7f0179c1ba ceph: update cert-manager version
this commit update cert-manager to v1.3.1

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-31 11:27:02 +05:30
subhamkrai 9773634cc1 ceph: wait for cainjector to inject the CA certificates
adding 20s sleep to let cainjector to inject the CA
certificates into cert-manager webhook.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-31 11:27:02 +05:30
subhamkrai c738982503 ceph: add more log to admission controller
adding more logs in case there are errors
starting admission controller.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-31 11:27:02 +05:30
Travis Nielsen 1af84e34c0 Merge pull request #7778 from henryzhangsta/ceph-cluster-helm-chart
ceph: implement CephCluster CRD Helm Chart
2021-05-27 12:41:01 -06:00
Henry Zhang ecf6bdcfae ceph: add build and tests for rook-ceph-cluster Helm chart
Adds rook-ceph-cluster chart to chart build, add tests.
Pulled out some common functionality between the Helm and non-Helm installers

Signed-off-by: Henry Zhang <me@henry.dev>
2021-05-27 01:08:27 -07:00
Blaine Gardner 285dad339b ceph: update references to v15.2.11 to v16.2.4
Ceph v16 will be the default in Rook v1.7. Update references in the
master branch from v15 (currently v15.2.11) to v16 (currently v16.2.4).

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-05-26 12:30:18 -06:00
Travis Nielsen e46dc1cd17 Merge pull request #7928 from leseb/fix-7927
ci: retry when fetching online manifests
2021-05-18 10:52:47 -06:00
Sébastien Han f1f3416f49 ci: retry when fetching online manifests
We now retry up to 3 times if the manifest fails to be fetched online.

Closes: https://github.com/rook/rook/issues/7927
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-18 17:22:12 +02:00
Satoru Takeuchi c2adbdd8a8 ceph: remove an missing field in crd
`CephObjectStore->gateway->type` is not used. Rook has only supported s3-like
interface and hasn't had no code which handles `type` field.

In addition, this field was removed from CRD in the following commit.

ceph: auto-gen crds
31db03fece

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-05-17 07:51:13 +00:00
subhamkrai f988e2e517 ci: update action api token
this commit update the api token
to use correct secret.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-07 11:24:15 +05:30
subhamkrai 4d66b7067b ci: right permission to build script
master build are failing due to bad
permission on build script. this
commit giv +rxw permission to the file.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-07 06:57:08 +05:30
Travis Nielsen 2b2a1f66c4 ceph: multicluster test cleanup of external cluster first
The multicluster test was never removing the finalizer of the external
cluster since the core cluster was being removed first. Now we remove
the external cluster first to ensure the finalizer will be removed
properly instead of forcefully by the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-06 14:51:07 -06:00
subhamkrai 4683254419 ci: push build images to Docker hub using gh action
We are moving towards github action.
This commit allows pushing build images
to Docker hub using gh action for master
build similar to what Jenkins does.

And, also removing `Publish if Master` step
from the Jenkinsfil file.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-06 12:45:24 +05:30
Rakshith R 7987e1b8a2 ceph: update snapshot APIs from v1beta1 to v1
This commit updates external-snapshotter version to
v4.0.0 which supports snapshots v1.
Rook now defaults to enabling RBD and CephFS snapshotter
for K8s >= v1.17 and disabling it for K8s <= v1.16.
Supporting changes in documents and examples yaml files
are made.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-05 21:04:10 +05:30
Sébastien Han 15a12577f5 ceph: fix external mode setup
Because of the recent CRD changes made, the mon count had a minimum of
1, making the configuration of the external cluster impossible. The
operator would fail to add the finalizer:

```
2021-04-29 16:57:38.429451 E | ceph-cluster-controller: failed to reconcile. failed to add finalizer: failed to add finalizer "cephcluster.ceph.rook.io" on "test-external": CephCluster.ceph.rook.io "test-external" is invalid: spec.mon.count: Invalid value: 0: spec.mon.count in body should be greater than or equal to 1
```

We now fixed the CRD as well as adding the CR status.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-03 12:07:40 +02:00
Sébastien Han 0c806588d2 Merge pull request #7790 from leseb/retry-vg
ci: retry vgcreate on failure
2021-04-29 19:01:35 +02:00
Sébastien Han c63a42a661 ci: retry vgcreate on failure
Try to mitigate the following error from the CI:

```
+ sudo vgcreate test-rook-vg /dev/sdb
  Can't open /dev/sdb exclusively.  Mounted filesystem?
Error: Process completed with exit code 5.

```

Now we retry up to 3 times to create the volume group.

Closes: https://github.com/rook/rook/issues/7487
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-29 18:24:48 +02:00
Sébastien Han 0f7128ece9 build: fix parallel build
We now the generate the CRDs as well as the CSV on a single GOARCH only:
amd64. This helps us avoiding builds colliding with each others.

Closes: https://github.com/rook/rook/issues/7655
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-29 17:31:22 +02:00
Jiffin Tony Thottan 3cacd8db32 test: add seperate test for rgw vault integration
RGW supports only v2 version of kv engine, current integration test is written for kv v1,
hence seperating existing test into two.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-28 14:40:08 +05:30
subhamkrai 7974d46b20 ci: improve github action writing
we have lots of duplicate codes for
github action. this commit refactor
them to a bash script which will minimize
duplicate codes.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-27 20:15:52 +05:30
Blaine Gardner 72b66c39c2 ceph: add regression test for obc infinite pending
Add an integration regression test for Ceph Object Storage Object Bucket
Claim (OBC) to verify that the OBC stays in "Bound" state after being
created. The claim should not fall back into "Pending" (or any other)
state soon after being created.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-22 15:23:34 -06:00
Sébastien Han a9641a9355 Merge pull request #7716 from leseb/debug-cii
ci: mitigate block locking issues and flapping mirroring test
2021-04-22 15:38:59 +02:00
Sébastien Han d17fc6ac12 ci: increase the number of lock retries
We have seen new cases where retrying to lock the device 3 times is not
enough. It's the same race we had experienced where Ceph tries to
acquire a lock on the device but systemd-udevd does the same too.
Retrying 20 times every 0.1sec seems to mitigate that issue in the CI.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:37 +02:00
Satoru Takeuchi 8a48dbb00d ceph: stop vendoring
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 02:01:56 +00:00
Travis Nielsen f6984255fc Merge pull request #7680 from subhamkrai/fix-build
ci: make build not failing
2021-04-20 09:46:46 -06:00
Sébastien Han 1d4488659e Merge pull request #7682 from leseb/fix-7669
ceph: use v14 image again for integration tests
2021-04-20 17:21:56 +02:00
Sébastien Han d35ddcbecc ceph: use v14 again
Since Rook 1.6 is using raw mode for simple OSD scenarios we can use v14
again without having issue with ceph-volume.

We keep the upgraade test with v14.2.12 since Rook 1.5 does not have the
raw code to deploy OSDs.

Closes: https://github.com/rook/rook/issues/7669
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 16:51:46 +02:00
Travis Nielsen 56e7af3ee0 Merge pull request #7331 from thotz/healthcheckwhensslenabled
ceph: fix healthcheck in case ssl enabled for rgw
2021-04-20 08:32:17 -06:00
Jiffin Tony Thottan bcbe707bcb ceph: fix healthcheck incase ssl enabled for rgw
In case ssl enabled for RGW, bucket healthcheck won't work since access is denied from the server.
In that case configure S3 agent using "InsecureSkipVerify: true" option.

Fixes: 7288
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-20 14:23:48 +05:30
subhamkrai 9305f12ecf ci: make build not failing
currently, the build action is not failing if there
is/are any modified files after the build. this commit
will validate build with `git status` after `make build`
command will fail if any changed file found.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-20 13:10:33 +05:30
bipuladh 7d5cc8d06b ceph: adds a container to support volume replication controller
this commit adds a new container inside of rbd-provisioner.

Signed-off-by: bipuladh <badhikar@redhat.com>
2021-04-15 21:39:26 +05:30
Yannis Zarkadas 760e81edba test: skip cassandra integration test in jenkins
Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2021-04-14 22:03:02 +03:00
Yannis Zarkadas a004942f0b cassandra: fix integration test waiting logic
The integration test for Cassandra waited for Pods to become Running
before trying to connect to the Cassandra cluster. However, the check
should be that the cluster is Ready, not that the container state is Running.

If we only check for the Running state, we bump into a scenario where:
- The Cassandra container has started (thus is in Running state)
- The Cassandra server has not stated yet

and then tests will fail.

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2021-04-14 21:40:50 +03:00
Yannis Zarkadas c40a116110 cassandra: add missing sidecar permissions to testing manifests
Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2021-04-14 21:40:50 +03:00
Yannis Zarkadas b71f1077b5 test: replace in-house config loader for controller-runtime's
Rook's testing utilities include a function for loading the default
kubeconfig for a cluster. This function requires maintainance effort and
also doesn't support authentication methods like Basic Authentication.
Instead of adding support for it, drop the config loader and use the one
provided by the controller-runtime library.

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2021-04-14 21:40:50 +03:00
Blaine Gardner d78b2a41d6 ceph: work around radosgw-admin fifo file io error
Work around issue https://github.com/rook/rook/issues/7573
and make sure integration tests check for regressions.

Eventually we should use the RADOS Gateway admin REST API, but for now
we need to work around an issue where the built version of
'radosgw-admin' has incompatibilities with the RADOS Gateway version
running in the Ceph cluster.

Of note, Rook built on the Ceph Pacific image will not support
some 'radosgw-admin' commands to Ceph Nautilus (v14) or Octopus (v15)
clusters.

Further complicating matters, the flag used for the workaround changes
between Ceph v16.2.0 and v16.2.1 (both Pacific).

This bug needs to be treated a little differently than most of the ways
Rook handles different commands for different Ceph versions because this
is based on the Ceph version that is installed in the container with the
Rook operator primarily and not the version of Ceph running in the
cluster.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-13 12:08:02 -06:00
Travis Nielsen 9546a2ef94 ceph: integration tests use master tag even in release branch
The integration tests pick up the manifests directly from the examples
folder, including with the release tag. In the release branch the Jenkins
build still uses the master build when building locally before tagging
it with the release tag. So the tests need to use the master tag.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 456e2f3a79)
2021-04-08 23:43:16 -06:00
Travis Nielsen 665856b90a ceph: enable pacific as a supported ceph version
With the Ceph Pacific release coming this week we add support
in Rook for Pacific with the Rook v1.6 release coming soon.
The integration tests will now run across nautilus, octopus,
and pacific to cover all supported Ceph versions. The default
examples still specify Octopus until there is more bake time
for Pacific.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-07 06:59:04 -06:00
Travis Nielsen f620df9fd8 ceph: remove duplicate function to validate rgw in the ci
The CI had a duplicate function for validating the rgw daemon
was running

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-07 06:59:04 -06:00
Sébastien Han 41781839c3 Merge pull request #7480 from leseb/ci-object-user
ci: try to mitigate object user error
2021-03-26 14:48:00 +01:00