Commit Graph
174 Commits
Author SHA1 Message Date
Travis Nielsen 054004ba76 Merge pull request #7946 from leseb/ci-arm64-run
ci: add arm64 runner
2021-06-02 10:13:31 -06:00
Sébastien Han ba8940800f Merge pull request #8005 from leseb/luks-label
ceph: add ceph cluster fsid to LUKS header
2021-06-02 14:52:56 +02:00
Sébastien Han b178c8ae2c ci: add arm64 hosted runner
Thanks to @xin3liang we now have 2 hosted runners on ARM64. So let's add
a small integration test for it.
The action will run every day at midnight.

Successful run: https://github.com/rook/rook/pull/7946/checks?check_run_id=2726403458

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-02 11:54:43 +02:00
Sébastien Han 0004869f66 ceph: add ceph cluster fsid to LUKS header
When configuring encrypted on-pvc clusters we now set the cluster fsid
in the LUKS header. If needed we can then determine if the encrypted
block is part of our cluster or not. We also attach the pvc_name in case
it might become useful.

Closes: https://github.com/rook/rook/issues/7991
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-31 15:01:52 +02:00
Sébastien Han 7c42b61f3c ci: add more debug logs to failed job
Gather more logs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-31 14:16:46 +02:00
subhamkrai 7f0179c1ba ceph: update cert-manager version
this commit update cert-manager to v1.3.1

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-31 11:27:02 +05:30
subhamkrai 9773634cc1 ceph: wait for cainjector to inject the CA certificates
adding 20s sleep to let cainjector to inject the CA
certificates into cert-manager webhook.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-31 11:27:02 +05:30
subhamkrai c738982503 ceph: add more log to admission controller
adding more logs in case there are errors
starting admission controller.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-31 11:27:02 +05:30
Blaine Gardner 285dad339b ceph: update references to v15.2.11 to v16.2.4
Ceph v16 will be the default in Rook v1.7. Update references in the
master branch from v15 (currently v15.2.11) to v16 (currently v16.2.4).

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-05-26 12:30:18 -06:00
subhamkrai f988e2e517 ci: update action api token
this commit update the api token
to use correct secret.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-07 11:24:15 +05:30
subhamkrai 4d66b7067b ci: right permission to build script
master build are failing due to bad
permission on build script. this
commit giv +rxw permission to the file.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-07 06:57:08 +05:30
subhamkrai 4683254419 ci: push build images to Docker hub using gh action
We are moving towards github action.
This commit allows pushing build images
to Docker hub using gh action for master
build similar to what Jenkins does.

And, also removing `Publish if Master` step
from the Jenkinsfil file.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-06 12:45:24 +05:30
Sébastien Han 0c806588d2 Merge pull request #7790 from leseb/retry-vg
ci: retry vgcreate on failure
2021-04-29 19:01:35 +02:00
Sébastien Han c63a42a661 ci: retry vgcreate on failure
Try to mitigate the following error from the CI:

```
+ sudo vgcreate test-rook-vg /dev/sdb
  Can't open /dev/sdb exclusively.  Mounted filesystem?
Error: Process completed with exit code 5.

```

Now we retry up to 3 times to create the volume group.

Closes: https://github.com/rook/rook/issues/7487
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-29 18:24:48 +02:00
Sébastien Han 0f7128ece9 build: fix parallel build
We now the generate the CRDs as well as the CSV on a single GOARCH only:
amd64. This helps us avoiding builds colliding with each others.

Closes: https://github.com/rook/rook/issues/7655
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-29 17:31:22 +02:00
Jiffin Tony Thottan 3cacd8db32 test: add seperate test for rgw vault integration
RGW supports only v2 version of kv engine, current integration test is written for kv v1,
hence seperating existing test into two.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-28 14:40:08 +05:30
subhamkrai 7974d46b20 ci: improve github action writing
we have lots of duplicate codes for
github action. this commit refactor
them to a bash script which will minimize
duplicate codes.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-27 20:15:52 +05:30
Satoru Takeuchi 8a48dbb00d ceph: stop vendoring
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 02:01:56 +00:00
subhamkrai 9305f12ecf ci: make build not failing
currently, the build action is not failing if there
is/are any modified files after the build. this commit
will validate build with `git status` after `make build`
command will fail if any changed file found.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-20 13:10:33 +05:30
Travis Nielsen 665856b90a ceph: enable pacific as a supported ceph version
With the Ceph Pacific release coming this week we add support
in Rook for Pacific with the Rook v1.6 release coming soon.
The integration tests will now run across nautilus, octopus,
and pacific to cover all supported Ceph versions. The default
examples still specify Octopus until there is more bake time
for Pacific.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-07 06:59:04 -06:00
Travis Nielsen f620df9fd8 ceph: remove duplicate function to validate rgw in the ci
The CI had a duplicate function for validating the rgw daemon
was running

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-07 06:59:04 -06:00
subhamkrai 3c591ff7a1 ci: update make crds-gen to make crds
this commit update make crds-gen to make crds

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-26 15:48:18 +05:30
Sébastien Han c43058a98c Merge pull request #7355 from leseb/more-disks-ci
ceph: add more osds to the ci
2021-03-18 10:56:19 +01:00
Travis Nielsen c23238cddb ceph: refactor integration tests for simplification
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 11:26:10 -06:00
Sébastien Han 31db03fece ceph: auto-gen crds
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:

* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`

will be autogenerated for us.

We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.

It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:

```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```

The corresponding API type markers are:

```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```

For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 09:16:24 +01:00
Sébastien Han cc822a531c ceph: add more osds to the ci
We now create 3 disks which will give us 2 OSDs and potentially help us
to catch more errors in our scenarios, especially on iterations.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-09 15:24:35 +01:00
Sébastien Han 128dc4cb79 Merge pull request #7325 from subhamkrai/cert-manager
ceph: use cert-manager for admission webhooks
2021-03-09 10:38:27 +01:00
subhamkrai d594a6c904 ceph: remove config-admission-controller.sh
this commit removes file `config-admission-controller.sh`
as we don't need this anymore and update the doc to point
`tests/scripts/deploy_admission_controller.sh` to start
admission controller.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-05 16:39:52 +05:30
subhamkrai 40a99a20ff ceph: webhook check if one of port or securePort are non-zero
this commit adds new crd to admission webhooks to check
if one of the port or securePort are set to non-zero
value in objectstore crd.

If both are set to 0 in crd we'll see error like
```
Error from server (invalid create: either of port or securePort
fields should be not be zero): error when creating "object-test.yaml":
admission webhook
"cephobjectstore-wh-rook-ceph-admission-controller-rook-ceph.rook.io"
denied the request: invalid create: either of port or securePort fields
should be not be zero.
```

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-03 21:22:04 +05:30
subhamkrai 8b6ec60653 ceph: use cert-manager for admission webhooks
we are now using cert-manager for certificates
instead of a self-signed certificate for admission
webhooks.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-03 20:22:53 +05:30
subhamkrai afed5bf109 ci: fix codegen err
codegen is currently broken in master.
this commit fix the broken codegen by fixing
the correct working directory in gh action.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-01 10:29:26 +05:30
Travis Nielsen 043f947321 build: remove yugabytedb operator
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-06 00:03:41 -07:00
Sébastien Han 685e0a7a55 ceph: fail of any codegen files are found
If any files are found after running `make codegen` we fail.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-01 15:13:37 +01:00
Sébastien Han c0123cf182 ceph: add cephfs mirroring support
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-28 19:21:18 +01:00
Sébastien Han 720380e3d3 Merge pull request #7024 from travisn/remove-cockroachdb
build: Remove the CockroachDB operator from Rook
2021-01-25 10:04:44 +01:00
Jiffin Tony Thottan 968b002a6f test: validation test for RGW vault authentication
Extending existing deploy-validate-vault.sh to include RGW as well

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-01-22 10:44:17 +05:30
subhamkrai 9898e00ac5 ceph: update admission controller to v1 from v1beta1
this commit update admission controller to v1 from
v1beta1. v1beta1 is deprecated in v1.16+ and unavailable
in v1.22+.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-21 22:26:02 +05:30
Travis Nielsen 2294851ca3 build: remove the cockroachdb operator from rook
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-20 17:28:58 -07:00
subhamkrai fcd1529b23 ci: add option to download artifacts in canary test
this commit add option to download artifacts in
canary integration test for better debug.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-12 21:06:30 +05:30
Sébastien Han 05fe296436 Merge pull request #6825 from binoue/rename-webhooks
ceph: rename webhooks
2021-01-07 09:08:14 +01:00
subhamkrai f2ea2daf07 ceph: remove extra space
this commit removes extra space from
func test_csi.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-05 14:58:35 +05:30
subhamkrai 6ea06713b6 ceph: remove leftover line
this commit removes the leftover line from PR #6800.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-05 10:46:51 +05:30
binoue 70ed282cc4 ceph: rename webhooks
Currently, ValidatingWebhookConfiguration has two same named webhooks.
This causes kube-apiserver to create error logs

Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
2020-12-25 17:21:26 +09:00
subhamkrai 24c3002307 ci: minor update to validate script
canary test failing because the validate
script not updating the argument passed.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-12-10 13:12:39 +05:30
Shachar Sharon 0786b72752 test: avoid minikube deprecated option
Minikube deprecates '--vm-driver' option. When user starts new minkube
run with '--driver=xxx', the test will fail and cause him to use the wrong
default value of 'virtualbox'.

Tested with minikube=v1.15.1, driver=kvm2 on Fedora32.

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2020-11-26 18:16:49 +02:00
Shachar Sharon 72791c7151 test: use proper shell variable in minikube script
Users may prefer to use docker alternatives, by setting DOCKERCMD
variable, which is set by 'common.sh'. Use it in 'copy_image_to_cluster'
instead of plain 'docker'.

Tested with DOCKERCMD=podman (podman-2.1.1).

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2020-11-26 18:16:42 +02:00
Sébastien Han 38e475601b ci: probe the device and wait for udev
When we are done creating the partitions it's good to give the kernel
some time to reprobe the device and for udev to finish syncing up.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-18 12:16:14 +01:00
Sébastien Han dcd84e63ac ci: use sudo test if the device is a block
Permissions on the disk might changed due to the partitions being
created. So the CI user is not able to read the device correctly.

Closes: https://github.com/rook/rook/issues/6580
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-18 12:15:57 +01:00
Pete Birley 152a05c85e ceph: update to helm 3 for the rook chart
This updates the chart to make use of helm3 which has been released
for some time, and also permits CRDs to be installed pror to other objects
allowing the chart to be deployed at the same time as CRs for rook objects.

Co-authored-by: Pete Birley <pete@port.direct>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-11 14:41:24 -07:00
Mateusz Gozdek 8ba3762fa4 docs: fix bunch of typos
Found by running the following command:

codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-06 10:01:04 +01:00