Commit Graph
201 Commits
Author SHA1 Message Date
Sébastien Han 522fa7a782 ci: wait longer on fs mirroring
It looks like the CI needs more time.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-26 11:01:11 +02:00
Sébastien Han d8e9885fd6 ci: only run snyk on merges and not on PRs
We cannot use the secret when PRs are pushed from forks so let's run the
security scan only after PRs are merged.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-26 10:55:43 +02:00
subhamkrai 0ebbb78373 ci: update snyk token
Update snyk token, it was pointing to wrong token.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-26 08:51:51 +05:30
Sébastien Han 3b5bbf28d5 Merge pull request #9003 from leseb/vault-k8s-auth
osd: add support for k8s with vault kms
2021-10-25 18:02:48 +02:00
subhamkrai faefd097b0 ci: add action for synk as rook licence
Adding github action for synk as licence for rook.
This will replace FOSSA licence in future.

Links to read about synk
1. https://snyk.io/
2. https://github.com/snyk/snyk

Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-25 13:33:09 +05:30
Sébastien Han 18a4047679 osd: add support for k8s with vault kms
Rook cluster-wide encryption can now use the native Kubernetes
authentication to interact with vault KMS instead of using the token
method.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-21 13:59:28 +02:00
Sébastien Han 07f006ac43 ci: fix mirror scenario by removing ownerref
When we copy the peer token secret from a namespace to another we also
copy the ownerref, however the creation succeeds but the controller
removes the secret since the uid of the owner do not exist in that
namespace.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-21 12:11:10 +02:00
subhamkrai 6d0a06365f ci: update minikube and kubernetes version
Updating minikube version to `v1.23.2` and
k8s version to `v1.22.2`

Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 10:04:31 +05:30
Blaine Gardner 65619c2677 test: get more partition info setting up ci disk
Add some commands to get more partition info when setting up the GH
action runner's disk for use in integration tests. This will both aid in
debugging and may "jog" the system such that it will no longer need to
reload the partition info when running the OSD prepare job.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-18 11:29:13 -06:00
Blaine Gardner ba7745d4af test: do not use head because of pipe errors
The `head` command exits once it has output which can result in a
SIGPIPE error if the command piping its output to head hasn't yet
finished. Use `awk 'FNR <= 1'` instead, which waits on the input pipe to
close before it exits.

See here for more info:
https://unix.stackexchange.com/a/256047

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-15 10:14:41 -06:00
Sébastien Han efa60b44c8 ci: only validate osds for mirror test
The Filesystem is deployed later in the run so at this point we only
need to validate the OSDs are running. This was failing since we were
validating the cephfs pool which is not present yet.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-15 10:36:15 +02:00
Blaine Gardner 34a8b42e97 test: try to un-flake multi-cluster-mirror test
Try to un-flake the multi-cluster-mirror test that keeps failing on this
PR.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-13 11:06:51 -06:00
Travis Nielsen 74dac725a4 test: run the daily arm tests against official builds
The daily arm test suite is failing due to the new local-build
image tag. Instead of waiting for the arm build to complete,
we can just pick up the latest tag from the same branch
that was already pushed to dockerhub and no need to build
again.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-10-12 15:26:18 -06:00
Travis Nielsen 7006b899c9 Merge pull request #8612 from subhamkrai/trigger-after-tag
ci: trigger push build action after tag creation
2021-10-12 11:38:04 -06:00
subhamkrai c53304c3b4 ci: trigger push build action after tag creation
earlier push build action was not triggered due
to github action limitation.
```
An action in a workflow run can’t trigger a new workflow run.
```

so now, we'll use user personal toke to create tag so that push
build action pick the user not github action who created the tag.

Closes: https://github.com/rook/rook/issues/8580
Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-12 10:22:38 +05:30
Blaine Gardner 956430826c rgw: add integration test for committing period
Add to the RGW multisite integration test a verification that the RGW
period is committed on the first reconcile and not committed on the
second reconcile.

Do this in the multisite test so that we verify that this works for
both the primary and secondary multi-site cluster.

To add this test, the github-action-helper.sh script had to be modified
to
1. actually deploy the version of Rook under test
2. adjust how functions are called to not lose the `-e` in a subshell
3. fix wait_for_prepare_pod helper that had a failure in the middle
   of its operation that didn't cause failures in the past

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-11 15:24:59 -06:00
Travis Nielsen af29d1d801 build: run canary tests against the local-build tag
The canary tests were still picking up the tag from operator.yaml
and toolbox.yaml instead of the new test local-build tag.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 6f48dce3f5)
2021-09-30 15:33:48 -06:00
Travis Nielsen aa085c93b9 Merge pull request #8895 from leseb/mgr-weekly
ci: run the orch ceph manager test daily
2021-09-30 15:31:24 -06:00
Blaine Gardner e2ba16f710 build: start tracking rbac generated from helm chart
This is a starting step to be able to generate common.yaml from Helm
charts. For right now, we merely want to be able to determine when the
rendered output of the Helm chart changes.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-09-30 12:20:24 -06:00
Sébastien Han 04e1589a33 ci: run the orch ceph manager test daily
Let's run the test every day at midnight since the code is moving
fast and tends to fail. This gives us some time to fix the issues.
It can be triggered on PR too if the "run-mgr-suite" label is set.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-30 18:54:09 +02:00
Blaine Gardner 24bf99f31c build: use intermediate tmp for offline image gen
Reading from a file, processing the file in a pipe, and outputting the
piped content to the same file can have undefined results, often leading
to an empty file. Use an intermediate temp file for generating the
offline image list.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-09-29 17:31:02 -06:00
Sébastien Han cb3d342515 ci: fix the test name for upgrade suite
Since we were pointing at CephUpgradeSuite test only this was running
all the tests and thus some resources were created by other tests.
Let's just run the test we need.

Closes: https://github.com/rook/rook/issues/8843
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-28 09:12:37 +02:00
Travis Nielsen 45fe8eadd5 Merge pull request #8838 from subhamkrai/comment-tmate-session
ci: run tmate session only on PR
2021-09-27 10:33:52 -06:00
subhamkrai fb13b5cad0 ci: run tmate session only on PR
run tmate only for `pull_request` event and
remove for other events. As during release
when tests fail it takes extra 30 min to fail the
test or someone manually has to end the test
which is painfully especially during release time.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-09-27 19:52:37 +05:30
Sébastien Han 33dbaba38b ci: add daily jobs
We have new jobs now:

* one that runs both smoke and object on the next Pacific version
* one that runs both smoke and object on Ceph master
* one that tests the upgrade from the current pacific stable to the
  pacific devel
* one that tests the upgrade from the current octopus stable to the
  octopus devel

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-27 11:59:04 +02:00
Travis Nielsen f5c1543ddf build: update min k8s version to 1.16
In Rook v1.8 the min version of K8s supported is updated to 1.16.
Users running on older versions of K8s are recommended to update
to 1.16 or newer before updating to Rook v1.8.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:56 -06:00
Travis Nielsen 0a0b9c98bd build: remove obsolete flex driver
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:20 -06:00
Travis Nielsen 0591753673 docs: update master doc links to latest
With the renaming of master docs to latest, all the links also
need to be updated.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-22 12:41:32 -06:00
Sébastien Han f040c37ad1 ci: fix mirror test with v16.2.6
The new version v16.2.6 has a different behavior when it comes to the
number of cephfs-mirror socket files. Previous version had exactly 3 and
now has like way more...
So let's just check for the presence of more sockets.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-21 09:16:33 +02:00
Blaine Gardner c60bf241e1 test: make object e2e test its own test
The CephSmokeSuite is becoming quite large and long, and most of the
length is now related to the object e2e tests. Separate the full object
e2e test into CephObjectSuite, and only test the object 'lite' test in
the CephSmokeSuite.

Resolves #8714

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-09-17 15:27:27 -06:00
Travis Nielsen e2e3fc83fc Merge pull request #8673 from leseb/stale-bot
bot: change issue and pr stale days
2021-09-16 08:31:31 -06:00
Sébastien Han 875aec74cc bot: bump action to v4
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-16 10:04:56 +02:00
Sébastien Han 796abd7805 bot: change issue and pr stale days
Let's be a bit more aggressive to ensure a consistent amount of
in-progress PRs and issues.
So now PR will go stale after 30 days and issues after 60 days.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-16 10:04:46 +02:00
Sébastien Han 73c340ded7 ci: fix pod list
We should not use .items[0].metadata.name if the array length is 0. This
is the case when nothing has been initialized yet. Instead, we should
use .items[*].metadata.name, the wildcard ensures to always return 0
even if nothing is present yet.

Fixes: https://github.com/rook/rook/issues/8676
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-09 17:24:56 +02:00
Sébastien Han 53ad7c73a1 ci: add failfast go flag to stop tests sooner (optionnal)
With the -failfast flag the tests will not continue when one of them
fails. This allows us to log in the runner and observe the exact state
of any failure.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-08 11:24:58 +02:00
Blaine Gardner 61f6429ea4 test: remove NFS and Cassandra github actions
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:34 -06:00
Sébastien Han d4dd0577f9 Merge pull request #8529 from sp98/id-mapping
ceph: add ClusterID and PoolID mappings between local and peer cluster
2021-08-31 17:32:14 +02:00
Santosh Pillai 3f8abec403 ceph: add ClusterID and PoolID mappings between local and peer cluster
During disaster recovery/migration of a cluster, as part of the failover, the
kubernetes artifacts like deployment, PVC, PV, etc will be restored to a new
cluster by the admin. Even if the kubernetes objects are restored the
corresponding RBD/CephFS subvolume cannot be retrieved during CSI operations as
the clusterID and poolID are not the same in both clusters

This PR creates a mapping between Cluster ID and RBD Pool ID between
local cluster and peer cluster.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-08-31 20:08:56 +05:30
subhamkrai ebe21c8f68 ci: add action for shellcheck linter
we are adding new linter for shellcheck.
As we are writing more shell scripts this
will help maintain quality.

Also, doing all the changes required in
bash files to pass this shellcheck.

Closes: https://github.com/rook/rook/issues/8431
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-26 10:03:32 +05:30
Sébastien Han 7c14b32435 ci: build on golang 1.16 and 1.17
The CI now runs the build test against Golang 1.16 and 1.17.
So 1.17 is been added as a supported version.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 11:18:51 +02:00
subhamkrai 029ab61348 ci: validate user before creating tag
we need to validate create tag action
so that only authorized users can trigger
that action.

[skip-ci]

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-19 20:47:05 +05:30
Travis Nielsen c00b290041 Merge pull request #8364 from parth-gr/update-certificates.k8s.io
ceph: update CSR from v1beta1 to v1
2021-08-18 08:00:01 -06:00
subhamkrai 94d8b9b2e6 ci: reduce tmate timeout to 60
reduce tmate timeout to 60 for the pull_request event.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-18 10:29:03 +05:30
subhamkrai 4233bb681e ci: validate user before creating tag
we need to validate create tag action
so that only authorized users can trigger
that action.

[skip-ci]

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-12 20:56:31 +05:30
parth-gr 88d907d9f6 ci: cephSmokeSuite test run on kubernetes 1.22
the CSR certificates k8s is updated to v1 version
so uprading the somkesuite test to run on 1.22 version

Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-12 20:27:17 +05:30
subhamkrai 14d0c402c8 ci: remove brackets when checking repo
having brackets around if is showing
errors. this commit removes the brackets.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-12 10:52:19 +05:30
Ali Maredia b6404ae2ab ceph: rgw multisite intgration testing
This test:
- starts up 2 minikube clusters
- create 2 ceph clusters
- creates object multisite CRDs on each cluster and
syncs the clusters
- writes an object to cluster 1 and reads it on
cluster 2

This commit also adds new functions in
github-action-helper.sh that aid in the multisite
test and the multi cluster mirroring test.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2021-08-11 13:25:43 -04:00
Travis Nielsen 5f64f5eb15 Merge pull request #8513 from subhamkrai/reduce-tmate-time
ci: reduce tmate session time
2021-08-11 08:17:41 -06:00
subhamkrai ea4a705b1a ci: reduce tmate session time
reduce tmate session timeout to 60min for carnary test
and 30min when push/tag event.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-11 15:53:01 +05:30
parth-gr cad1c5320a ci: CephSmokeSuite test run on kubernetes 1.21
Kubernetes 1.22 version didn't support
certificates.k8s.io v1beta1 version
reverting the k8s 1.22 version to 1.21
till the certificates.k8s.io #8364 Pr
didn't get completed
so the CI can run successfully

Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-10 21:52:28 +05:30