Commit Graph
119 Commits
Author SHA1 Message Date
Sébastien Han b178c8ae2c ci: add arm64 hosted runner
Thanks to @xin3liang we now have 2 hosted runners on ARM64. So let's add
a small integration test for it.
The action will run every day at midnight.

Successful run: https://github.com/rook/rook/pull/7946/checks?check_run_id=2726403458

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-02 11:54:43 +02:00
Madhu Rajanna 3a3bfdefb4 ceph: add github action for helm lint
adding github action for helm lint to
catch the static check errors in helm
charts.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-26 14:27:59 +05:30
Travis Nielsen e1b64e3624 Merge pull request #7945 from subhamkrai/bump-golang1.16
build: bump to golang version 1.16
2021-05-24 23:20:52 -06:00
Sébastien Han c9f28b85cf Merge pull request #7948 from subhamkrai/nfs-action
nfs: add github action for NFS integration test
2021-05-20 17:37:04 +02:00
Sébastien Han 003e2535b5 ci: wait a bit for the python script to succeed
We still see the CI failing with:

```
Traceback (most recent call last):
  File "/etc/ceph/create-external-cluster-resources.py", line 801, in <module>
    raise err
  File "/etc/ceph/create-external-cluster-resources.py", line 798, in <module>
    rjObj.main()
  File "/etc/ceph/create-external-cluster-resources.py", line 779, in main
    generated_output = self.gen_json_out()
  File "/etc/ceph/create-external-cluster-resources.py", line 613, in gen_json_out
    self._gen_output_map()
  File "/etc/ceph/create-external-cluster-resources.py", line 598, in _gen_output_map
    self.out_map['MONITORING_ENDPOINT_PORT'] = self.get_active_and_standby_mgrs()
  File "/etc/ceph/create-external-cluster-resources.py", line 389, in get_active_and_standby_mgrs
    self.endpoint_dial(monitoring_endpoint)
  File "/etc/ceph/create-external-cluster-resources.py", line 245, in endpoint_dial
    "unable to connect to endpoint: {}".format(endpoint_str))
__main__.ExecutionFailureException: unable to connect to endpoint: 172.17.0.7:9283
command terminated with exit code 1
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-20 09:44:49 +02:00
subhamkrai a19a7f90e1 nfs: add github action for NFS integration test
this commit add github action for NFS
integration test.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-20 10:53:14 +05:30
subhamkrai de96a14e31 ci: restrict push image build action to run on fork
this commit restrict push image build action to run on
fork repo.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-19 13:37:35 +05:30
subhamkrai bf7b4609b1 build: bump to golang version 1.16
this commit bump to golang version 1.16.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-19 13:35:20 +05:30
Travis Nielsen f61083bb2b Revert "ci: restrict push image build action to run on fork"
This reverts commit e0206c56d5.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-18 11:39:58 -06:00
subhamkrai 9d8853d49a ci: add missing tag in master push images
tags are missing in master pushed images
to shallow problem. this commit add command
to unshallow.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-18 12:14:17 +05:30
Sébastien Han ffd263c539 Merge pull request #7891 from leseb/wait-for-prom
ceph: wait for the prometheus module to be ready
2021-05-12 17:22:45 +02:00
Sébastien Han cc763df671 ceph: wait for the prometheus module to be ready
Before running the external script test let's make sure the prometheus
exporter is loaded. We often see in the CI errors like:

```
  kubectl -n rook-ceph exec $(kubectl get pod -l app=rook-ceph-tools -n rook-ceph -o jsonpath='{.items[0].metadata.name}') -- python3 /etc/ceph/create-external-cluster-resources.py --rbd-data-pool-name replicapool
  shell: /bin/bash -e {0}
  env:
    GOROOT: /opt/hostedtoolcache/go/1.16.4/x64
    MINIKUBE_HOME: /home/runner/work/_temp
Execution Failed: unable to connect to endpoint: 172.17.0.10:9283
Traceback (most recent call last):
  File "/etc/ceph/create-external-cluster-resources.py", line 801, in <module>
    raise err
  File "/etc/ceph/create-external-cluster-resources.py", line 798, in <module>
    rjObj.main()
  File "/etc/ceph/create-external-cluster-resources.py", line 779, in main
    generated_output = self.gen_json_out()
  File "/etc/ceph/create-external-cluster-resources.py", line 613, in gen_json_out
    self._gen_output_map()
  File "/etc/ceph/create-external-cluster-resources.py", line 598, in _gen_output_map
    self.out_map['MONITORING_ENDPOINT_PORT'] = self.get_active_and_standby_mgrs()
  File "/etc/ceph/create-external-cluster-resources.py", line 389, in get_active_and_standby_mgrs
    self.endpoint_dial(monitoring_endpoint)
  File "/etc/ceph/create-external-cluster-resources.py", line 245, in endpoint_dial
    "unable to connect to endpoint: {}".format(endpoint_str))
__main__.ExecutionFailureException: unable to connect to endpoint: 172.17.0.10:9283
command terminated with exit code 1
```

but then when we log into the runner, we can see the command works.
So let's wait a little bit for the module to be up.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-12 14:59:57 +02:00
subhamkrai e0206c56d5 ci: restrict push image build action to run on fork
this commit restrict push image build action to run on
fork repo.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-12 12:23:11 +05:30
subhamkrai f988e2e517 ci: update action api token
this commit update the api token
to use correct secret.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-07 11:24:15 +05:30
Travis Nielsen 6b27cdd2fb build: include aws region when publishing build
For the github action to publish the build to aws, we also need to
include the aws-region setting in the action settings.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-06 12:23:38 -06:00
subhamkrai 4683254419 ci: push build images to Docker hub using gh action
We are moving towards github action.
This commit allows pushing build images
to Docker hub using gh action for master
build similar to what Jenkins does.

And, also removing `Publish if Master` step
from the Jenkinsfil file.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-06 12:45:24 +05:30
Sébastien Han 0f7128ece9 build: fix parallel build
We now the generate the CRDs as well as the CSV on a single GOARCH only:
amd64. This helps us avoiding builds colliding with each others.

Closes: https://github.com/rook/rook/issues/7655
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-29 17:31:22 +02:00
Sébastien Han 4b3007bf73 Merge pull request #7735 from leseb/mirror-pacific-tip
ci: use latest pacific for mirroring test
2021-04-29 09:45:25 +02:00
Jiffin Tony Thottan 3cacd8db32 test: add seperate test for rgw vault integration
RGW supports only v2 version of kv engine, current integration test is written for kv v1,
hence seperating existing test into two.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-28 14:40:08 +05:30
subhamkrai 7974d46b20 ci: improve github action writing
we have lots of duplicate codes for
github action. this commit refactor
them to a bash script which will minimize
duplicate codes.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-27 20:15:52 +05:30
Sébastien Han 6a97721576 ci: use latest pacific for mirroring test
So we can get https://github.com/ceph/ceph/pull/40985 and have a more
stable CI. Once v16.2.2 is released we can revert this.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 18:30:05 +02:00
Sébastien Han a9641a9355 Merge pull request #7716 from leseb/debug-cii
ci: mitigate block locking issues and flapping mirroring test
2021-04-22 15:38:59 +02:00
Sébastien Hanandsubhamkrai srai@redhat.com a59df3c4dd ci: add missing variable to the release/master action
The UDEV rule was missing the device to apply the rule out.

Co-authored-by: subhamkrai srai@redhat.com
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 14:07:46 +02:00
Sébastien Han 77be184f13 ci: wait a bit longer for the mirroring to happen
30sec was not enough see:

```
  # let's wait a bit for the image to be present
  timeout 30 sh -c 'until [ "$(kubectl exec -n rook-ceph-secondary deploy/rook-ceph-tools -t -- rbd -p replicapool ls|grep -c test)" -eq 1 ]; do echo "waiting for image to be mirrored" && sleep 1; done'
  shell: /bin/bash -e {0}
  env:
    GOROOT: /opt/hostedtoolcache/go/1.16.3/x64
    MINIKUBE_HOME: /home/runner/work/_temp
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:47.053+0000 7f824b13e2c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:47.053+0000 7f824b13e2c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:48.317+0000 7f31f6d372c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:48.317+0000 7f31f6d372c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:49.597+0000 7fa3a2ac52c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:49.597+0000 7fa3a2ac52c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:50.873+0000 7fab17fb12c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:50.873+0000 7fab17fb12c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:52.169+0000 7f944f52f2c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:52.169+0000 7f944f52f2c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
Error: Process completed with exit code 124.
```

However, when logging into the runner, the image was replicated.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:16 +02:00
Sébastien Han 834e5b2685 ci: let master/release runners alive on failure
If some jobs from master/release branches fail then we keep the runner
alive so we can debug it.
By default, runners will naturally timeout after 6 hours. See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-21 18:36:12 +02:00
Satoru Takeuchi 957465af88 ceph: prevent disk flapping problem in master
We resolved disk flapping problem (#7405) with #7467. However, this fix
was not applied to integration test in master.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 04:10:09 +00:00
Travis Nielsen 8c1ac86325 Merge pull request #7691 from leseb/macos-build
ci: build and run various make commands on MacOS
2021-04-20 10:36:42 -06:00
Travis Nielsen f6984255fc Merge pull request #7680 from subhamkrai/fix-build
ci: make build not failing
2021-04-20 09:46:46 -06:00
Sébastien Han 57131e1a90 ci: build and run various make commands on MacOS
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00
Sébastien Han 2b87efef89 ci: add missing condition for action
Let's skip multi-cluster-mirroring if skip-ci label is set.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00
Sébastien Han 1028cd41f9 Merge pull request #7623 from leseb/add-rbd-mirroring-ci
ci: add rbd-mirror integration test
2021-04-20 16:53:59 +02:00
Sébastien Han 9a28e130b6 ci: add rbd-mirror integration test
Long due but finally there, the CI now test the block mirroring between
2 clusters.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 16:27:53 +02:00
subhamkrai 9305f12ecf ci: make build not failing
currently, the build action is not failing if there
is/are any modified files after the build. this commit
will validate build with `git status` after `make build`
command will fail if any changed file found.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-20 13:10:33 +05:30
Sébastien Han 6a33395c4a ci: separate PR and release/master branch push into different files
We now have a single file with all the integration test that will run
when code is pushed to a release branch or merged in master.

Also, we have separate files with integration tests that run on pull
requests against master or a release branch.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:43:31 +02:00
Sébastien Han 5a7f405010 ci: remove the build action
This action is not needed anymore since other actions build already.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:41:47 +02:00
Sébastien Han aecec6f870 ci: only run cassandra job on cassandra code
Only run the cassandra integration if the PR involves cassandra, has the
"cassandra" label on it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:41:47 +02:00
Blaine Gardner a818a01a13 test: do not allow go.test parallel dependencies
Dependencies for `make go.test` have to run in order, so call them
explicitly from the target rather than allowing them to be done in
parallel if `make -j` flag is used.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-16 12:24:39 -06:00
Yannis Zarkadas e3c747db47 test: add GitHub Action for Cassandra integration test
Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2021-04-14 22:03:02 +03:00
Sébastien Han 01ad7afe53 ci: ability to skip ci with a label
Additionally to skip ci tag in the commit message from https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/
We can now skip integration tests by applying the "skip-ci" label at the
**creation time** of the PR. Obviously adding later works but will still
trigger the test in the initial opening of the PR. Subsequent pushes
will skip integration tests.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-14 12:17:34 +02:00
Travis Nielsen 10f350adc9 Merge pull request #7371 from travisn/go-1.16
build: Update the build to use go 1.16 while still supporting 1.15
2021-04-13 07:33:19 -06:00
Travis Nielsen 31c4ef622e build: update the build to use go 1.16
With the release of go 1.16, we update the tools. The -i flag is
now deprecated, no longer necessary for building in a private directory.
The go modules are cleaned up at the same time.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-12 15:01:14 -06:00
Travis Nielsen bfa302f2c2 ci: run integration tests on k8s 1.21
With the release of K8s 1.21.0 we now run the integration tests
on v1.21.0. To keep consistent with running master builds on five
recent releases, we keep master builds back to v1.15 and skip other
less changed versions such as v1.16 and 1.19. Jenkins builds just run
five versions, while github actions run four versions.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-12 13:10:43 -06:00
Sébastien Han 89f4c4a81a ci: always run golangci-lint
This action wasn't not running on release branch. Now enabling it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-12 17:37:59 +02:00
Sébastien Han 15901277d5 ci: use authenticated requests for minikube action on suite
We have seen the CI failing to run the minikube action, reaching the API
limit. By using authenticated requests, we can overcome this limitation.
Follow-up of https://github.com/rook/rook/pull/7503/commits/1504fb869b5ed5b8f7df362bc9df3b45dd8ac9ea

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-12 14:26:38 +02:00
subhamkrai 81624b255f ci: update minikube version
this commit update the github action
minikube version as minikube setup on
latest k8s version are failing.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-08 12:32:04 +05:30
subhamkrai 20ab5befb0 ci: enable mgr suite for PR only
this commit enable mgr suite to run only
for PRs.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-07 11:34:41 +05:30
subhamkrai 24ec105b58 ci: enable it test on push
this commit enable IT test to run on master push.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-07 11:32:42 +05:30
Sébastien Han 350379d43e ci: always run crd gen
Always run crd-gen so we can catch issues even if no go code was
modified.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-06 14:12:58 +02:00
Sébastien Han d6caae5004 Merge pull request #7503 from leseb/action-auth
ci: use authenticated requests for minikube action
2021-03-30 15:00:11 +02:00
Sébastien Han 1504fb869b ci: use authenticated requests for minikube action
We have seen the CI failing to run the minikube action, reaching the API
limit. By using authenticated requests, we can overcome this limitation.

Closes: https://github.com/rook/rook/issues/7482
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-30 14:46:40 +02:00