Commit Graph
78 Commits
Author SHA1 Message Date
Travis Nielsen bfa302f2c2 ci: run integration tests on k8s 1.21
With the release of K8s 1.21.0 we now run the integration tests
on v1.21.0. To keep consistent with running master builds on five
recent releases, we keep master builds back to v1.15 and skip other
less changed versions such as v1.16 and 1.19. Jenkins builds just run
five versions, while github actions run four versions.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-12 13:10:43 -06:00
Sébastien Han 89f4c4a81a ci: always run golangci-lint
This action wasn't not running on release branch. Now enabling it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-12 17:37:59 +02:00
Sébastien Han 15901277d5 ci: use authenticated requests for minikube action on suite
We have seen the CI failing to run the minikube action, reaching the API
limit. By using authenticated requests, we can overcome this limitation.
Follow-up of https://github.com/rook/rook/pull/7503/commits/1504fb869b5ed5b8f7df362bc9df3b45dd8ac9ea

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-12 14:26:38 +02:00
subhamkrai 81624b255f ci: update minikube version
this commit update the github action
minikube version as minikube setup on
latest k8s version are failing.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-08 12:32:04 +05:30
subhamkrai 20ab5befb0 ci: enable mgr suite for PR only
this commit enable mgr suite to run only
for PRs.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-07 11:34:41 +05:30
subhamkrai 24ec105b58 ci: enable it test on push
this commit enable IT test to run on master push.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-07 11:32:42 +05:30
Sébastien Han 350379d43e ci: always run crd gen
Always run crd-gen so we can catch issues even if no go code was
modified.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-06 14:12:58 +02:00
Sébastien Han d6caae5004 Merge pull request #7503 from leseb/action-auth
ci: use authenticated requests for minikube action
2021-03-30 15:00:11 +02:00
Sébastien Han 1504fb869b ci: use authenticated requests for minikube action
We have seen the CI failing to run the minikube action, reaching the API
limit. By using authenticated requests, we can overcome this limitation.

Closes: https://github.com/rook/rook/issues/7482
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-30 14:46:40 +02:00
Blaine Gardner 660d2fe442 Merge pull request #7493 from BlaineEXE/codespell-tidy
test: document or clean up codespell flags
2021-03-29 10:07:53 -06:00
Blaine Gardner 71c41180be test: document or clean up codespell flags
Document codespell flags that are necessary. Remove codespell flags with
minor code changes if possible.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-03-26 15:00:49 -06:00
Sébastien Han cf00a86fc2 ceph: force yq and controller-gen versions
Let's not use any controllen-gen version we might find on the system,
always pull the version we need.
Same goes for yq.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-26 09:21:33 +01:00
Sébastien Han eac743ffc7 ceph: shorten make crds
Let's use "makge crds" instead of "make crds-gen", it's shorter and
simplier to remember.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-26 09:08:46 +01:00
Sébastien Han 7069aa4f16 ci: fix disk issue with crashing osd
Let's try to create a 'ceph' user with the uid 167 and add it to the
'disk' group so that if the permissions on the disk change then the OSD
should be able to access the disk.

Closes: https://github.com/rook/rook/issues/7405
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-25 09:55:35 +01:00
Sébastien Han 89e57dc807 Merge pull request #7051 from subhamkrai/gh-action-matrix
ci: it test to run on multiple k8s version for master/release
2021-03-24 14:46:27 +01:00
Sébastien Han 3f6257c751 Merge pull request #7431 from subhamkrai/yamllint-silence-warning
ci: silence yamllint warnings
2021-03-22 11:31:12 +01:00
subhamkrai 61ba7ec45a ci: it test to run on multiple k8s version for master/release
this commit enables IT to run on multiple k8s version
and for master/release, all the 5 k8s version
'v1.15.12','v1.16.15','v1.18.15','v1.19.7','v1.20.0'
similar to jenkins.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-22 15:55:01 +05:30
subhamkrai 378c1c8506 ci: silence yamllint warnings
this commit silence yamllint warnings

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-18 15:38:01 +05:30
Sébastien Han c43058a98c Merge pull request #7355 from leseb/more-disks-ci
ceph: add more osds to the ci
2021-03-18 10:56:19 +01:00
subhamkrai 07df8220d8 ci: run gh action on multiple k8s version
this commit enables gh action to run

Flex test on only 'v1.15.12',
Smoke and Helm tests on both 'v1.15.12','v1.20.2',
Mgr, Multicluster and upgrade tests on only 'v1.20.2'

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-18 09:03:56 +05:30
Travis Nielsen bdbf264e68 ceph: integration tests in github actions skip cleanup
The integration tests run independently in the github actions so there is
no need to cleanup from every test. The cleanup is still needed in the
Jenkins tests where all the suites run serially.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 11:26:11 -06:00
Travis Nielsen 32c628108c Merge pull request #7416 from leseb/debug-ci
ci: debug runners on failures
2021-03-16 08:57:32 -06:00
Sébastien Han ab50e75e08 Merge pull request #7402 from subhamkrai/yaml-ignore
ceph: ignore monitoring folder from yamllint
2021-03-16 10:46:17 +01:00
Sébastien Han 88c0eb57cd ci: debug runners on failures
If one of the suite fails, it is sometimes handy to ssh into the
instance to precisely understand what's going on.
So by applying a debug label so as "debug-smoke-suite" you will be able
to ssh into the runner to debug. By default it looks
like the runner will be cancelled automatically after 6 hours as per the
official doc: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes

I've had a different experience, and somehow 45min seems to be the limit
but I might be wrong.

Once the label is applied, go into the check and the once you are
interested in and search the step "setup tmate session for debugging".
It should output lines with the connection details.

To get the list of all labels see: https://github.com/rook/rook/labels?q=debug-
Example run is available here: https://github.com/rook/rook/pull/7416/checks?check_run_id=2120103989

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 10:43:38 +01:00
Sébastien Han 31db03fece ceph: auto-gen crds
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:

* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`

will be autogenerated for us.

We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.

It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:

```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```

The corresponding API type markers are:

```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```

For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 09:16:24 +01:00
subhamkrai acfbc78392 ceph: ignore the monitoring folder from yamllint
this commit adds the monitoring folder to
yamllint ignore list.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-16 12:33:37 +05:30
Blaine Gardner 6324746c18 Merge pull request #7409 from BlaineEXE/fix-ci
ci: update minikube github action to fix apt-get
2021-03-15 10:39:26 -06:00
Blaine Gardner b744a08c54 ci: update minikube github action to fix apt-get
Fix an apt-get update issue by bumping to the latest (fixed) version of
manusa/actions-setup-minikube (v2.3.1).

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-03-15 10:10:34 -06:00
Satoru Takeuchi 4766954dfc ceph: suppress golanglint-ci complaints
Suppress the following complaints.

```
$ golangci-lint run -E gosec
pkg/operator/test/client.go:244:13: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
        randIdx := rand.Intn(len(nodes.Items))
                   ^
tests/framework/installer/ceph_manifests_v1.5.go:44:19: G107: Potential HTTP request made with variable url (gosec)
        response, err := http.Get(url)
                         ^
pkg/daemon/ceph/client/pool.go:182:6: ineffectual assignment to stats (ineffassign)
        var stats = new(PoolStatistics)
            ^
pkg/operator/k8sutil/pod_test.go:34:2: ineffectual assignment to container (ineffassign)
        container, err := GetMatchingContainer([]v1.Container{}, expectedName)
        ^
```

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-12 15:34:27 +00:00
subhamkrai b46e31dbee ci: add gh action for yaml linter
this commit adds gh action for yaml linter.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-11 08:23:18 +05:30
Sébastien Han cc822a531c ceph: add more osds to the ci
We now create 3 disks which will give us 2 OSDs and potentially help us
to catch more errors in our scenarios, especially on iterations.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-09 15:24:35 +01:00
Blaine Gardner bc9c5f2057 ceph: start using latest Ceph v15 in integration
Do not use old Ceph v15.2.7 in integration that was working around the
issue of ceph-volume partition support being removed in v15.2.8.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
(cherry picked from commit 99cee7648cfaf72b02d62e281a2ac3a18bce4132)
2021-03-03 14:06:19 +01:00
Sébastien Han 5fccaf5e62 Merge pull request #7328 from leseb/ubuntu-18-ci
ceph: stick with ubuntu-18.04 on the CI
2021-03-01 18:44:41 +01:00
Sébastien Han 0adf7624d9 ceph: stick with ubuntu-18.04 on the CI
Github just switched to use Ubuntu 20.04 for the ubuntu-latest and it is
breaking some of our scenarios so far. Especially scenario creating
partitions.

Closes: https://github.com/rook/rook/issues/7327
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-01 18:20:42 +01:00
Sébastien Han bd9010e7fc Merge pull request #7296 from subhamkrai/fix-codegen
ci: fix codegen err
2021-03-01 11:35:06 +01:00
subhamkrai 9b6bf82925 ci: update checkout version
this commit update checkout version.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-01 10:30:00 +05:30
subhamkrai fd6f6d4953 ci: replace cd with working-directory
this commit replace `cd` with `working-directory`
in gh action.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-01 10:30:00 +05:30
subhamkrai afed5bf109 ci: fix codegen err
codegen is currently broken in master.
this commit fix the broken codegen by fixing
the correct working directory in gh action.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-01 10:29:26 +05:30
Arun Kumar Mohan bc4dd24306 ceph: making changes to fix CI test failures
Changed the random port creation's max value from 255 to 254, as the IP
with value 255 fails.
Re-raise/propagate the 'ExecutionFailureException' exception so that we
don't end-up in ZERO exit value.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-02-24 23:54:42 +05:30
Arun Kumar Mohan 72b88b4267 ceph: adding the test-data file into the image fs
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-02-24 23:54:42 +05:30
Satoru Takeuchi 2b073c3bcd ceph: add the integration test of osd on lv-backed pvc
Add missing OVC on LV-backed PVC test. Although we already have
the pvc test case, the code path of lv-backed pvc is a bit different
from raw device.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-02-09 10:29:11 +00:00
subhamkrai 6d66f0a6f2 ci: update canary-integration-test
this commit update canary-integration-test
to remove `object-openshift.yaml` file from
`dry-run`.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-02-01 13:51:16 +05:30
Travis Nielsen 050abd14b6 bot: update stale bot to a github action
The stale bot has stopped working. Now we use a github
action to mark and close the stale issues.
Issues will be marked as stale after 90 days
and closed after one week. PRs will be marked
as stale after 90 days and closed after two weeks.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-26 08:53:19 -07:00
Jiffin Tony Thottan 968b002a6f test: validation test for RGW vault authentication
Extending existing deploy-validate-vault.sh to include RGW as well

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-01-22 10:44:17 +05:30
Sébastien Han 94d9c69361 ceph: add kubectl-check-ownerreferences in ci
After each non-pvc and pvc scenario we run the
kubectl-check-ownerreferences tool to analyze potential incorrect owner
references. We had issues in the past with incorrect owner ref so this
tool might help us ensure we are doing things right.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-20 10:51:11 +01:00
Sébastien Han 2c021fb357 ci: bump golang version to 1.15
The rest of the CI already uses 1.15 so let's homogenize that.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-18 08:47:31 +01:00
subhamkrai fcd1529b23 ci: add option to download artifacts in canary test
this commit add option to download artifacts in
canary integration test for better debug.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-01-12 21:06:30 +05:30
Sébastien Han adaa90b668 ci: split the test into their own files
If each has its own file we can restart them independently in case of
intermittent failures.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-08 16:21:36 +01:00
Marc Nuri 801ee22887 ci: upgrade actions-setup-minikube to v2.2.0
Signed-off-by: Marc Nuri <marc@marcnuri.com>
2020-12-20 07:58:51 +01:00
Travis Nielsen 114a949cfb ceph: test latest ceph on raw devices and previous ceph on partitions
The latest ceph-volume does not support creating OSDs on partitions.
The github actions only have a partition available, so we will run
the github tests on v14.2.12 and v15.2.7 that still support partitions,
while the Jenkins environment has a raw device available where we
can run the latest versions of Ceph in the tests.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-17 18:21:16 -07:00