Commit Graph
113 Commits
Author SHA1 Message Date
Sébastien Han a044b27650 ci: fix MockExecuteCommandWithOutput mock definition
With the recent exex update we missed this one.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 17:24:30 +01:00
Sébastien Han ca0a30f38d ceph: convert Filesystem controller to the controller-runtime
The CRD watcher has been replaced by the new controller-runtime
framework.
This brings robustness in our operator, meaning that any resources that
are modified will be reconciled into the desired state.

Closes: https://github.com/rook/rook/issues/4940
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-19 23:34:58 +01:00
Travis Nielsen e8f9cfcb71 exec: always write commands to debug log
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-19 07:49:54 -06:00
Travis Nielsen f2ecaa2bda exec: remove the unused actionName param
The helpers for executing a process have long required an actionName
param which is not being used. Now we remove the old param
while also cleaning up various other usages of the exec
package.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-19 07:49:53 -06:00
Travis Nielsen 84b8cdcf75 exec: simplify the exec package from unused methods and logging
The methods and arguments to the exec methods are not all used anymore.
This cleans up the methods to only what is necessary to improve
the readability and maintainability.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-19 07:49:53 -06:00
Travis Nielsen 607e900f2c ceph: increase timeout for file test pod start
The integration tests have failed intermittently due to needing just
a little longer to start the file test pod. This increases the
wait timeout.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-02-19 13:34:09 -07:00
Travis Nielsen a1db52c5a0 ceph: move integration test to csi driver
The integration tests have been mostly running on the flex driver
with only a newer test on the csi driver. With the CSI driver being
the preferred driver going forward, now the integration tests will
all be running with the CSI driver with the exception of a test
suite that is only dedicated to the flex driver.

A number of other test improvements are also made for code
readability, test stability, and removing unused options.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-02-17 16:24:28 -07:00
Blaine Gardner 3cab1cf4fd Merge pull request #4608 from SUSE/upgrade-test-v1-1-to-v1-3
upgrade v1.0->v1.1->v1.2->v1.3 in upgrade test
2020-01-15 11:36:46 -07:00
Travis Nielsen 2d892b0e88 tests: allow integration tests in minimal config to run on multiple versions
When the tests run in a PR, they can only run against a single version
of K8s by default. If more than five k8s versions are supported in hte
CI, we will need to run some of the suites on multiple versions.
The versions are comma-separated in the list.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-01-13 16:54:49 -07:00
Blaine Gardner c40763c5c6 integration: retry pod logs with kubectl on fail
Sometimes we fail to get logs for pods using this method, notably the
operator pod. It is unknown why this happens. Pod logs are VERY
important, so try again using kubectl.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2020-01-13 08:56:33 -07:00
Blaine Gardner 3cc5bf5191 integration: k8s helper, raise RetryLoop by 50%
To stabilize tests, raise the `RetryLoop` variable 50%.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-12-11 11:27:36 -07:00
Blaine Gardner 20e5639461 ceph: integration:upgrade: long wait for mgr module
Reset the integration test k8s helper's `RetryLoop` to its original
value, and instead only wait an extra long time to allow the mgr module
updates to take a long time after Ceph is updated from Mimic to Nautilus
as part of Ceph's upgrade integration test.

Updating mgr modules can hang for quite a while, which causes the tests
to time out waiting for the OSDs to be updated. Allow this to take a
long time so the tests aren't as flaky.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-12-11 11:27:36 -07:00
Blaine Gardner b9a5717356 integration: raise retry loop count
Raise the retry loop count to stabilize the integration tests.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-11-13 12:23:18 -07:00
Blaine Gardner 79160abd18 ceph: in upgrade test, ensure legacy osds run
During upgrade tests, Rook should verify that it can still run legacy
OSDs. This includes directory-based OSDs, filestore disk OSDs, and
bluestore disk OSDs installed without ceph-volume (i.e., before mimic
v13.2.2) can still be run after upgrade.

This necessitates running the upgrade test twice; once with filestore
and once with bluestore.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-11-13 12:23:17 -07:00
Sébastien Han eccea9fa05 ci: more debug
When we give up on waiting for the pod to be running which describe it
to see what's wrong.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-22 15:14:28 +02:00
travisn f5eb11d863 tests: add logging to track down file test cleanup issue
Adding more logging until we can track down the file-test pod
cleanup issue

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-15 17:13:25 -06:00
Blaine Gardner 70810dd55a ceph: osd: do not init ceph.conf for dir OSDs
Just as with the mon, mgr, mds, rgw, and rbd-mirror daemons, do not
generate a ceph.conf in an init container for directory-based OSDs.
Instead use the mon config database and the commandline to supply all
the needed arguments for running these OSDs.

Also generate a keyring secret to mount to OSD pods. Use this
secret for directory-based OSDs for now with the intention to
use this for other OSDs in the future.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-10-15 12:12:46 -06:00
travisn a92850b307 tests: simplify filesystem tests and client pod deletion
The filesystem test deletes the client test pod by deleting the entire yaml
and sometimes hangs the integration tests. This is the most common intermittent
error in the CI. Now the deletion will happen more directly with a pod delete
command. If it does still fail deletion at least the deletion should timeout
sooner.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:45 -06:00
travisn bd672de7db ceph: allow running minimal test matrix for ceph tests
There are six test suites and six k8s versions where we currently run
the tests. For efficiency we can restrict the testing to one suite
per k8s version. Bigger or riskier changes should still run the full
set of suites on all versions. To trigger the smaller test matrix, add
[test ceph min] to the PR description

Signed-off-by: travisn <tnielsen@redhat.com>
2019-09-18 23:06:49 -06:00
travisn 86968c5064 ceph: boolean settings not applied if set to false and default
THe boolean helm settings were only being applied if their value was true.
If the desired value was false, the value would be skipped in the chart
instead of adding it with the value of false.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-09-09 23:27:35 +03:00
Maksim Nabokikh 179e1a310d ceph: Add dynamic flexvolume expansion
Allow to dynamic resize of rook volumes

Signed-off-by: Maksim Nabokikh <maksim.nabokikh@flant.com>
2019-09-05 02:40:22 +04:00
Travis Nielsen 35adf26d79 Merge pull request #3611 from phlogistonjohn/jjm-support-non-docker-in-tests
tests: do not require actual docker cli
2019-08-13 21:23:48 -06:00
Blaine Gardner 1fce599d4c Merge pull request #3613 from SUSE/integration-use-apply-instead-of-create
Integration tests: use 'kubectl apply' vs create
2019-08-13 16:21:10 -06:00
Blaine Gardner 9009326ad7 Integration tests: use 'kubectl apply' vs create
Some integration tests fail due to resources from a previous integration
run not being cleaned up properly. Instead of using 'kubectl create' --
which fails with an error if the resource already exists -- to create
resources, use 'kubectl apply' -- which does not fail for pre-existing
resources. 'kubectl apply' will give a warning that it should be used to
apply changes to resources created with 'create' or 'apply', but there
is no error.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-08-13 15:03:39 -06:00
John Mulligan 28cb59943b tests: do not require actual docker cli
Other CLI tools, such as podman, and equivalent to docker so support
setting an environment var to use it.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2019-08-13 10:43:10 -04:00
travisn 7e670aa01f tests: collect operator log after each failed test
Signed-off-by: travisn <tnielsen@redhat.com>
2019-08-13 07:23:00 -06:00
travisn be391de73c resolve changes from merge conflict in upgrade changes
The upgrade changes for #2901 added the check for the correct version
of the ceph image before continuing with an upgrade. This commit is
to refactor that change to work with the new code path to validate
the ceph version.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-08-13 07:23:00 -06:00
travisn 1598a048a1 tests: collect all logs for pods in the test namespaces
Now we collect logs for all pods and all their init and main
containers during the integration tests. No longer will we be
missing logs from the integration tests as long as the pods
are available when they are collected at the end of the test.
The pod descriptions are also written to a log file instead
of being included inline with the test output.

Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 6e0bc338f90b3f36a89cf37b542ff4ff873b70bd)
2019-08-02 11:10:46 -06:00
travisn ef9dc0be74 tests: ensure cluster does not exist before starting a test
The CI instances are not always being properly cleaned up
between runs. This is an attempt to get the tests
to ensure a clean install before proceeding with the test.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-07-05 16:41:52 -06:00
Santosh Pillai d6f63166a0 Optionally collect all CI logs using [all logs] flag
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2019-05-03 12:46:18 +05:30
travisn bc87b440fb update to the k8s 1.14 client libraries
Signed-off-by: travisn <tnielsen@redhat.com>
2019-04-18 07:51:41 -06:00
travisn 91e9a5cd04 reference K8s1.13 and client-go 1.10 and the external provisioner for flex
Signed-off-by: travisn <tnielsen@redhat.com>
2019-04-18 07:51:41 -06:00
travisn b8ab35e6c5 tests: collect the operator log prior to a pod restart
Signed-off-by: travisn <tnielsen@redhat.com>
2019-04-17 22:21:53 -06:00
travisn eece96eea4 tests: retry the file mount integration test for stability
Signed-off-by: travisn <tnielsen@redhat.com>
2019-03-12 17:30:06 -06:00
Blaine Gardner 51cc7c2e85 Ceph integration tests: add more verbose logging
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-03-05 14:14:09 -07:00
travisn b24ebc8168 tests: remove unnecessary string returns from kubectl calls
Various kubectl helper methods return strings from calls to
create, delete, or apply resources. There is no need for this.
It is sufficient and complete to check the err from these calls to determine failure.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-03-01 16:16:41 -07:00
travisn 0230400e27 build: remove 1.8 and 1.9 and add 1.13 to integration tests
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-19 23:42:01 -07:00
travisn a29b876337 rename ceph v1 crds with the ceph prefix
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-05 16:23:55 -07:00
travisn 62f7e5d6ea ceph: update docs, code, and tests to use v1 crd types
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-05 14:33:11 -07:00
Alexander Trost ac19f85fbb Allow specifying custom Ceph user and secret name for mounting
Introduce mount security mode for basic multi tenancy

Fixes #2164.

This adds three new parameters/options to StorageClass/flexvolume entry:
* `mountUser`
* `mountSecret`
* `mountSecretNamespace`

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2018-12-04 20:17:15 +01:00
Alexander Trost 5325d4dcec Use a very long testprefix for nodes
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2018-11-23 10:13:53 +01:00
travisn c1f73bb24b tests: collect init container logs
Signed-off-by: travisn <tnielsen@redhat.com>
2018-11-20 15:56:05 -07:00
travisn 06a2461cac cleanup some errors in integration tests
Signed-off-by: travisn <tnielsen@redhat.com>
2018-11-01 11:49:36 -06:00
travisn 7d97b6ad76 osd: use the node hostname labels instead of node names
Signed-off-by: travisn <tnielsen@redhat.com>
2018-10-01 08:09:06 -06:00
travisn 15f6ee5b14 osd: enable filestore on device scenario to run with ceph image
update osd specs to be consistent with other daemons

Signed-off-by: travisn <tnielsen@redhat.com>
2018-09-26 10:36:16 -06:00
travisn 1266c63135 tests: ensure previous crds are deleted before continuing with tests
Signed-off-by: travisn <tnielsen@redhat.com>
2018-09-17 08:10:14 -06:00
Ratnadeep Debnath 7ae2149961 Add support for storageclass' reclaimPolicy when creating PV.
Signed-off-by: Ratnadeep Debnath <rtnpro@gmail.com>
2018-09-14 10:05:34 +05:30
Alexander Trost c1da6999bf Merge pull request #2060 from galexrt/fix_981
build: Only install helm if needed and use a temp path
2018-09-12 19:22:19 +02:00
Alexander Trost 5d8d2b5b5a build: Only install helm if needed and use a temp path
Resolves #981

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2018-09-11 14:21:48 +02:00
travisn f840e3574e mon: start a deployment instead of replicaset
Signed-off-by: travisn <tnielsen@redhat.com>
2018-09-01 00:12:42 -06:00