Commit Graph
1162 Commits
Author SHA1 Message Date
Travis Nielsen 5c478e5e04 test: wait for resource deletion during helm uninstall
The helm integration test was immediately removing both the cluster
and the operator charts, without waiting for any resource deletion
between the deletions. This causes intermittent failures in the
test since the cluster CR and mon secret and configmap to never
be deleted because the operator will be stopped before their
finalizers can be removed. Now we wait for their finalizers
to be removed before removing the operator chart.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-09-02 09:37:47 -06:00
Travis Nielsen a6dfb77827 build: update min version to k8s 1.19
Rook will support the most recent seven K8s releases,
so we update the min version to 1.19 for Rook v1.10.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-30 15:15:20 -06:00
Travis Nielsen 4028b50de5 test: remove obsolete gateway type setting
Long ago the gateway type for s3 was removed from the object
store CRD, and k8s 1.25 is now complaining about the obsolete
setting still included in the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-29 16:44:09 -06:00
Travis Nielsen 0cd9361b3e test: upgrade from v1.9.10 to pick up psp changes
K8s 1.25 is not capable of creating PSPs, so we need to pick
up v1.9.10 as the base for the upgrade test before upgrading
to master.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-29 15:50:42 -06:00
Travis Nielsen 44d62aaaaf Merge pull request #10699 from subhamkrai/upgrade-to-ubuntu22.04
ci: ubuntu 18.04 is deprecated upgrade to ubuntu 20.04
2022-08-29 09:51:25 -06:00
Travis Nielsen 3696a560d8 build: test upgrade from 1.9 to master
With the pending release of 1.10, the upgrade tests
in master should be testing the upgrade from 1.9
to the master branch.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-26 10:04:37 -06:00
Travis Nielsen fb13df1d33 Merge pull request #10784 from subhamkrai/update-log-collector-script
test: update logcollector to get secrets -oyaml
2022-08-26 09:39:21 -06:00
subhamkrai d1e3023439 ci: collect operator logs in debug mode
collect operator logs in debug mode for both
canary tests and integration tests.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-26 19:32:59 +05:30
subhamkrai aec9ac3060 ci: ubuntu 18.04 is deprecated upgrade to ubuntu 22.04
Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-26 15:19:26 +05:30
subhamkrai 0ae55eb472 ci: add right indentation for ci yaml and sh files
these indenation where added by vscode extentions.
for yaml I have extention ` YAML` by Red Hat
for bash I have `shell-format` and `shellCheck`.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-26 08:37:40 +05:30
Blaine Gardner 1e9bbae583 docs: move PSPs from common.yaml to psp.yaml
Also update docs. Use `make gen-rbac` to generate psp.yaml.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-08-25 16:39:52 -06:00
subhamkrai a9b7894fb4 test: update logcollector to get secrets -oyaml
in case of secrets we need to get `-oyaml` instead
of `describe` since in describe it will be hidden.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-24 21:41:42 +05:30
Travis Nielsen 528fc6a127 Merge pull request #10752 from subhamkrai/fix-links
ci: fix multus depment cr link
2022-08-22 11:29:52 -06:00
subhamkrai 4a05115aef ci: fix multus depment cr link
Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-22 16:58:55 +05:30
Travis Nielsen 1ff27e2fd6 Merge pull request #10716 from koor-tech/fix_tests_log_collection
test: fix test pod log collector
2022-08-17 12:26:35 -06:00
gauravsitlani b806f0f912 docs: fixed the developer guide link in README.md
Signed-off-by: gauravsitlani <gauravsitlani@riseup.net>
2022-08-16 14:30:00 +05:30
Alexander Trost 49440e8b20 test: fix test pod log collector
When a test has a slash in it's name the log collection can fail due to
the "directory" not existing, this makes sure the slashes are replaced
by underscores.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2022-08-12 23:44:13 +02:00
Blaine Gardner b98efbb320 nfs: add support for running SSSD as a sidecar
Implement the SSSD sidecar design for NFS.

Because NFS documents are getting long, create an NFS
`Storage-Configuration` section with separated topics to keep the `NFS
Overview` document sane.

Adjust the SSSD design to allow any VolumeSource, not just ConfigMaps.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-08-11 14:59:23 -06:00
Travis Nielsen a596471757 test: pass test suite by value
The latest go modules come with a sync mutex that requires
us to pass the test suite by reference for proper use
of the mutex.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-11 12:31:49 -06:00
Travis Nielsen 13d527b590 test: increase timeout waiting for file deletion status
The file deletion status has been intermittently failing for several
months. Investigation shows that the timeout is just missing by
a few seconds. Now we increase the timeout from 15 to 45 seconds
to stabilize the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-02 13:14:05 -06:00
subhamkrai c4742278d1 Revert "ci: use ceph v17.2.1 instead of latest v17"
This reverts commit 3279cb91870e59eccabb9b5eff903d71beb31594.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-01 21:05:48 +05:30
subhamkrai 4501cc25c6 ci: use ceph v17.2.1 instead of latest v17
there could be something breaking with latest ceph.
So, let's stick to v17.2.1.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-07-25 20:14:52 +05:30
Travis Nielsen fc541764f1 Merge pull request #10484 from jsoref/spelling
core: fix spelling
2022-07-15 08:31:41 -06:00
Travis Nielsen 53955790f3 ci: pin the whereabouts version to v0.5.3
The whereabouts manifests in the master branch
have moved around, so until the new approach
is investigated we pin to the latest release
version v0.5.3

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-07-13 10:37:15 -06:00
Satoru Takeuchi 7e571f6114 osd: support OSD on logical volume in host-based cluster
Rook supports raw mode OSD in host-based cluster. So we can also
support OSD on logical volume in this kind of cluster.

Logical volumes aren't picked by filters (i.e. `useAllDevices: true`
and `device{Path,}Filter` to avoid unwanted LV consumption on upgrade.

Closes: https://github.com/rook/rook/issues/2047

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-07-11 21:08:50 +00:00
Josh Soref 6e7b8767f3 core: fix spelling
* another
* are
* availability
* available
* bootstrap
* boundaries
* ceph
* certificate
* class
* codifies
* consuming
* corrupted
* createor
* csi
* deployments
* exceeded
* execute
* filesystem
* healthiness
* heuristics
* immediately
* insecure
* installed
* isolated
* maintained
* maximum
* minute
* monitor
* new
* nginx
* nonexistent
* not
* occurs
* omitempty
* operator
* orchestration
* persistentvolumes
* placement
* preexisting
* prometheus
* protecting
* provisioner
* purposes
* reconcile
* regex
* related
* requests
* returns
* rubbish
* running
* schedulable
* schedule
* serviceaccount
* simulating
* snapshots
* statement
* static
* tenants
* the
* unavailable
* volumeattachment
* waiting
* with
* wrapper
* zonegroup

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-07-07 18:10:47 -04:00
Travis Nielsen dad97f3425 core: remove support for ceph octopus
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-07-07 15:03:26 -06:00
Rakshith R 034c4756d3 ci: make sure PVC is deleted before proceeding for file test
Signed-off-by: Rakshith R <rar@redhat.com>
2022-07-04 11:36:37 +05:30
Rakshith R 16d34e1425 ci: skip nfs csi e2e to avoid flaky e2e
Skip nfs csi e2e until https://github.com/rook/rook/issues/10518
is resolved to avoid flakiness.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-07-04 11:36:37 +05:30
Travis Nielsen e30a068e68 test: skip cluster cleanup in upgrade test
The cluster cleanup is tested in all of the test suites. Now that the
correct manifest test installer is being used for cleanup, the
upgrade tests are failing to delete the namespace due to some
finalizer that is holding on for some resource that wasn't cleaned
up properly. Let's just disable the cleanup for these tests since
it's tested in other test suites and not worth the time to
troubleshoot every resource getting cleaned up.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-06-30 16:19:04 -06:00
Travis Nielsen e86b2f8b02 test: upgrade test to use correct manifests for uninstall
The uninstall after the upgrade test was attempting to read
the crds from github, instead of the local repo as expected.
Now the correct manifests interface is updated during
the upgrade so it will be used during the uninstall.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-06-30 15:33:31 -06:00
Travis Nielsen 8e65d69c7a Merge pull request #10495 from Rakshith-R/nfs-ci-cleanup
ci: rename EnableCSINFS to TestNFSCSI & explain "nolock" mountOption
2022-06-29 07:59:40 -06:00
Markus Schmidleitner 00689476df test: delete object sc before upgrade tests (#10153)
Signed-off-by: Markus Schmidleitner <markus.schmidleitner@ocilion.com>
2022-06-28 10:39:58 +02:00
Rakshith R 621dd2a159 ci: explain why "nolock" mountOption is needed for nfs
"nolock" mountOptions has been added to prevent the following error in ci:
Output: mount.nfs: rpc.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-06-23 16:53:08 +05:30
Rakshith R 928dd46593 ci: rename TestCephSettings.EnableCSINFS to TestNFSCSI
Signed-off-by: Rakshith R <rar@redhat.com>
2022-06-23 16:48:13 +05:30
Rakshith R 5b017cfd67 ci: add tests for nfs csi pvc
This commit adds nfs csi pvc test into
ceph smoke suite.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-06-20 15:01:22 +05:30
Blaine Gardner 800d3e5050 Merge pull request #9915 from BlaineEXE/dependents-cephfilesystem
file: block deletion on more dependents
2022-06-17 14:07:07 -06:00
Blaine Gardner a63844f8bf file: block deletion on more dependents
Block deletion of CephFilesystems when there are any raw Ceph
subvolumegroups present that have subvolumes in them. Empty
subvolumegroups will not block deletion.

One important subvolume group is "csi" which is the default location
where CSI subvolumes are kept. If this group is empty, it means that
there are no PVCs created based on the CephFilesystem in question. This
also holds true if there are external consumers of the filesystem in
external cluster mode.

Similarly, if there are any subvolumegroups (for example "_nogroup",
which includes subvolumes in the filesystem root) that contain
manually-created subvolumes, Rook will also see this and block deletion.
This comes into play currently with manually-created NFS exports.

A work-in progress aims to create a Ceph-CSI NFS export provisioner
which will likely create subvolumes in the "csi" group as well. This
implementation will catch this case also.

Rook still checks for CephFilesystemSubVolumeGroups explicitly in
addition to the check added here. This is to ensure that even empty
groups will block deletion if they are created via this CR type.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-06-14 13:02:35 -06:00
Satoru Takeuchi 6b0f539f61 test: add canary test for encrypted osd in host-based clusters
It's better to test the creation of encrypted devices in host-based clusters.
This test will reduce the potential risks of regression when modifying
the osd-creation code.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-06-08 22:34:01 +00:00
Travis Nielsen 1acc0621af Merge pull request #10342 from Madhu-1/fix-csi-pod-network
ceph: Add holder pod if csi host networking is disabled
2022-06-07 10:16:51 -06:00
Travis Nielsen 6200195275 Merge pull request #10376 from subhamkrai/webhokk-e2e
test: add e2e test for webhook validation
2022-06-07 09:40:59 -06:00
Travis Nielsen 10efd34fd5 Merge pull request #10306 from Madhu-1/update-snap-v6
csi: update sidecar to latest release
2022-06-07 08:41:13 -06:00
Madhu Rajanna 767cfcbf09 csi: update sidecar to latest release
updating node-driver-registrar to v2.5.1 release
, nfsplugin to 4.0.0 and csi-snapshotter to v6.0.1

Note:- with snapshotter v6.0.1 v1beta1 snapshot CRD
is not supported. If you want betav1 snapshot support
please switch to v5.x.x snapshotter.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-07 19:27:13 +05:30
subhamkrai 4d92c2770a test: add e2e test for webhook validation
adding e2e test for webhook validation to test invalid
cluster, pool,objectStore, radosNamespace, subVolumeGroup CR
that will each be rejected by their admission controller.

Closes: https://github.com/rook/rook/issues/10024
Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-07 14:01:02 +05:30
subhamkrai d5bac59a74 ci: use python formatting tool black
formatting other python files using black

Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-07 13:49:14 +05:30
Madhu Rajanna 276005cfd5 csi: add holder pod if csi hostnetworking is disabled
If csi is configured not the use the
hostnetworking, deploy the holder
pod for executing the commands with nsenter.
The implementation is same as multus.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-07 13:09:12 +05:30
Satoru Takeuchi 5b3633bfd6 test: add canary integration test for osd with metadata device
"metadataDevice" field in host based cluster is not tested yet.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-05-25 01:36:33 +00:00
Satoru Takeuchi a9465efa07 Merge pull request #9931 from cybozu-go/test-add-canary-integration-tests-for-osd-on-device
test: add canary integration tests for osd on device
2022-05-06 05:25:57 +09:00
Satoru Takeuchi 805163e2e2 test: add canary integration tests for osd on device
OSD on PVC is tested in many patterns but OSD on device is not.
It's preferable to add the following patterns for OSD on device.

- An OSD on a raw disk
- Two OSDs on a device

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-05-05 13:15:18 +00:00
Satoru Takeuchi ab38c10c2d test: avoid to create a corrupted GPT headers
The following steps in CI scripts create corrupted GPT headers.

```console
$ sudo sgdisk --zap-all --clear --mbrtogpt -g -- "$DISK"
$ sudo dd if=/dev/zero of="$DISK" bs=1M count=10
```

It results in the failure of the succeeding `sgdisk --print "$DISK".

Here is an example.

```console
$ sudo parted /dev/sdb mklabel msdos
...
$ sudo sgdisk --zap-all --clear --mbrtogpt -g -- /dev/sdb
...
$ sudo dd if=/dev/zero of=/dev/sdb bs=1M count=10
...
$ sudo sgdisk --print /dev/sdb
Caution: invalid main GPT header, but valid backup; regenerating main header
from backup!

Warning: Invalid CRC on main header data; loaded backup partition table.
Warning! One or more CRCs don't match. You should repair the disk!
Main header: ERROR
Backup header: OK
Main partition table: OK
Backup partition table: OK

Invalid partition data!
$ echo $?
2
```

We can safely use the simple `sgdisk --zap-all "$DEVICE"` here.
It's not necessary to convert "mbr" to "gpt" because we'll make new GPT
labels just after this command.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-05-05 13:11:21 +00:00