Commit Graph
1490 Commits
Author SHA1 Message Date
Travis Nielsen 25ac236b5c core: remove support for machine disruption budgets
The machine disruption budgets for handling openshift
machines and machinesets are now removed since they
have been unused and unmaintained since implemented.
This feature is expected to be handled with the more
common Pod Disruption Budgets. A workaround is for the
cluster admin to set up their machine sets so they
match the zone topology. See the original design
doc from the feature here:
https://github.com/rook/rook/blob/master/design/ceph/ceph-openshift-fencing-mitigation.md

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-12-16 11:15:12 -07:00
parth-gr 9319eaa86d ci: watch for pods that restart unexpectedly
will check if the podrestartcount is greater than 1,
If it is we will alert it and fail the CI
It is important to understand intermittent failures
to avoid too many false positives

Closes: https://github.com/rook/rook/issues/11380
Signed-off-by: parth-gr <paarora@redhat.com>
2022-12-14 21:50:15 +05:30
Travis Nielsen b693a5cca5 core: refactor crash collector for more node daemons
The crash collector controller is designed for watching nodes where
ceph daemons are running, and ensuring a special daemon is running
on that node to provide additional support for ceph on that node.
The crash collector is the first example of a daemon that should be
running on all the ceph daemon nodes. The next example of such a
node daemon will be the ceph exporter that will listen for the
ceph metrics as described in the design doc.
https://github.com/rook/rook/blob/master/design/ceph/ceph-exporter.md

Now the crash collector controller is renamed to the node daemon controller
so the ceph exporter daemon can also be managed by the same controller.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-12-09 16:57:05 -07:00
Shinya Hayashi 05875a3f4f osd: support loop devices for test clusters
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.

This feature is intended to be used for testing purposes only.

Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
2022-11-09 06:45:41 +00:00
Blaine Gardner fc56108865 object: allow status endpoints to be null
For the upgrade case, Rook can fail to set status information
on CephObjectStores after CRDs are updated but before the operator is
updated. To fix this, merely allow the slices to be null in the
CephObjectStore's status.endpoints.

This PR seems to be aggravating the helm filesystem upgrade test. Allow
30 more seconds for CRDs to be done before bailing. In debugging, the
filesystem often needed only an extra 3 seconds to succeed.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-11-08 13:36:15 -07:00
Blaine Gardner a777b1d7d1 object: do not create service for external object stores
External CephObjectStores already have endpoints defined by
spec.gateway.externalRgwEndpoints, and if the external store is
configured with TLS (HTTPS), the store's certificates will likely not
accept connections intended for the Service endpoint Rook creates. Some
users might not be able to easily add the service endpoint to their
certificates. Therefore, don't even bother creating a Service for
external clusters.

This does introduce a few issues. The Service seems to have been
initially created to allow multiple external RGW endpoints to be
addressable via a single address in Rook. For all connections to an
external CephObjectStore with multiple endpoints, simply choose an
endpoint at random. Random selection will prevent Rook from failing to
create buckets or users on an external store if one of the external
store's endpoints fails.

The latest OBC library (lib-bucket-provisioner) allows updating the
endpoints on ObjectBuckets after they are created. This allows Rook
users to change endpoints on external CephObjectStores without breaking
all existing OBCs. It requires implementation of the new GetUserID()
library call, requires updating Provision() and Grant() calls to be
idempotent, and it requires removing the Update() call.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-11-04 17:30:30 -06:00
Blaine Gardner 0937eaee38 Merge pull request #11124 from BlaineEXE/object-revise-health-check
object: remove health checker
2022-10-24 11:31:23 -06:00
Travis Nielsen d695a0f930 Merge pull request #11163 from gauravsitlani/gdb-coredump-tests
core: enabling logCollector by default for coredump collection
2022-10-19 12:34:54 -06:00
gauravsitlani cc02399c4c core: enabling logCollector by default for coredump
enabling logCollector by default will enable the coredump
generation in case a process terminates with a segmentation fault.

Signed-off-by: gauravsitlani <gauravsitlani@riseup.net>
2022-10-19 23:04:22 +05:30
Blaine Gardner a7c0c7ee93 object: remove health checker
Remove the health checker for CephObjectStore. The liveness and
readiness probes go through the same code paths in RGW as creating
buckets without as much affect on the storage backend.

Full discussion: https://github.com/rook/rook/issues/11031

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-10-18 13:41:46 -06:00
Rakshith R 914e01e77d ci: add nfs clone e2e testcase
Signed-off-by: Rakshith R <rar@redhat.com>
2022-10-18 15:07:01 +05:30
Rakshith R db56ba22d2 ci: add nfs snap & restore e2e testcases
Signed-off-by: Rakshith R <rar@redhat.com>
2022-10-18 15:06:58 +05:30
Blaine Gardner a809de2574 Merge pull request #10510 from Rakshith-R/fix-nfs-ci
ci: enable and fixes for nfs ci
2022-10-06 12:03:24 -06:00
Blaine Gardner 9d649ca3b6 Merge pull request #11110 from BlaineEXE/fix-nfs-multus
nfs: fix nfs grace period when multus is enabled
2022-10-06 11:24:47 -06:00
Rakshith R be10dac98c ci: enable and fixes for nfs ci
This commit anabled nfs csi ci and
add fixes/improvements to it like the
following:
- verify deletion of cephnfs and .nfs pool before proceeding
- verify pv deletion
- do not enable rook module
- reduce activeCount to 1 to save resources
- run cephnfs ci before cephfs ci since it cephfs
  ci is more resource intensive.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-10-06 11:53:46 +00:00
Travis Nielsen 25ee33f029 ci: ceph master images renamed to main
The ceph master images were recently renamed to main
so we need to pick up the new tag.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-10-05 16:46:04 -06:00
Blaine Gardner 51fac0a993 nfs: fix nfs grace period when multus is enabled
Run the ganesha-rados-grace command in a remote pod when multus
networking is enabled.

Signed-off-by: parth-gr <paarora@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-10-05 16:28:36 -06:00
subhamkrai cc65134d55 ci: create vault secret manually for k8s 1.24
The service account generated a secret that is required for
configuration automatically in Kubernetes 1.23. In Kubernetes
1.24+, you need to create the secret explicitly.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-09-30 18:36:33 +05:30
Satoru Takeuchi 5c2d325cc4 ci: move to ubuntu 2004 completely
github-hosted runner ubuntu-18.04 will be removed by Dec. 1st, 2022.

https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/

So, let's change the all runnners to ubuntu-20.04.

We need to uninstall `snapd` because `snapd` prevents kernel from
unmounting `/mnt` completely.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-09-27 04:32:20 +00:00
Blaine Gardner 2965373ea9 Merge pull request #10898 from parth-gr/test-nfs
nfs: fix nfs if multus is enabled
2022-09-22 12:27:47 -06:00
parth-gr ddea97ebde nfs: fix nfs if multus is enabled
Closes: https://github.com/rook/rook/issues/10812
Signed-off-by: parth-gr <paarora@redhat.com>
2022-09-21 14:09:40 +05:30
Liang Zheng 9a78f6af07 osd: update ceph status parse
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2022-09-20 12:48:53 +08:00
Liang Zheng f9c360e609 osd: optimize device probe
1. Eliminate possible memory leaks of timer.
2. Eliminate duplicated events between udev events and kernel events.
3. Empty struct have the lowest size.

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2022-09-20 10:41:51 +08:00
Travis Nielsen c6f45545bd Merge pull request #10908 from cybozu-go/ci-print-unknown-reason-of-intermittent-build-failure
ci: improve the log on intermittent build failure
2022-09-07 10:31:09 -06:00
Satoru Takeuchi 47f7aefa20 ci: improve the log on intermittent build failure
The build process sometimes fails with intermittent problems. Some of them
are known problems and then we retry build process. However, there still
are unknown problems. In this case, it's hard to find the reason because
the build process exits immediately.

ref.
https://github.com/rook/rook/runs/8225144002?check_suite_focus=true#step:3:926

```
+ case "$o" in
+ exit 1
Error: Process completed with exit code 1.
```

To make debugging easier, let's print the output of `make`. This log won't be
too long since `make` prints most messages to stderr.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-09-07 10:48:21 +00:00
yati1998 9b4361b379 rbdmirror: remove volume replication sidecar
The volume replication operator is being moved to
kubernetes-csi-addons. This commit hence, removes
the volume replication sidecar and updates the
related documentation.
It will also update the csi-addons sidecar version
to the latest one.

Closes: #10655

Signed-off-by: yati1998 <ypadia@redhat.com>
2022-09-07 14:49:19 +05:30
Travis Nielsen 1bbf8f7a43 Merge pull request #10791 from Rakshith-R/use-cephcsi-nfsnodeplugin
csi: use cephcsi image for nfs nodeserver + holder design
2022-09-06 10:52:30 -06:00
Travis Nielsen 5c478e5e04 test: wait for resource deletion during helm uninstall
The helm integration test was immediately removing both the cluster
and the operator charts, without waiting for any resource deletion
between the deletions. This causes intermittent failures in the
test since the cluster CR and mon secret and configmap to never
be deleted because the operator will be stopped before their
finalizers can be removed. Now we wait for their finalizers
to be removed before removing the operator chart.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-09-02 09:37:47 -06:00
Rakshith R 07aac106df ci: add e2e for csi-nfsplugin restart
This commit adds e2e for csi-nfsplugin restart
when it is not on host-networking.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-09-02 14:16:15 +05:30
Travis Nielsen a6dfb77827 build: update min version to k8s 1.19
Rook will support the most recent seven K8s releases,
so we update the min version to 1.19 for Rook v1.10.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-30 15:15:20 -06:00
Travis Nielsen 4028b50de5 test: remove obsolete gateway type setting
Long ago the gateway type for s3 was removed from the object
store CRD, and k8s 1.25 is now complaining about the obsolete
setting still included in the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-29 16:44:09 -06:00
Travis Nielsen 0cd9361b3e test: upgrade from v1.9.10 to pick up psp changes
K8s 1.25 is not capable of creating PSPs, so we need to pick
up v1.9.10 as the base for the upgrade test before upgrading
to master.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-29 15:50:42 -06:00
Travis Nielsen 44d62aaaaf Merge pull request #10699 from subhamkrai/upgrade-to-ubuntu22.04
ci: ubuntu 18.04 is deprecated upgrade to ubuntu 20.04
2022-08-29 09:51:25 -06:00
Travis Nielsen 3696a560d8 build: test upgrade from 1.9 to master
With the pending release of 1.10, the upgrade tests
in master should be testing the upgrade from 1.9
to the master branch.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-26 10:04:37 -06:00
Travis Nielsen fb13df1d33 Merge pull request #10784 from subhamkrai/update-log-collector-script
test: update logcollector to get secrets -oyaml
2022-08-26 09:39:21 -06:00
subhamkrai d1e3023439 ci: collect operator logs in debug mode
collect operator logs in debug mode for both
canary tests and integration tests.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-26 19:32:59 +05:30
subhamkrai aec9ac3060 ci: ubuntu 18.04 is deprecated upgrade to ubuntu 22.04
Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-26 15:19:26 +05:30
subhamkrai 0ae55eb472 ci: add right indentation for ci yaml and sh files
these indenation where added by vscode extentions.
for yaml I have extention ` YAML` by Red Hat
for bash I have `shell-format` and `shellCheck`.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-26 08:37:40 +05:30
Blaine Gardner 1e9bbae583 docs: move PSPs from common.yaml to psp.yaml
Also update docs. Use `make gen-rbac` to generate psp.yaml.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-08-25 16:39:52 -06:00
subhamkrai a9b7894fb4 test: update logcollector to get secrets -oyaml
in case of secrets we need to get `-oyaml` instead
of `describe` since in describe it will be hidden.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-24 21:41:42 +05:30
Travis Nielsen 528fc6a127 Merge pull request #10752 from subhamkrai/fix-links
ci: fix multus depment cr link
2022-08-22 11:29:52 -06:00
subhamkrai 4a05115aef ci: fix multus depment cr link
Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-22 16:58:55 +05:30
Travis Nielsen 1ff27e2fd6 Merge pull request #10716 from koor-tech/fix_tests_log_collection
test: fix test pod log collector
2022-08-17 12:26:35 -06:00
gauravsitlani b806f0f912 docs: fixed the developer guide link in README.md
Signed-off-by: gauravsitlani <gauravsitlani@riseup.net>
2022-08-16 14:30:00 +05:30
Alexander Trost 49440e8b20 test: fix test pod log collector
When a test has a slash in it's name the log collection can fail due to
the "directory" not existing, this makes sure the slashes are replaced
by underscores.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2022-08-12 23:44:13 +02:00
Blaine Gardner b98efbb320 nfs: add support for running SSSD as a sidecar
Implement the SSSD sidecar design for NFS.

Because NFS documents are getting long, create an NFS
`Storage-Configuration` section with separated topics to keep the `NFS
Overview` document sane.

Adjust the SSSD design to allow any VolumeSource, not just ConfigMaps.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-08-11 14:59:23 -06:00
Travis Nielsen a596471757 test: pass test suite by value
The latest go modules come with a sync mutex that requires
us to pass the test suite by reference for proper use
of the mutex.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-11 12:31:49 -06:00
Travis Nielsen 13d527b590 test: increase timeout waiting for file deletion status
The file deletion status has been intermittently failing for several
months. Investigation shows that the timeout is just missing by
a few seconds. Now we increase the timeout from 15 to 45 seconds
to stabilize the test.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-08-02 13:14:05 -06:00
subhamkrai c4742278d1 Revert "ci: use ceph v17.2.1 instead of latest v17"
This reverts commit 3279cb91870e59eccabb9b5eff903d71beb31594.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-08-01 21:05:48 +05:30
subhamkrai 4501cc25c6 ci: use ceph v17.2.1 instead of latest v17
there could be something breaking with latest ceph.
So, let's stick to v17.2.1.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-07-25 20:14:52 +05:30