The integration tests run independently in the github actions so there is
no need to cleanup from every test. The cleanup is still needed in the
Jenkins tests where all the suites run serially.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The rbac required for multus was present in role which makes
only the network-attachment-definitions in the rook cluster namespace
accessible to the operator. Moved it to clusterrole for cluster wide
access to NAD.
Signed-off-by: rohan47 <rohgupta@redhat.com>
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:
* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`
will be autogenerated for us.
We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.
It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:
```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```
The corresponding API type markers are:
```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```
For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html
Signed-off-by: Sébastien Han <seb@redhat.com>
This is a follow up to:
https://github.com/rook/rook/pull/7264
which added pool quotas. It was stated that it would be nice to
be able to set pool max_bytes quotas using standard k8s quantity
formats rather than an integer representing the number of bytes.
I was too busy at the time to make the changes. This PR adds the
functionality.
Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
If there are multiple mgrs running, the operator will automatically
add pod antiaffinity across hosts by default. For stretch clusters,
antiaffinity across the stretch failure domain (e.g. zones)
will be added. Required antiaffinity will be created unless the
test setting allowMultiplePerNode is set to true.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The mgr daemon may be failed over by ceph if the active mgr is not
responding and the standby mgr is available. If the active mgr changes
the services for the dashboard and metrics will be updated with a
label selector for the new active mgr. The services cannot direct
traffic to the standby mgr or else they will be incorrectly redirected
to the active mgr.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The GRPC metrics exposed by both the provisioner pod
and the node plugin pod on some port. Provisioner pod
is running on the pod network and the daemonset pods
run on the host network. sometimes starting the GRPC
metrics by default can lead to node plugin pods
crashloopback state this is due to the port conflict.
Moreover, the GRPC metrics are not for the user it's
for the one which will help to debug the time taken
by cephcsi to serve each GRPC call. Enabling it by
default won't be a good idea. So the plan is to
disable it by default, If someone faces any issue it
can be enabled later at some point in time.
closes#7378
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
We now create 3 disks which will give us 2 OSDs and potentially help us
to catch more errors in our scenarios, especially on iterations.
Signed-off-by: Sébastien Han <seb@redhat.com>
The integration tests will now read the manifests from the same source
that is used in the examples for installation. No longer is there a need
for a copy of the CRDs in the integration test code.
The tests will read the manifests directly from disk under a path
relative to the rook root. The upgrade test will read the manifests from
a github url where the base version has been published.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Since the 1.5 releaase we need the integration test to upgrade from
1.5 to master instead of from 1.4 to master.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit removes file `config-admission-controller.sh`
as we don't need this anymore and update the doc to point
`tests/scripts/deploy_admission_controller.sh` to start
admission controller.
Signed-off-by: subhamkrai <srai@redhat.com>
this commit adds new crd to admission webhooks to check
if one of the port or securePort are set to non-zero
value in objectstore crd.
If both are set to 0 in crd we'll see error like
```
Error from server (invalid create: either of port or securePort
fields should be not be zero): error when creating "object-test.yaml":
admission webhook
"cephobjectstore-wh-rook-ceph-admission-controller-rook-ceph.rook.io"
denied the request: invalid create: either of port or securePort fields
should be not be zero.
```
Signed-off-by: subhamkrai <srai@redhat.com>
we are now using cert-manager for certificates
instead of a self-signed certificate for admission
webhooks.
Signed-off-by: subhamkrai <srai@redhat.com>
This reverts commit 114a949cfb since
14.2.14 is out and has a fix to use raw mode instead of LVM mode on
partitions.
Signed-off-by: Sébastien Han <seb@redhat.com>
Do not use old Ceph v15.2.7 in integration that was working around the
issue of ceph-volume partition support being removed in v15.2.8.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
(cherry picked from commit 99cee7648cfaf72b02d62e281a2ac3a18bce4132)
Even though CRs for bucket health check was added, CRDs were missing from
crds.yaml file. So the health check always enabled for RGW.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
The vault env variable VAULT_BACKEND is read by the secret library so we
must not change it. This is reverting
ee6a03dfb8 with a few extra fixes for the
rgw code.
Signed-off-by: Sébastien Han <seb@redhat.com>
codegen is currently broken in master.
this commit fix the broken codegen by fixing
the correct working directory in gh action.
Signed-off-by: subhamkrai <srai@redhat.com>
Remove features and design that supports adding OSDs to Ceph clusters
via `spec:driveGroups`. Update the Ceph upgrade doc that informs users
who currently use Drive Groups (we believe there are none of these
users) how to migrate to using the `spec:storage` config.
Resolves https://github.com/rook/rook/issues/7275
Revert "ceph: fix drive group deployment failure"
This reverts commit 76f1d9944e.
Revert "ceph: osd: add drive groups spec to cluster CR"
This reverts commit 7117fc12b7.
Revert "design: ceph orchestrator module add/remove OSDs"
This reverts commit 178187d035.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Add the cephfilesystemmirrors definition for CI runs where the
Kubernetes version is 1.15 and extension v1 extensions are not
supported.
Closes: https://github.com/rook/rook/issues/7168
Signed-off-by: Sébastien Han <seb@redhat.com>
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
integration test was failing frequently due to some
intermittent failures.
```
Error from server (InternalError): error when creating
"STDIN": Internal error occurred: failed calling webhook.
```
(This error is not coming from us most probably)
This is kind of workaround so we get green build
as we still don't know the root cause.
Signed-off-by: subhamkrai <srai@redhat.com>
The deviceClass was missing from the object store CRD schema,
disallowing the device class from being specified on
the object store pools.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
disable admission controller for helm suite.
v1 version of admissionregistration.k8s.io/v1
require minimum v1.16.0 of k8s. All other suite
was disabled earlier this was left.
Signed-off-by: subhamkrai <srai@redhat.com>
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.
The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050
The same goes for integration tests, they will get added later once we
start testing on Pacific.
Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
The PDBs have been stabilizing for a good length of time now since
they were created in v1.1, and later redesigned in v1.5. The time
has come to enable the feature by default so everyone can enjoy
the stable Rook storage even while draining K8s nodes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Sébastien Han
Travis Nielsen
rohan47
Satoru Takeuchi
Frank Ritchie
Madhu Rajanna
subhamkrai
Blaine Gardner
Santosh Pillai