This commits removes controller-runtime dependencies
from the apis dir and to achieve that we are removing
webhook.
Signed-off-by: subhamkrai <srai@redhat.com>
This commit anabled nfs csi ci and
add fixes/improvements to it like the
following:
- verify deletion of cephnfs and .nfs pool before proceeding
- verify pv deletion
- do not enable rook module
- reduce activeCount to 1 to save resources
- run cephnfs ci before cephfs ci since it cephfs
ci is more resource intensive.
Signed-off-by: Rakshith R <rar@redhat.com>
The latest go modules come with a sync mutex that requires
us to pass the test suite by reference for proper use
of the mutex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
adding e2e test for webhook validation to test invalid
cluster, pool,objectStore, radosNamespace, subVolumeGroup CR
that will each be rejected by their admission controller.
Closes: https://github.com/rook/rook/issues/10024
Signed-off-by: subhamkrai <srai@redhat.com>
Check the radosgw-admin realm user list per object store instead of relying
on the ceph dashboard get-rgw-api- command.
Resolves#9099
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The mon failover test was failing if the cluster reconcile
was being triggered in the middle of the failover and if
the ordering of the mons happened to attempt to start the
mons that were not failed over. The other mons couldn't be updated
due to the upgrade checks, so the test would timeout waiting
for the mons that would never update in time.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
msgr2 allows for encryption and/or compression across the wire.
Settings for enabling the encryption and compression are now
available in the cluster CR to that ceph will be automatically
configured with these settings when desired.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The test suites were triggered by Jenkins potentially
using different versions of K8s and environment variables.
There is no longer a need for these environment vars to
determine which test suites should be run. The test
suites to run can just be triggered directly.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The generation of a long node name in the integration tests was
being done based on the k8s version. In the past, older K8s versions
did not support the changing name. Now it's more maintainable if
we generate the long name depending on the test suite.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The cluster info is important context for the cluster controller to
create the cluster, and all the fields must be properly set.
A test cluster name was being set temporarily, resulting in
mons incorrectly getting the wrong cluster CR name. There is no
known issue from the temporary value, it was just exposed by
https://github.com/rook/rook/pull/8678 setting the value to a label.
Now the functions are more clearly named so only unit and
integration tests should be using the test value for the cluster
name where it is not important.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The volume replication CRDs are an external component, not owned by Rook.
Therefore, they should be installed as any other independent component
in case the admin will install other consumers of the volumereplication CRDs
in the future in addition to Rook and the CSI driver.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In general, make the output from the object store test easier to follow
and debug.
- Make sure assertions are associated with the appropriate subtest.
- Remove `t.Helper()` from complex helpers.
- Add info to log messages.
- Move long parts of (sub-)test names to log messages.
- Move some reused functions back to ceph_base_object_test.go
- Refactor runObjectE2ETestLite() to be suitable for use as the second
store created in the "run a second object store" test.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
We have new jobs now:
* one that runs both smoke and object on the next Pacific version
* one that runs both smoke and object on Ceph master
* one that tests the upgrade from the current pacific stable to the
pacific devel
* one that tests the upgrade from the current octopus stable to the
octopus devel
Signed-off-by: Sébastien Han <seb@redhat.com>
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The integration tests must always be run against the local
build of rook, and an image should never be pulled from dockerhub.
To prevent pulling a release or master tag, the local build
will use a tag specific to the build and not ever published
elsewhere.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit a8a40428b0)
The `*SuiteMinimalTestVersion` vars in
`tests/integration/ceph_base_deploy_test.go` are no longer needed with
Jenkins no longer being used. Remove them.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The CephSmokeSuite is becoming quite large and long, and most of the
length is now related to the object e2e tests. Separate the full object
e2e test into CephObjectSuite, and only test the object 'lite' test in
the CephSmokeSuite.
Resolves#8714
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Similarly to block volume replication, Ceph is capable of replicating the
content of a Ceph Filesystem from one cluster to another.
For this, during the 1.6 cycle, we introduced a new CRD called
CephFilesystemMirror which effectively deploys a cephfs-mirror daemon.
However, configuring peers to enable replication between two clusters
had to be done manually.
Also various bug fix made it in Ceph eventually and the minimum required
version for this to work is to run on Ceph Pacific 16.2.5 at least.
So the automatic configuration of Ceph Filesystem peers is now possible.
By editing the CephFilesystem CRD, you can now turn on mirroring:
```yaml
mirroring:
enabled: false
# list of Kubernetes Secrets containing the peer token
# for more details see: https://docs.ceph.com/en/latest/dev/cephfs-mirroring/#bootstrap-peers
peers:
secretNames:
- secondary-cluster-peer
```
Also, the mirroring status is displayed in the CR status:
```
status:
info:
fsMirrorBootstrapPeerSecretName: fs-peer-token-myfs
mirroringStatus:
daemonsStatus:
- daemon_id: 4186
filesystems:
- filesystem_id: 2
name: myfs
lastChecked: "2021-07-01T14:16:29Z"
phase: Ready
snapshotScheduleStatus:
lastChecked: "2021-07-01T14:16:29Z"
snapshotSchedules:
- fs: myfs
path: /
rel_path: /
retention: {}
schedule: 24h
```
Closes: https://github.com/rook/rook/issues/7063
Signed-off-by: Sébastien Han <seb@redhat.com>
With the Ceph Pacific release coming this week we add support
in Rook for Pacific with the Rook v1.6 release coming soon.
The integration tests will now run across nautilus, octopus,
and pacific to cover all supported Ceph versions. The default
examples still specify Octopus until there is more bake time
for Pacific.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The integration tests run independently in the github actions so there is
no need to cleanup from every test. The cleanup is still needed in the
Jenkins tests where all the suites run serially.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This reverts commit 114a949cfb since
14.2.14 is out and has a fix to use raw mode instead of LVM mode on
partitions.
Signed-off-by: Sébastien Han <seb@redhat.com>
The latest ceph-volume does not support creating OSDs on partitions.
The github actions only have a partition available, so we will run
the github tests on v14.2.12 and v15.2.7 that still support partitions,
while the Jenkins environment has a raw device available where we
can run the latest versions of Ceph in the tests.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Due to #6492, preservePoolsOnDelete is not useful at all for CephFS;
after the filesystem is deleted, the leftover pools cannot be
reassocaited with a newly created filesystem without wiping all
metadata. The only way we can actually preserve data is keeping around
the entire filesystem.
This commit implements a `preserveFilesystemOnDelete` option which work
similar to the existing pool preservation option but instead keeps the
whole CephFS while taking it down and removing all MDSes.
This commit also changes all documentation to refer to this new option
with the intent of essentially deprecating `preservePoolsOnDelete`. IMO,
keeping around `preservePoolsOnDelete` is actively harmful because it
lulls users into thinking their data will be safe but, in reality,
recovering from this situation is highly complex and has large potential
for data loss.
Signed-off-by: Lalit Maganti <lalitm@google.com>
The smoke suite relies on the csi driver for all the file and block
tests. In the release branch on k8s 1.11 we only need to run the
flex suite and can skip the smoke suite.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit handle golangci-lint linter errcheck.
`errcheck` - Errcheck is a program for checking for
unchecked errors in go programs. These unchecked errors
can be critical bugs in some cases
To see only staticcheck linter output
`golangci-lint run --disable-all -E errcheck`
Signed-off-by: subhamkrai <srai@redhat.com>
this commit will enable one more linter ineffassign
in golangci-lint.
This linter throws an error when variable is assigned and never used.
`golangci-lint run --disable-all -E ineffassign` is used detects ineffassign
errors only.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
this commit enables the rook integration
testing capability on openshift. currently
this commit enable CephSmokeSuite testing
only.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
Rook is now capable of configuring mirroring between sites. The
implementation works at different levels:
* CephBlockPool: which introduces a new `mirroring` configuration as well
as `statusCheck`. When turned on, Rook will enable mirroring on the
pool. It will also create a bootstrap peer token and store it in a
Kubernetes Secret. The name of that Secret can be found in the Status
field of the CephBlockPool CRD. This token can be fetched and used by
other clusters to configure the site as a peer. Mirroring can be
configured either at the pool or the image level.
* CephRBDMirror: which introduces a new `peers` configuration allowing
Rook to connect to peers by passing a Secret name. The administrator will
create a Kubernetes Secret with 2 keys: 'token' for the bootstrap peer
token and 'pool' for the name of pool. Once detected the rbd-mirror
controller will go ahead and import the peer configuration.
Pool mirroring status example:
```
status:
info:
rbdMirrorBootstrapPeerSecretName: pool-peer-token-test
mirroringInfo:
lastChanged: "2020-09-17T14:47:27Z"
lastChecked: "2020-09-17T14:48:27Z"
summary:
summary:
mode: image
peers:
- client_name: client.rbd-mirror-peer
direction: rx-tx
mirror_uuid: ""
site_name: rhcs
uuid: c50522a4-28a4-4bd3-ba68-e11780308882
site_name: 91eae0dd-06b1-4d2c-91f3-1311c9df382b-rook-ceph
mirroringStatus:
lastChecked: "2020-09-17T14:48:27Z"
summary:
summary:
daemon_health: OK
health: OK
image_health: OK
states:
replaying: 1
```
Signed-off-by: Sébastien Han <seb@redhat.com>
Currently, SmokeTest is not cleaned up after the test.
As this can affect other following tests, set rookCephCleanup as true.
Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In Octopus 15.2.2 there is an rbd change that causes the rbd ls command
to hang if there are not sufficient OSDs to meet the replica requirement.
The smoke suite just started hanging today since the 15.2.2 image was
released with this change. Therefore, the test that resizes the pool
to replica 3 must resize back to 1 before cleaning up.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Ceph orchestrator device ls
Ceph orchestrator status
Ceph orchestrator host ls
Ceph orchestrator create OSD
Ceph orchestrator ls
[test ceph]
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Refactor the StartTestCluster method to make it more readable.
I have put most of the parameters used in this function in the struct.
In this way the call to this function is more easy to understand and to use.
The addition of new parameters to this method started to be annoying.
[test ceph]
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
Previously, the rbd-mirror daemon was integrated into the `CephCluster`
CRD. This wasn't really practical since we would have to wait for the
whole orchestration to be done to actually set it up. The same goes for
any CR update. Let's say you want to change the number of daemons, Rook
would go through mons, mgrs and osds until it get to rbd-mirror.
This triggers an undesired full orchestration.
With its own CRD this component just gains a lot more flexibility.
Closes: https://github.com/rook/rook/issues/5084
Signed-off-by: Sébastien Han <seb@redhat.com>
With the release of Octopus we need to run the integration tests
on Octopus in order to have validation as a supported platform.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In order to ensure proper clean up of all the rook-ceph data when the cluster is deleted, we need to clean up the dataDirHostPath (var/lib/rook)
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
With a finalizer on the pools, the pools were not always being purged
during the integration tests. Now the multicluster suite will ensure
its pool is purged.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The integration tests have been mostly running on the flex driver
with only a newer test on the csi driver. With the CSI driver being
the preferred driver going forward, now the integration tests will
all be running with the CSI driver with the exception of a test
suite that is only dedicated to the flex driver.
A number of other test improvements are also made for code
readability, test stability, and removing unused options.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit enables testing OSDs over PVCs by modifying the
multi-cluster test to use PVC for provisioning OSDs when `manual`
storageClass is present in the cluster.
Signed-off-by: Ashish Ranjan <aranjan@redhat.com>
There are two separate integration tests for file. To simplify
and streamline and potentially eliminate some random failures,
these tests now consume the same file system instead of creating
a separate file system.
Signed-off-by: travisn <tnielsen@redhat.com>
There are six test suites and six k8s versions where we currently run
the tests. For efficiency we can restrict the testing to one suite
per k8s version. Bigger or riskier changes should still run the full
set of suites on all versions. To trigger the smaller test matrix, add
[test ceph min] to the PR description
Signed-off-by: travisn <tnielsen@redhat.com>
subhamkrai
Rakshith R
Travis Nielsen
Alexander Trost
Blaine Gardner
Sébastien Han
bipuladh
Arun Kumar Mohan
Lalit Maganti
subhamkrai
binoue
Jiffin Tony Thottan
Juan Miguel Olmo Martínez
Santosh Pillai
Ashish Ranjan