0 Fork 0

Files

T

Blaine Gardner 52f5119cef multus: allow node profiles in validation test

Add the ability to specify node profiles in the multus validation test.

This addresses a few points of early feedback on the validation tool.
Statements below critique the tool's behavior before this patch.
1. The tool assumes all daemons are on public and cluster network, which
   means users who have a significantly smaller cluster net (a
   design choice) cannot run a single test to determine if Rook is
   likely to install correctly.
2. The tool does not have placement options to select only a subset of
   Kubernetes nodes to run validation on.
3. Users of multus seem to have a dedicated pool of storage nodes more
   often than the average Rook install. This makes sense for security-
   and perforance-minded users. The tool cannot run a single test to
   verify storage-only and general-workload nodes at one time.

These points are addressed by allowing users to specify configurations
for different "NodeTypes."

Each NodeType config has options for selecting the number of OSDs as
well as the number of other (non-OSD) Ceph daemons. This limits the
unnecessary exhaustion of cluster network addresses from critique 1.

Each NodeType config has its own placement (critique 2).

Users can define as many NodeTypes as needed to test the network for
their planned CephCluster. Specifically, this allows the tool to test
storage-only nodes and generalized-workload nodes at the same time. An
arbitrary number of NodeTypes are allowed to support even more highly
specialized cluster setups, such as multiple tiers of storage nodes
where some storage-only nodes may run more OSDs than others.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 0c721e05d5)

2023-10-04 15:46:21 -06:00

framework

docs: fixing some spelling issues

2023-10-03 14:06:59 +00:00

integration

object: create cosi user for each object store

2023-10-04 16:10:52 +00:00

manifests

core: update default image to ceph v17.2.6

2023-04-11 14:07:11 -06:00

scripts

multus: allow node profiles in validation test

2023-10-04 15:46:21 -06:00

.gitignore

tests: refactor integration tests

2017-06-29 15:20:28 -07:00

ceph-status-out

core: change directory layout

2021-11-30 09:12:53 +01:00

README.md

ci: enable tmate when in debug mode

2023-08-28 14:32:18 +00:00

README.md

Integration Tests

The integration tests run end-to-end tests on Rook in a running instance of Kubernetes. The framework includes scripts for starting Kubernetes so users can quickly spin up a Kubernetes cluster. The tests are generally designed to install Rook, run tests, and uninstall Rook.

The CI runs the integration tests with each PR and each master or release branch build. If the tests fail in a PR, access the tmate for debugging.

This document will outline the steps to run the integration tests locally in a minikube environment, should the CI not be sufficient to troubleshoot.

!!! hint The CI is generally much simpler to troubleshoot than running these tests locally. Running the tests locally is rarely necessary.

!!! warning A risk of running the tests locally is that a local disk is required during the tests. If not running in a VM, your laptop or other test machine could be destroyed.

Install Minikube

Follow Rook's developer guide to install Minikube.

Build Rook image

Now that the Kubernetes cluster is running we need to populate the Docker registry to allow local image builds to be easily used inside Minikube.

eval $(minikube docker-env -p minikube)

make build will now build and push the images to the Docker registry inside the Minikube virtual machine.

make build

Tag the newly built images to rook/ceph:local-build for running tests, or rook/ceph:master if creating example manifests::

docker tag $(docker images|awk '/build-/ {print $1}') rook/ceph:local-build
docker tag rook/ceph:local-build rook/ceph:master

Run integration tests

Some settings are available to run the tests under different environments. The settings are all configured with environment variables. See environment.go for the available environment variables.

Set the following variables:

export TEST_HELM_PATH=/tmp/rook-tests-scripts-helm/linux-amd64/helm
export TEST_BASE_DIR=WORKING_DIR
export TEST_SCRATCH_DEVICE=/dev/vdb

Set TEST_SCRATCH_DEVICE to the correct block device name based on the driver that's being used.

!!! hint If using the virtualbox minikube driver, the device should be /dev/sdb

!!! warning The integration tests erase the contents of TEST_SCRATCH_DEVICE when the test is completed

To run a specific suite, specify the suite name:

go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration

After running tests, see test logs under tests/integration/_output.

To run specific tests inside a suite:

go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration -testify.m TestARookClusterInstallation_SmokeTest

!!! info Only the golang test suites are documented to run locally. Canary and other tests have only ever been supported in the CI.

Running tests on OpenShift

Setup OpenShift environment and export KUBECONFIG
Make sure oc executable file is in the PATH.
Only the CephSmokeSuite is currently supported on OpenShift.
Set the following environment variables depending on the environment:

export TEST_ENV_NAME=openshift
export TEST_STORAGE_CLASS=gp2
export TEST_BASE_DIR=/tmp

Run the integration tests