my-rook-config/tests

Rook Test Framework

The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of Kubernetes. The framework also provides scripts for starting Kubernetes so users can quickly spin up a Kubernetes cluster. The Test framework is designed to install Rook, run tests, and uninstall Rook.

Install Kubernetes

You can choose any Kubernetes install of your choice. The test framework only depends on kubectl being configured. To install kubectl, please see the official guide.

Minikube

To install Minikube follow the official guide. It is recommended to use the kvm2 driver when running on a Linux machine and the hyperkit driver when running on a MacOS. Both allow to create and attach additional disks to the virtual machine. This is required for the Ceph OSD to consume one drive. We don't recommend any other drivers for Rook. You will need a Minikube version 1.23 or higher.

Starting the cluster on Minikube is as simple as running:

# On Linux
minikube start --disk-size=40g --extra-disks=1 --driver kvm2

# On MacOS
minikube start --disk-size=40g --extra-disks=1 --driver hyperkit

It is recommended to install a Docker client on your host system too. Depending on your operating system follow the official guide.

Stopping the cluster and destroying the Minikube virtual machine can be done with:

minikube delete

Install Helm

Use helm.sh to install Helm and set up Rook charts defined under _output/charts (generated by build):

• To install and set up Helm charts for Rook run tests/scripts/helm.sh up.
• To clean up tests/scripts/helm.sh clean.

NOTE: These helper scripts depend on some artifacts under the _output/ directory generated during build time. These scripts should be run from the project root.

NOTE: If Helm is not available in your PATH, Helm will be downloaded to a temporary directory (/tmp/rook-tests-scripts-helm) and used from that directory.

Run Tests

1. Build rook

Now that the Kubernetes cluster is running we need to populate the Docker registry environment variables:

eval $(minikube docker-env -p minikube)

The build process will build and push the images to the Docker registry inside the Minikube virtual machine.

make build

Now tag the newly built images to rook/ceph:local-build for running tests, or rook/ceph:master if creating sample manifests::

docker tag $(docker images|awk '/build-/ {print $1}') rook/ceph:local-build
docker tag rook/ceph:local-build rook/ceph:master

2. Run integration tests

Some settings are available to run the tests under different environments. The settings are all configured with environment variables. See environment.go for the available environment variables. At least you should set the following variables.

export TEST_HELM_PATH=/tmp/rook-tests-scripts-helm/linux-amd64/helm
export TEST_BASE_DIR=WORKING_DIR
export TEST_SCRATCH_DEVICE=/dev/vdb

Please note that the integration tests erase the contents of TEST_SCRATCH_DEVICE.

To run all integration tests:

go test -v -timeout 7200s github.com/rook/rook/tests/integration

After running tests, you can get test logs under "tests/integration/_output".

To run a specific suite (uses regex):

go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration

To run specific tests inside a suite:

go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration -testify.m TestARookClusterInstallation_SmokeTest

3. To run tests on OpenShift environment

• Setup OpenShift environment and export KUBECONFIG before executing the tests.
• Make sure oc executable file is in the PATH.
• Only CephSmokeSuite is currently supported on OpenShift.
• Set few environment variables:

export TEST_ENV_NAME=openshift
export TEST_STORAGE_CLASS=gp2
export TEST_BASE_DIR=/tmp
export RETRY_MAX=40

To run the CephSmokeSuite:

go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration