From e452ec8c47f6cd9060383245cd69c99c19c705d4 Mon Sep 17 00:00:00 2001 From: travisn Date: Fri, 19 May 2023 13:05:28 -0600 Subject: [PATCH] docs: update the integration test guide The integration test guide is updated to be more accurate and provide more hints that the CI is the recommended place to troubleshoot the integration tests Signed-off-by: travisn (cherry picked from commit fa5b758f9edeef0b7d62bb760b7b40ab194fcc84) --- tests/README.md | 80 +++++++++++++++++++++++++++---------------------- 1 file changed, 45 insertions(+), 35 deletions(-) diff --git a/tests/README.md b/tests/README.md index 8b1dd7ede..0bfe535b3 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,42 +1,57 @@ -# Rook Test Framework +# Integration Tests -The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of Kubernetes. -The framework also provides scripts for starting Kubernetes so users can -quickly spin up a Kubernetes cluster. The Test framework is designed to install Rook, run tests, and uninstall Rook. +The integration tests run end-to-end tests on Rook in a running instance of Kubernetes. +The framework includes scripts for starting Kubernetes so users can +quickly spin up a Kubernetes cluster. The tests are generally designed to install Rook, run tests, and uninstall Rook. -## Install Kubernetes +The CI runs the integration tests with each PR and each master or release branch build. +If the tests fail in a PR, a tmate session is started which will allow you to connect via ssh and troubleshoot the failure. +The CI is the most efficient way to troubleshoot the tests since the environment is started automatically +and you will only need to connect to investigate. -To install Kubernetes follow Rook's [developer guide](https://rook.io/docs/rook/latest/Contributing/development-environment/). +This document will outline the steps to run the integration tests locally in a minikube environment, should the CI +not be sufficient to troubleshoot. -## Run Tests +!!! hint + The CI is generally much simpler to troubleshoot than running these tests locally. Running the tests locally is rarely necessary. -### 1. Build rook +!!! warning + A risk of running the tests locally is that a local disk is required during the tests. If not running + in a VM, your laptop or other test machine could be destroyed. -Now that the Kubernetes cluster is running we need to populate the Docker registry environment variables: +## Install Minikube + +Follow Rook's [developer guide](https://rook.io/docs/rook/latest/Contributing/development-environment/) to install Minikube. + +## Build Rook image + +Now that the Kubernetes cluster is running we need to populate the Docker registry to allow local image builds +to be easily used inside Minikube. ```console eval $(minikube docker-env -p minikube) ``` -The build process will build and push the images to the Docker registry **inside** the Minikube +`make build` will now build and push the images to the Docker registry **inside** the Minikube virtual machine. ```console make build ``` -Now tag the newly built images to `rook/ceph:local-build` for running tests, or `rook/ceph:master` if creating sample manifests:: +Tag the newly built images to `rook/ceph:local-build` for running tests, or `rook/ceph:master` if creating example manifests:: ```console docker tag $(docker images|awk '/build-/ {print $1}') rook/ceph:local-build docker tag rook/ceph:local-build rook/ceph:master ``` -### 2. Run integration tests +## Run integration tests Some settings are available to run the tests under different environments. The settings are all configured with environment variables. See [environment.go](/tests/framework/installer/environment.go) for the available environment variables. -At least you should set the following variables. + +Set the following variables: ```console export TEST_HELM_PATH=/tmp/rook-tests-scripts-helm/linux-amd64/helm @@ -44,47 +59,42 @@ export TEST_BASE_DIR=WORKING_DIR export TEST_SCRATCH_DEVICE=/dev/vdb ``` -Change the `TEST_SCRATCH_DEVICE` and point it to the correct block device name based on the driver that's being used. -If you are using the `virtualbox` minikube driver, the device should be `/dev/sdb`. +Set `TEST_SCRATCH_DEVICE` to the correct block device name based on the driver that's being used. -Please note that the integration tests erase the contents of `TEST_SCRATCH_DEVICE`. +!!! hint + If using the `virtualbox` minikube driver, the device should be `/dev/sdb` -To run all integration tests: +!!! warning + The integration tests erase the contents of `TEST_SCRATCH_DEVICE` when the test is completed -```console -go test -v -timeout 7200s github.com/rook/rook/tests/integration -``` - -After running tests, you can get test logs under `tests/integration/_output`. - -To run a specific suite (uses regex): +To run a specific suite, specify the suite name: ```console go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration ``` +After running tests, see test logs under `tests/integration/_output`. + To run specific tests inside a suite: ```console go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration -testify.m TestARookClusterInstallation_SmokeTest ``` -### 3. To run tests on OpenShift environment +!!! info + Only the golang test suites are documented to run locally. Canary and other tests have only ever been supported in the CI. -- Setup OpenShift environment and export KUBECONFIG before executing the tests. -- Make sure `oc` executable file is in the PATH. -- Only `CephSmokeSuite` is currently supported on OpenShift. -- Set few environment variables: +## Running tests on OpenShift + +1. Setup OpenShift environment and export KUBECONFIG +2. Make sure `oc` executable file is in the PATH. +3. Only the `CephSmokeSuite` is currently supported on OpenShift. +4. Set the following environment variables depending on the environment: ```console export TEST_ENV_NAME=openshift export TEST_STORAGE_CLASS=gp2 export TEST_BASE_DIR=/tmp -export RETRY_MAX=40 ``` -To run the `CephSmokeSuite`: - -```console -go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration -``` +5. Run the integration tests