Blaine Gardner
a777b1d7d1
External CephObjectStores already have endpoints defined by spec.gateway.externalRgwEndpoints, and if the external store is configured with TLS (HTTPS), the store's certificates will likely not accept connections intended for the Service endpoint Rook creates. Some users might not be able to easily add the service endpoint to their certificates. Therefore, don't even bother creating a Service for external clusters. This does introduce a few issues. The Service seems to have been initially created to allow multiple external RGW endpoints to be addressable via a single address in Rook. For all connections to an external CephObjectStore with multiple endpoints, simply choose an endpoint at random. Random selection will prevent Rook from failing to create buckets or users on an external store if one of the external store's endpoints fails. The latest OBC library (lib-bucket-provisioner) allows updating the endpoints on ObjectBuckets after they are created. This allows Rook users to change endpoints on external CephObjectStores without breaking all existing OBCs. It requires implementation of the new GetUserID() library call, requires updating Provision() and Grant() calls to be idempotent, and it requires removing the Update() call. Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Rook Test Framework
The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of Kubernetes. The framework also provides scripts for starting Kubernetes so users can quickly spin up a Kubernetes cluster. The Test framework is designed to install Rook, run tests, and uninstall Rook.
Install Kubernetes
To install Kubernetes follow Rook's developer guide.
Run Tests
1. Build rook
Now that the Kubernetes cluster is running we need to populate the Docker registry environment variables:
eval $(minikube docker-env -p minikube)
The build process will build and push the images to the Docker registry inside the Minikube virtual machine.
make build
Now tag the newly built images to rook/ceph:local-build for running tests, or rook/ceph:master if creating sample manifests::
docker tag $(docker images|awk '/build-/ {print $1}') rook/ceph:local-build
docker tag rook/ceph:local-build rook/ceph:master
2. Run integration tests
Some settings are available to run the tests under different environments. The settings are all configured with environment variables. See environment.go for the available environment variables. At least you should set the following variables.
export TEST_HELM_PATH=/tmp/rook-tests-scripts-helm/linux-amd64/helm
export TEST_BASE_DIR=WORKING_DIR
export TEST_SCRATCH_DEVICE=/dev/vdb
Change the TEST_SCRATCH_DEVICE and point it to the correct block device name based on the driver that's being used.
If you are using the virtualbox minikube driver, the device should be /dev/sdb.
Please note that the integration tests erase the contents of TEST_SCRATCH_DEVICE.
To run all integration tests:
go test -v -timeout 7200s github.com/rook/rook/tests/integration
After running tests, you can get test logs under tests/integration/_output.
To run a specific suite (uses regex):
go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration
To run specific tests inside a suite:
go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration -testify.m TestARookClusterInstallation_SmokeTest
3. To run tests on OpenShift environment
- Setup OpenShift environment and export KUBECONFIG before executing the tests.
- Make sure
ocexecutable file is in the PATH. - Only
CephSmokeSuiteis currently supported on OpenShift. - Set few environment variables:
export TEST_ENV_NAME=openshift
export TEST_STORAGE_CLASS=gp2
export TEST_BASE_DIR=/tmp
export RETRY_MAX=40
To run the CephSmokeSuite:
go test -v -timeout 1800s -run CephSmokeSuite github.com/rook/rook/tests/integration