my-rook-config/tests/README.md

# Rook Test Framework

The Rook Test Framework is used to run end to end and integration tests on Rook. The framework depends on a running instance of kubernetes. 
The framework also provides scripts for starting kubernetes using kubeadm or minikube so users can 
quickly spin up a kubernetes cluster.The Test framework is designed to install Rook, run tests, and uninstall Rook.

## Requirements

1. Docker version => 1.2 && < 17.0
2. Ubuntu 16 (the framework has only been tested on this version)
3. Kubernetes with kubectl configured
4. Rook

## Instructions

### Setup

#### Install Kubernetes
You can choose any kubernetes flavor of your choice.  The test framework only depends on kubectl being configured. 
The framework also provides scripts to install Kubernetes. There are two scripts to start the cluster:
1. **Minikube** (recommended for MacOS): Run [minikube.sh](/tests/scripts/minikube.sh) to setup a single-node Minikube kubernetes.
    1. Minikube v0.21.0 and higher is supported. Older minikube versions do not have cephfs or rbd tools installed.
    1. If using Kubernetes v1.7.3 or higher, then Minikube v0.22.2 or higher is required.
1. **Kubeadm** (recommended for ubuntu): run [kubeadm.sh](/tests/scripts/kubeadm.sh) to setup a single-node K8s cluster using kubeadm

These start-up scripts are designed to start any version of kubernetes, Based on `KUBE_VERSION` environment variable
By default they start kubernetes version v1.7.5.
e.g. To kubernetes version v1.6.7
```
export KUBE_VERSION=v1.6.7
tests/scripts/kubeadm.sh up
```

To stop the cluster : `tests/scripts/kubeadm.sh clean` or `tests/scripts/miniube.sh clean`
  
To copy rook images generated by build into minikube vm, run the following commands after `minikube.sh up`
```
tests/scripts/minikube.sh update
tests/scripts/minikube.sh helm
```
kubeadm.sh starts kubernetes  on the host, so all the images generated by build should be available to the cluster.


### Install Helm
Use [helm.sh](/tests/scripts/helm.sh) to install helm and set up rook charts defined under `_output/charts` (generated by build). 
 - To install and set up Helm charts for rook run `tests/scripts/helm.sh up` 
 - To clean up `tests/scripts/helm.sh clean`

**Note:**  *kubeadm.sh, minikube.sh and helm.sh scripts depend on some artifacts under _output dir generated during build time, these scripts 
should be run from project root. eg* `tests/script/kubeadm.sh up`

## Run Tests
From the project root do the following:
#### 1. Build rook: 
 run `make build`

#### 2. Start kubernetes 
using one of the following:
   
   -  using Kubeadm  
   ```
    tests/scripts/kubeadm.sh up
    tests/scripts/helm.sh up  
  ```   
   - using minikube
   ```
   tests/scripts/minikube.sh up
   tests/scripts/minikube.sh helm
   tests/scripts/helm.sh up
   ```

#### 3. Run integration tests: 
Integration tests can be run using tests binary `_output/tests/${platform}/integration` that is generated during build time
eg ` ~/integration -test.v `
       

### Test parameters
In addition to standard go tests parameters, the following custom parameters are available while running tests

 Parameter | Description | Possible values | Default
 --- |--- | --- | ---
rook_platform| platform rook needs to be installed on  | kubernetes | kubernetes
k8s_version  | version of Kubernetes to be installed  | v1.5+  | v1.7.5
rook_image | rook image name to be installed | valid image name | rook/rook
toolbox_image | toolbox image name to be installed | valid image name | rook/toolbox
skip_install_rook | skips installing rook (if already installed) | true or false  | false
load_parallel_runs | performs concurrent operations (optional used for load test) | any number | 20

If the `install_rook` flag is set to false, then all the other flags are ignored
and tests are run without rook being installed and setup. Use this flag to run tests against
a pre-installed/configured rook.


### Running Tests with parameters.

#### To run all integration tests run 
```
go test -v -timeout 1800s github.com/rook/rook/tests/integration
```

#### To run a specific suite (uses regex)
```
go test -v -timeout 1800s -run SmokeSuite github.com/rook/rook/tests/integration
```

#### To run specific tests inside a suite:
```
go test -v -timeout 1800s -run SmokeSuite github.com/rook/rook/tests/integration -testify.m TestRookClusterInstallation_smokeTest

```

##### To run specific without installing rook
```
go test -v -timeout 1800s -run SmokeSuite github.com/rook/rook/tests/integration --skip_install_rook=true
```
If the `skip_install_rook` flag is set to true, then rook is not uninstalled either. 

#### Run Longhaul Tests
Using the `go test -count n` option, any tests can be repeated `n` number of times to simulate load or longhaul test. Although 
any test can be converted to a longhaul test, it's ideal to run integration tests under load for an extended period. Also a new custom test flag `load_parallel_runs` is added to control the number of concurrent operations being performed.
For example look at the [block long haul test](/tests/block/k8s/longhaul/basicBlockonghaul_test.go)
 
 To run a longhaul test you can run any integration test with `-count` and `--load_parallel_runs` options
 e.g.
 ```
 go test -run TestK8sBlockLongHaul github.com/rook/rook/tests/longhaul --load_parallel_runs=20 -count=1000
 ```
 The Longhaul test just like other test is going to install rook if it's not already installed, but it is not going to clean up test data or uninstall rook after the run. 
 Longhaul test is designed to run multiple times on the same setup and installation of rook to tests its stability. Test Data and rook should be cleaned up manually after the test.
 
 

Prerequisites :
* Go installed and GO_PATH set
* Dep installed 
* when running tests locally, make sure `kubectl` is accessible globally as the test framework uses kubectl