feedc0de/my-rook-config
1
0 Fork 0
forked from rook/rook
Code Pull Requests Activity

docs: clarifications for the 0.8 release

Browse Source 
Signed-off-by: travisn <tnielsen@redhat.com>
This commit is contained in:
travisn committed 2018-07-13 14:29:57 -06:00
1 parent d93bd7603f
commit 0854cbaf55
22 files changed
+230 -104
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Documentation/README.md
+1 -1
View File
@@ -11,7 +11,7 @@ We plan to continue adding support for other storage systems and environments ba
## Getting Started
Starting Rook in your cluster is as simple as two `kubectl` commands. See our [Quickstart](quickstart.md) guide for the details on what you need to get going.
Starting Rook in your cluster is as simple as two `kubectl` commands. See our [Quickstart](ceph-quickstart.md) guide for the details on what you need to get going.
Once you have a Rook cluster running, walk through the guides for block, object, and file to start consuming the storage in your cluster:
- **[Block](block.md)**: Create block storage to be consumed by a pod
Documentation/advanced-configuration.md
+2 -2
View File
@@ -78,7 +78,7 @@ do
done
```
The output should look something like this:
The output should look something like this. Note that OSDs on the same node will show duplicate information.
```bash
Pod: osd-m2fz2
@@ -300,7 +300,7 @@ Of course you will only have 1/3 the capacity as a tradeoff.
### Setting PG Count
Be sure to read the [placement group sizing](#placement-group-sizing) section
before changing the numner of PGs.
before changing the number of PGs.
```bash
# Set the number of PGs in the rbd pool to 512
Documentation/block.md
+1 -1
View File
@@ -10,7 +10,7 @@ Block storage allows you to mount storage to a single pod. This example shows ho
## Prerequisites
This guide assumes you have created a Rook cluster as explained in the main [Quickstart](quickstart.md) guide.
This guide assumes you have created a Rook cluster as explained in the main [Quickstart](ceph-quickstart.md) guide.
## Provision Storage
Documentation/ceph-cluster-crd.md
+59 -27
View File
@@ -18,7 +18,6 @@ Settings can be specified at the global level to apply to the cluster as a whole
### Cluster Settings
- `dataDirHostPath`: The path on the host ([hostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)) where config and data should be stored for each of the services. If the directory does not exist, it will be created. Because this directory persists on the host, it will remain after pods are deleted.
- On **Minikube** environments, use `/data/rook`. Minikube boots into a tmpfs but it provides some [directories](https://github.com/kubernetes/minikube/blob/master/docs/persistent_volumes.md) where files can be persisted across reboots. Using one of these directories will ensure that Rook's data and configuration files are persisted and that enough storage space is available.
- If a path is not specified, an [empty dir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) will be used and the config will be lost when the pod or host is restarted. This option is **not recommended**.
- **WARNING**: For test scenarios, if you delete a cluster and start a new cluster on the same hosts, the path used by `dataDirHostPath` must be deleted. Otherwise, stale keys and other config will remain from the previous cluster and the new mons will fail to start.
If this value is empty, each pod will get an ephemeral directory to store their config files that is tied to the lifetime of the pod running on that node. More details can be found in the Kubernetes [empty dir docs](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir).
- `dashboard`: Settings for the Ceph dashboard. To view the dashboard in your browser see the [dashboard guide](ceph-dashboard.md).
@@ -44,7 +43,7 @@ This will bring up your default text editor and allow you to add and remove stor
This feature is only available when `useAllNodes` has been set to `false`.
### Mon Settings
- `count`: set the number of mons to be started. The number should be odd and between `1` and `9`. If not specified will be set to `3` and `allowMultiplePerNode` is also set to `true`.
- `count`: set the number of mons to be started. The number should be odd and between `1` and `9`. If not specified the default is set to `3` and `allowMultiplePerNode` is also set to `true`.
- `allowMultiplePerNode`: enable (`true`) or disable (`false`) the placement of multiple mons on one node. Default is `false`.
### Node Settings
@@ -113,13 +112,10 @@ For more information on resource requests/limits see the official Kubernetes doc
- `memory`: Limit for Memory (example: one gigabyte of memory `1Gi`, half a gigabyte of memory `512Mi`).
## Samples
Here are several samples for configuring Ceph clusters. Each of the samples must also include the namespace and corresponding access granted for management by the Ceph operator. See the [common cluster resources](#common-cluster-resources) below.
### Storage configuration: All devices
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
---
apiVersion: ceph.rook.io/v1beta1
kind: Cluster
metadata:
@@ -147,11 +143,6 @@ Individual nodes and their config can be specified so that only the named nodes
Each node's 'name' field should match their 'kubernetes.io/hostname' label.
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
---
apiVersion: ceph.rook.io/v1beta1
kind: Cluster
metadata:
@@ -189,11 +180,6 @@ This example is based up on the [Storage Configuration: Specific devices](#stora
Individual nodes can override the cluster wide specified directories list.
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
---
apiVersion: ceph.rook.io/v1beta1
kind: Cluster
metadata:
@@ -225,11 +211,6 @@ The example under 'all' would have all services scheduled on kubernetes nodes la
tolerate taints with a key of 'storage-node'.
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
---
apiVersion: ceph.rook.io/v1beta1
kind: Cluster
metadata:
@@ -269,11 +250,6 @@ You can override these requests/limits for OSDs per node when using `useAllNodes
**WARNING** Before setting resource requests/limits, please take a look at the Ceph documentation for recommendations for each component: [Ceph - Hardware Recommendations](http://docs.ceph.com/docs/master/start/hardware-recommendations/).
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
---
apiVersion: ceph.rook.io/v1beta1
kind: Cluster
metadata:
@@ -296,3 +272,59 @@ spec:
cpu: "2"
memory: "4096Mi"
```
## Common Cluster Resources
Each Ceph cluster must be created in a namespace and also give access to the Rook operator to manage the cluster in the namespace. Creating the namespace and these controls must be added to each of the examples previously shown.
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: rook-ceph-cluster
namespace: rook-ceph
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: rook-ceph-cluster
namespace: rook-ceph
rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: [ "get", "list", "watch", "create", "update", "delete" ]
---
# Allow the operator to create resources in this cluster's namespace
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: rook-ceph-cluster-mgmt
namespace: rook-ceph
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: rook-ceph-cluster-mgmt
subjects:
- kind: ServiceAccount
name: rook-ceph-system
namespace: rook-ceph-system
---
# Allow the pods in this namespace to work with configmaps
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: rook-ceph-cluster
namespace: rook-ceph
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: rook-ceph-cluster
subjects:
- kind: ServiceAccount
name: rook-ceph-cluster
namespace: rook-ceph
```
Documentation/quickstart.md → Documentation/ceph-quickstart.md
+19 -17
View File
@@ -1,14 +1,12 @@
---
title: Quickstart
weight: 2
title: Ceph Storage
weight: 3
indent: true
---
# Quickstart Guide
# Ceph Storage Quickstart
Welcome to Rook! We hope you have a great experience installing the Rook storage platform to enable highly available, durable storage
in your cluster. If you have any questions along the way, please don't hesitate to ask us in our [Slack channel](https://rook-io.slack.com). Sign up for our Slack [here](https://rook-slackin.herokuapp.com/).
This guide will walk you through the basic setup of a Rook cluster. This will enable you to consume block, object, and file storage
This guide will walk you through the basic setup of a Ceph cluster and enable you to consume block, object, and file storage
from other pods running in your cluster.
## Minimum Version
@@ -40,7 +38,7 @@ The first step is to deploy the Rook system components, which include the Rook a
cd cluster/examples/kubernetes/ceph
kubectl create -f operator.yaml
# verify the rook-ceph-operator and rook-ceph-agent pods are in the `Running` state before proceeding
# verify the rook-ceph-operator, rook-ceph-agent, and rook-discover pods are in the `Running` state before proceeding
kubectl -n rook-ceph-system get pod
```
@@ -56,7 +54,7 @@ For versions of Kubernetes prior to 1.8, the Kubelet process on all nodes will r
## Create a Rook Cluster
Now that the Rook operator and agent pods are running, we can create the Rook cluster. For the cluster to survive reboots,
Now that the Rook operator, agent, and discover pods are running, we can create the Rook cluster. For the cluster to survive reboots,
make sure you set the `dataDirHostPath` property. For more settings, see the documentation on [configuring the cluster](ceph-cluster-crd.md).
@@ -91,16 +89,20 @@ Create the cluster:
kubectl create -f cluster.yaml
```
Use `kubectl` to list pods in the `rook` namespace. You should be able to see the following pods once they are all running:
Use `kubectl` to list pods in the `rook` namespace. You should be able to see the following pods once they are all running.
The number of osd pods will depend on the number of nodes in the cluster and the number of devices and directories configured.
```bash
$ kubectl -n rook-ceph get pod
NAME READY STATUS RESTARTS AGE
rook-ceph-mgr0-1279756402-wc4vt 1/1 Running 0 5m
rook-ceph-mon0-jflt5 1/1 Running 0 6m
rook-ceph-mon1-wkc8p 1/1 Running 0 6m
rook-ceph-mon2-p31dj 1/1 Running 0 6m
rook-ceph-osd-0h6nb 1/1 Running 0 5m
NAME READY STATUS RESTARTS AGE
rook-ceph-mgr-a-75cc4ccbf4-t8qtx 1/1 Running 0 24m
rook-ceph-mon0-72vx7 1/1 Running 0 25m
rook-ceph-mon1-rrpm6 1/1 Running 0 24m
rook-ceph-mon2-zff9r 1/1 Running 0 24m
rook-ceph-osd-id-0-5fd8cb9747-dvlsb 1/1 Running 0 23m
rook-ceph-osd-id-1-84dc695b48-r5mhf 1/1 Running 0 23m
rook-ceph-osd-id-2-558878cd84-cnp67 1/1 Running 0 23m
rook-ceph-osd-prepare-minikube-wq4f5 0/1 Completed 0 24m
```
# Storage
@@ -125,4 +127,4 @@ To learn how to set up monitoring for your Rook cluster, you can follow the step
# Teardown
When you are done with the test cluster, see [these instructions](teardown.md) to clean up the cluster.
When you are done with the test cluster, see [these instructions](ceph-teardown.md) to clean up the cluster.
Documentation/teardown.md → Documentation/ceph-teardown.md
+6 -7
View File
@@ -1,6 +1,6 @@
---
title: Cleanup
weight: 5
title: Ceph Cleanup
weight: 4
indent: true
---
@@ -21,8 +21,8 @@ First you will need to clean up the resources created on top of the Rook cluster
These commands will clean up the resources from the [block](block.md#teardown) and [file](filesystem.md#teardown) walkthroughs (unmount volumes, delete volume claims, etc). If you did not complete those parts of the walkthrough, you can skip these instructions:
```console
kubectl delete -f wordpress.yaml
kubectl delete -f mysql.yaml
kubectl delete -f ../wordpress.yaml
kubectl delete -f ../mysql.yaml
kubectl delete -n rook-ceph pool replicapool
kubectl delete storageclass rook-ceph-block
kubectl delete -f kube-registry.yaml
@@ -39,10 +39,9 @@ Verify that the cluster CRD has been deleted before continuing to the next step.
kubectl -n rook-ceph get cluster.ceph.rook.io
```
## Delete the Operator and Agent
This will begin the process of all cluster resources being cleaned up, after which you can delete the rest of the deployment with the following:
## Delete the Operator
This will begin the process of all cluster resources being cleaned up, after which you can delete the operator and related resources such as the agent and discover daemonsets with the following:
```console
kubectl delete -n rook-ceph-system daemonset rook-ceph-agent
kubectl delete -f operator.yaml
```
Documentation/cockroachdb.md
+2 -2
View File
@@ -1,10 +1,10 @@
---
title: CockroachDB
weight: 27
weight: 5
indent: true
---
# CockroachDB
# CockroachDB Quickstart
CockroachDB is a cloud-native SQL database for building global, scalable cloud services that survive disasters.
Rook provides an operator to deploy and manage CockroachDB clusters.
Documentation/common-issues.md
+54 -3
View File
@@ -16,6 +16,7 @@ If after trying the suggestions found on this page and the problem is not resolv
- [Cluster failing to service requests](#cluster-failing-to-service-requests)
- [Only a single monitor pod starts](#only-a-single-monitor-pod-starts)
- [OSD pods are failing to start](#osd-pods-are-failing-to-start)
- [OSDs are not created on my devices](#osd-pods-are-not-created-on-my-devices)
- [Node hangs after reboot](#node-hangs-after-reboot)
- [Rook Agent modprobe exec format error](#rook-agent-modprobe-exec-format-error)
- [Using multiple shared filesystem (CephFS) is attempted on a kernel version older than 4.7](#using-multiple-shared-filesystem-cephfs-is-attempted-on-a-kernel-version-older-than-47)
@@ -51,7 +52,7 @@ After you verify the basic health of the running pods, next you will want to run
```
### Tools in other pods
The Ceph tools are found in all of the Rook pods where Ceph is running, such as the operator, monitors, and OSDs. Rather than starting the toolbox pod, you can connect to the existing pods to more quickly execute the Ceph tools. For example, to connect to the operator pod:
The Ceph tools are found in all of the Rook pods where Ceph is running, such as the operator and monitors. Rather than starting the toolbox pod, you can connect to the existing pods to more quickly execute the Ceph tools. For example, to connect to the operator pod:
```bash
kubectl -n rook-ceph-system exec -it $(kubectl -n rook-ceph-system get pods -l app=rook-operator -o jsonpath='{.items[0].metadata.name}') -- bash
@@ -59,7 +60,7 @@ kubectl -n rook-ceph-system exec -it $(kubectl -n rook-ceph-system get pods -l a
Now from inside the operator pod you can execute the Ceph tools.
It is preferred to connect to the operator pod rather than monitors and OSDs since the operator always has the latest configuration files. If you have multiple Rook clusters, it is preferred to connect to the Rook toolbox for a specific cluster. Otherwise, your ceph commands may connect to the wrong cluster.
It is preferred to connect to the operator pod rather than monitors since the operator always has the latest configuration files. If you have multiple Rook clusters, it is preferred to connect to the Rook toolbox for a specific cluster. Otherwise, your ceph commands may connect to the wrong cluster.
### Ceph Commands
Here are some common commands to troubleshoot the cluster.
@@ -150,7 +151,7 @@ A symptom of this can be found in the Kubelet's log/journal, with the following
```console
Oct 30 22:23:03 core-02 kubelet-wrapper[31926]: E1030 22:23:03.524159 31926 desired_state_of_world_populator.go:285] Failed to add volume "mysql-persistent-storage" (specName: "pvc-9f273fbc") for pod "9f2ff89a-bdbf" to desiredStateOfWorld. err=failed to get Plugin from volumeSpec for volume "pvc-9f273fbc" err=no volume plugin matched
```
If you encounter this, just **restart the Kubelet process**, as described in the **[Restart Kubelet](./quickstart.md#restart-kubelet)** section of the Rook deployment guide.
If you encounter this, just **restart the Kubelet process**, as described in the **[Restart Kubelet](./ceph-quickstart.md#restart-kubelet)** section of the Rook deployment guide.
### Volume Creation
The volume must first be created in the Rook cluster and then bound to a volume claim before it can be mounted to a pod.
@@ -365,6 +366,56 @@ This directory is the `dataDirHostPath` setting in the cluster CRD and is typica
To fix the issue you will need to delete all components of Rook and then delete the contents of `/var/lib/rook` (or the directory specified by `dataDirHostPath`) on each of the hosts in the cluster.
Then when the cluster CRD is applied to start a new cluster, the rook-operator should start all the pods as expected.
# OSD pods are not created on my devices
## Symptoms
* No OSD pods are started in the cluster
* Devices are not configured with OSDs even though specified in the Cluster CRD
* One OSD pod is started on each node instead of multiple pods for each device
## Investigation
First, ensure that you have specified the devices correctly in the CRD.
The [Cluster CRD](ceph-cluster-crd.md#storage-selection-settings) has several ways to specify the devices that are to be consumed by the Rook storage:
- `useAllDevices: true`: Rook will consume all devices it determines to be available
- `deviceFilter`: Consume all devices that match this regular expression
- `devices`: Explicit list of device names on each node to consume
Second, if Rook determines that a device is not available (has existing partitions or a formatted file system), Rook will skip consuming the devices.
If Rook is not starting OSDs on the devices you expect, Rook may have skipped it for this reason. To see if a device was skipped, view the OSD preparation log
on the node where the device was skipped.
```
# get the prepare pods in the cluster
$ kubectl -n rook-ceph get pod -l app=rook-ceph-osd-prepare
NAME READY STATUS RESTARTS AGE
rook-ceph-osd-prepare-node1-fvmrp 0/1 Completed 0 18m
rook-ceph-osd-prepare-node2-w9xv9 0/1 Completed 0 22m
rook-ceph-osd-prepare-node3-7rgnv 0/1 Completed 0 22m
# view the logs for the node of interest
$ kubectl -n rook-ceph logs rook-ceph-osd-prepare-node1-fvmrp
```
Towards the begining of the log you will see messages such as the following:
```
# message that the device sda was skipped
cephosd: skipping device sda that is in use (not by rook)
# message that the devices sdb and sdc are being configured
cephosd: configuring osd devices: {"Entries":{"sdb":{"Data":-1,"Metadata":null},"sdc":{"Data":-1,"Metadata":null}}}
```
## Solution
After you have either updated the CRD with the correct settings, or you have cleaned the partitions or file system from your devices,
you can trigger the operator to analyze the devices again by restarting the operator. Each time the operator starts, it
will ensure all the desired devices are configured.
```
# Restart the operator to ensure devices are configured. A new pod will automatically be started when the current operator pod is deleted.
$ kubectl -n rook-ceph-system delete pod -l app=rook-ceph-operator
```
# Node hangs after reboot
## Symptoms
Documentation/crds.md
+4 -1
View File
@@ -15,4 +15,7 @@ has its own CRD defined.
- [File System](ceph-filesystem-crd.md): A file system provides shared storage for multiple Kubernetes pods.
## CockroachDB
- [Cluster](cockroachdb-cluster-crd.md): CockroachDB is an open-source distributed SQL database that is highly scalable across multiple global regions and also highly durable.
- [Cluster](cockroachdb-cluster-crd.md): CockroachDB is an open-source distributed SQL database that is highly scalable across multiple global regions and also highly durable.
## Minio
- [Object Store](minio-object-store-crd.md): Minio is a high performance distributed object storage server, designed for large-scale private cloud infrastructure.
Documentation/development-flow.md
+8 -8
View File
@@ -10,7 +10,7 @@ don't hesitate to reach out to us on our [Slack](https://Rook-io.slack.com) dev
## Prerequisites
1. [GO 1.8](https://golang.org/dl/) or greater installed
1. [GO 1.10](https://golang.org/dl/) or greater installed
2. Git client installed
3. Github account
@@ -63,9 +63,9 @@ For new features of significant scope and complexity, a design document is recom
For smaller, straightforward features and bug fixes, there is no need for a design document.
Authoring a design document for big features has many advantages:
* helps flesh out the approach by forcing the author to think critically about the feature and can identify potential issues early on
* gets agreement amongst the community before code is written that could be wasted effort in the wrong direction
* serves as an artifact of the architecture that is easier to read for visitors to the project than just the code by itself
* Helps flesh out the approach by forcing the author to think critically about the feature and can identify potential issues early on
* Gets agreement amongst the community before code is written that could be wasted effort in the wrong direction
* Serves as an artifact of the architecture that is easier to read for visitors to the project than just the code by itself
Note that writing code to prototype the feature while working on the design may be very useful to help flesh out the approach.
@@ -112,13 +112,13 @@ Rebasing is a very powerful feature of Git. You need to understand how it works
## Submitting a Pull Request
Once you have implemented the feature or bug fix in your branch, you will open a PR to the upstream rook repo. Before opening the PR ensure you have added unit tests, are passing the E2E tests, cleaned your commit history, and have rebased on the latest upstream.
Once you have implemented the feature or bug fix in your branch, you will open a PR to the upstream rook repo. Before opening the PR ensure you have added unit tests, are passing the integration tests, cleaned your commit history, and have rebased on the latest upstream.
In order to open a pull request (PR) it is required to be up to date with the latest changes upstream. If other commits are pushed upstream before your PR is merged, you will also need to rebase again before it will be merged.
### Regression Testing
All pull requests must pass the unit and e2e smoke tests before they can be merged. These tests automatically
All pull requests must pass the unit and integration tests before they can be merged. These tests automatically
run as a part of the build process. The results of these tests along with code reviews and other criterias determine whether
your request will be accepted into the `rook/rook` repo. It is prudent to run all tests locally on your development box prior to submitting a pull request to the `rook/rook` repo.
@@ -137,9 +137,9 @@ go test -coverprofile=coverage.out
go tool cover -html=coverage.out -o coverage.html
```
#### Running the E2E Tests
#### Running the Integration Tests
For instructions on how to execute the end to end smoke test suite,
follow the [e2e instructions](../e2e/README.md).
follow the [test instructions](https://github.com/rook/rook/blob/master/tests/README.md).
### Commit History
Documentation/direct-tools.md
+2 -2
View File
@@ -45,7 +45,7 @@ mount /dev/rbd0 /tmp/rook-volume
Write and read a file:
```bash
echo "Hello Rook!" > /tmp/rook-volume/hello
echo "Hello Rook" > /tmp/rook-volume/hello
cat /tmp/rook-volume/hello
```
@@ -88,7 +88,7 @@ ls /tmp/registry
Try writing and reading a file to the shared file system.
```bash
echo "Hello Rook!" > /tmp/registry/hello
echo "Hello Rook" > /tmp/registry/hello
cat /tmp/registry/hello
# delete the file when you're done
Documentation/disaster-recovery.md
+1 -1
View File
@@ -19,7 +19,7 @@ that it is the only mon in quorum, and then restart the good mon.
### Stop the operator
First, stop the operator so it will not try to failover the mons while we are modifying the monmap
```bash
kubectl -n rook-ceph-system delete deployment rook-operator
kubectl -n rook-ceph-system delete deployment rook-ceph-operator
```
### Inject a new monmap
Documentation/filesystem.md
+1 -1
View File
@@ -12,7 +12,7 @@ This example runs a shared file system for the [kube-registry](https://github.co
### Prerequisites
This guide assumes you have created a Rook cluster as explained in the main [Kubernetes guide](quickstart.md)
This guide assumes you have created a Rook cluster as explained in the main [Kubernetes guide](ceph-quickstart.md)
### Multiple File Systems Not Supported
Documentation/helm-operator.md
+13 -14
View File
@@ -34,26 +34,25 @@ kubectl --namespace kube-system patch deploy/tiller-deploy -p '{"spec": {"templa
## Installing
The Ceph Operator helm chart will install the basic components necessary to create a storage platform for your Kubernetes cluster.
After the helm chart is installed, you will need to [create a Rook cluster](quickstart.md#create-a-rook-cluster).
After the helm chart is installed, you will need to [create a Rook cluster](ceph-quickstart.md#create-a-rook-cluster).
The `helm install` command deploys rook on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. It is recommended that the rook operator be installed into the `rook-ceph-system` namespace (you will install your clusters into separate namespaces).
Rook currently publishes builds to the `alpha` and `master` channels. In the future, `beta` and `stable` will also be available.
Rook currently publishes builds of the Ceph operator to the `beta`, `alpha`, and `master` channels. In the future `stable` will also be available.
### Beta
The beta channel is the most recent release of Rook that is considered nearly stable for the community, starting with the v0.8 release.
```console
helm repo add rook-beta https://charts.rook.io/beta
helm install --namespace rook-ceph-system rook-beta/rook-ceph
```
### Alpha
The alpha channel is the most recent release of Rook that is considered ready for testing by the community.
```console
helm repo add rook-alpha https://charts.rook.io/alpha
```
For the v0.7 release (see the [v0.7 documentation](https://rook.io/docs/rook/v0.7/helm-operator.html)):
```console
helm install --namespace rook-system rook-alpha/rook
```
After the v0.8 release is available:
```console
helm install --namespace rook-ceph-system rook-alpha/rook-ceph
```
@@ -69,7 +68,7 @@ helm install --namespace rook-ceph-system rook-master/rook-ceph --version <versi
For example:
```
helm install rook-master/rook-ceph --version v0.6.0-156.gef983d6
helm install --namespace rook-ceph-system rook-master/rook-ceph --version v0.7.0-278.gcbd9726
```
### Development Build
@@ -121,14 +120,14 @@ You can pass the settings with helm command line parameters. Specify each parame
`--set key=value[,key=value]` argument to `helm install`. For example, the following command will install rook where RBAC is not enabled.
```console
$ helm install --namespace rook-ceph-system --name rook-ceph rook-alpha/rook-ceph --set rbacEnable=false
$ helm install --namespace rook-ceph-system --name rook-ceph rook-beta/rook-ceph --set rbacEnable=false
```
### Settings File
Alternatively, a yaml file that specifies the values for the above parameters (`values.yaml`) can be provided while installing the chart.
```console
$ helm install --namespace rook-ceph-system --name rook-ceph rook-alpha/rook-ceph -f values.yaml
$ helm install --namespace rook-ceph-system --name rook-ceph rook-beta/rook-ceph -f values.yaml
```
Here are the sample settings to get you started.
Documentation/k8s-pre-reqs.md
+2 -2
View File
@@ -67,12 +67,12 @@ Then wait for the cluster to come up and verify that kubernetes is done initiali
kubectl cluster-info
```
Once you see a url response, your cluster is [ready for use by Rook](quickstart.md#deploy-rook).
Once you see a url response, your cluster is [ready for use by Rook](ceph-quickstart.md#deploy-rook).
## Using Rook in Kubernetes
Now that you have a Kubernetes cluster running, you can start using Rook with [these steps](quickstart.md#deploy-rook).
Now that you have a Kubernetes cluster running, you can start using Rook with [these steps](ceph-quickstart.md#deploy-rook).
## Using Rook on Tectonic Bare Metal
Documentation/minio-object-store.md
+2 -2
View File
@@ -1,10 +1,10 @@
---
title: Minio Object Store
weight: 28
weight: 6
indent: true
---
# Minio
# Minio Object Store Quickstart
Minio is a high performance distributed object storage server, designed for
large-scale private cloud infrastructure.
Documentation/object.md
+3 -3
View File
@@ -10,7 +10,7 @@ Object storage exposes an S3 API to the storage cluster for applications to put
## Prerequisites
This guide assumes you have created a Rook cluster as explained in the main [Kubernetes guide](quickstart.md)
This guide assumes you have created a Rook cluster as explained in the main [Kubernetes guide](ceph-quickstart.md)
## Create an Object Store
@@ -51,7 +51,7 @@ kubectl -n rook-ceph get pod -l app=rook-ceph-rgw
## Create a User
Creating an object storage user requires running a `radosgw-admin` command with the [Rook toolbox](quickstart.md#tools) pod. This will be simplified in the future with a CRD for the object store users.
Creating an object storage user requires running a `radosgw-admin` command with the [Rook toolbox](ceph-quickstart.md#tools) pod. This will be simplified in the future with a CRD for the object store users.
```bash
radosgw-admin user create --uid rook-user --display-name "A rook rgw User" --rgw-realm=my-store --rgw-zonegroup=my-store
@@ -116,7 +116,7 @@ List buckets in the object store
Upload a file to the newly created bucket
```bash
echo "Hello Rook!" > /tmp/rookObj
echo "Hello Rook" > /tmp/rookObj
s3cmd put /tmp/rookObj --no-ssl --host=${AWS_HOST} --host-bucket= s3://rookbucket
```
Documentation/quickstart-toc.md
+17
View File
@@ -0,0 +1,17 @@
---
title: Quickstart
weight: 2
---
# Quickstart Guides
Welcome to Rook! We hope you have a great experience installing the Rook storage platform to enable highly available, durable storage
in your cluster. If you have any questions along the way, please don't hesitate to ask us in our [Slack channel](https://rook-io.slack.com). You can sign up for our Slack [here](https://rook-slackin.herokuapp.com/).
Rook provides three types of storage providers to a Kubernetes cluster, each with its own operator to deploy and manage the cluster. Follow these guides to get started with each provider.
| Storage Provider | Status | Description |
|---|---|---|
| [Ceph](ceph-quickstart.md) | Beta | Ceph is a highly scalable distributed storage solution for block storage, object storage, and shared file systems with years of production deployments. |
| [CockroachDB](cockroachdb.md) | Alpha | CockroachDB is a cloud-native SQL database for building global, scalable cloud services that survive disasters. |
| [Minio](minio-object-store.md) | Alpha | Minio is a high performance distributed object storage server, designed for large-scale private cloud infrastructure. |
Documentation/storage.md
+3 -3
View File
@@ -1,11 +1,11 @@
---
title: Storage
title: Ceph Storage
weight: 20
---
# Storage
# Ceph Storage
Rook provides three types of storage to the Kubernetes cluster.
Rook provides three types of storage to the Kubernetes cluster with the Ceph operator.
- [Block Storage](block.md): Mount storage to a single pod
- [Object Storage](object.md): Expose an S3 API to the storage cluster for applications to put and get data that is accessible from inside or outside the Kubernetes cluster
- [Shared File System](filesystem.md): Mount a file system that can be shared across multiple pods
Documentation/tectonic.md
+1 -1
View File
@@ -44,5 +44,5 @@ Visit the official [Tectonic manual boot](https://coreos.com/tectonic/docs/lates
**Remark:** The Tectonic installer contains the correct terraform binary out of the box. This terraform binary can be found in following directory `~/tectonic/tectonic-installer/linux`.
## Start Rook
After the Tectonic Installer ran and the Kubernetes cluster is started and ready, you can follow the [Rook installation guide](quickstart.md).
After the Tectonic Installer ran and the Kubernetes cluster is started and ready, you can follow the [Rook installation guide](ceph-quickstart.md).
If you want to specify which disks Rook uses, follow the instructions in [creating Rook clusters](ceph-cluster-crd.md)
Documentation/toolbox.md
+4 -2
View File
@@ -10,7 +10,7 @@ The toolbox is based on CentOS, so more tools of your choosing can be easily ins
## Running the Toolbox in Kubernetes
The rook toolbox can run as a pod in a Kubernetes cluster. After you ensure you have a running Kubernetes cluster with rook deployed (see the [Kubernetes](quickstart.md) instructions),
The rook toolbox can run as a pod in a Kubernetes cluster. After you ensure you have a running Kubernetes cluster with rook deployed (see the [Kubernetes](ceph-quickstart.md) instructions),
launch the rook-ceph-tools pod.
Save the tools spec as `toolbox.yaml`:
@@ -92,4 +92,6 @@ kubectl -n rook-ceph delete pod rook-ceph-tools
```
## Troubleshooting without the Toolbox
The Ceph tools will commonly be the only tools needed to troubleshoot a cluster. In that case, you can connect to any of the rook pods and execute the ceph commands in the same way that you would in the toolbox pod. For example, you can connect to the mon, osd, or even the operator pod to execute commands such as `ceph status`.
The Ceph tools will commonly be the only tools needed to troubleshoot a cluster. In that case, you can connect to any of the rook pods and execute the ceph commands in the same way that you would in the toolbox pod such as the mon pods or the operator pod.
If connecting to the mon pods, make sure you connect to the mon most recently started. The mons keep the config updated in memory after starting and may not have the latest config on disk.
For example, after starting the cluster connect to the `mon2` pod instead of `mon0`.
Documentation/upgrade.md
+25 -4
View File
@@ -14,7 +14,9 @@ The goal is to provide prescriptive guidance and knowledge on how to upgrade a l
We welcome feedback and opening issues!
## Supported Versions
The supported version for this upgrade guide is **from an 0.7 release to the latest builds**. Until 0.8 is released, the latest builds are labeled such as `v0.7.0-27.gbfc8ec6`. Build-to-build upgrades are not guaranteed to work. This guide is to test upgrades only between the official releases.
The supported version for this upgrade guide is **from an 0.7 release to the latest builds**. Until 0.8 is released,
the latest builds are labeled such as `v0.7.0-280.g41829b1`. Build-to-build upgrades are not guaranteed to work.
This guide is to test upgrades only between the official releases.
For a guide to upgrade previous versions of Rook, please refer to the version of documentation for those releases.
- [Upgrade 0.6 to 0.7](https://rook.io/docs/rook/v0.7/upgrade.html)
@@ -35,7 +37,7 @@ This persists metadata on host nodes, enabling pods to be terminated during the
More details about `dataDirHostPath` can be found in the [Cluster CRD readme](./ceph-cluster-crd.md#cluster-settings).
* All pods consuming Rook storage should be created, running, and in a steady state. No Rook persistent volumes should be in the act of being created or deleted.
The minimal sample Cluster spec that will be used in this guide can be found below (note that the specific configuration may not be applicable to all environments):
The minimal sample v0.7 Cluster spec that will be used in this guide can be found below (note that the specific configuration may not be applicable to all environments):
```
apiVersion: v1
kind: Namespace
@@ -125,7 +127,7 @@ Any pod that is using a Rook volume should also remain healthy:
## Upgrade Process
The general flow of the upgrade process will be to upgrade the version of a Rook pod, verify the pod is running with the new version, then verify that the overall cluster health is still in a good state.
In this guide, we will be upgrading a live Rook cluster running `v0.7.0` to the next available version of `v0.8`. Until the `v0.8` release is completed, we will instead use the latest `v0.7` tag such as `v0.7.0-27.gbfc8ec6`.
In this guide, we will be upgrading a live Rook cluster running `v0.7.1` to the next available version of `v0.8`. Until the `v0.8` release is completed, we will instead use the latest `v0.7` tag such as `v0.7.0-280.g41829b1`.
Let's get started!
@@ -322,6 +324,19 @@ kubectl -n rook delete daemonset rook-ceph-osd
kubectl -n rook delete replicaset rook-ceph-osd-<node>
```
Now that the 0.7 OSD pods have been deleted, soon you should see the new 0.8 OSD pods in the `Running` state. It may take a few minutes until Kubernetes retries starting the pods.
Once they have all been started, you will see the pods running. If they do not start in a timely manner, you can delete the pods and their K8s deployment will immediately create new pods to replace them.
```
$ kubectl -n rook get pod -l app=rook-ceph-osd
NAME READY STATUS RESTARTS AGE
rook-ceph-osd-id-0-5675d6f5f8-r5b2g 1/1 Running 6 6m
rook-ceph-osd-id-1-69cc6bd8f6-59tcn 1/1 Running 6 6m
rook-ceph-osd-id-2-74b7cf67c5-mtl92 1/1 Running 6 6m
rook-ceph-osd-id-3-757b845567-bk259 1/1 Running 6 6m
rook-ceph-osd-id-4-6cccb5f7d8-wxl2w 1/1 Running 6 6m
rook-ceph-osd-id-5-5b8598cc9f-2pnfb 1/1 Running 6 6m
```
### Ceph Manager
The ceph manager has been renamed in 0.8. The new manager will be started automatically by the operator.
The old manager and its secret can simply be deleted.
@@ -334,7 +349,7 @@ kubectl -n rook delete secret rook-ceph-mgr0
### Legacy Custom Resource Definitions (CRDs)
During this upgrade process, the new Ceph operator automatically migrated legacy custom resources to their new `rook.io/v1alpha2` and `ceph.rook.io/v1alpha1` types.
During this upgrade process, the new Ceph operator automatically migrated legacy custom resources to their new `rook.io/v1alpha2` and `ceph.rook.io/v1beta1` types.
First confirm that there are no remaining legacy CRD instances:
```bash
@@ -355,6 +370,12 @@ kubectl delete crd pools.rook.io
kubectl delete crd volumeattachments.rook.io
```
After the legacy CRDs are deleted, you will see some errors in the operator log since the operator was trying to watch the legacy CRDs.
To run with a clean log, restart the operator again by deleting the pod.
```bash
kubectl -n rook-system delete pod -l app=rook-ceph-operator
```
### Optional Components
If you have optionally installed either [object storage](./object.md) or a [shared file system](./filesystem.md) in your Rook cluster, the sections below will provide guidance on how to update them as well.
They are both managed by deployments, which we have already covered in this guide, so the instructions will be brief.