From 898528f99ab3b7149933fb7c58e4470fd61696f9 Mon Sep 17 00:00:00 2001 From: Alexander Trost Date: Sun, 1 Apr 2018 18:31:47 +0200 Subject: [PATCH] docs: Combined flexvolume docs docs: Flexvolume config: Added Atomic host to the list docs: Added empty new line to all doc files Signed-off-by: Alexander Trost --- Documentation/README.md | 4 ++-- Documentation/common-issues.md | 34 +++++++++++++-------------- Documentation/crds.md | 2 +- Documentation/direct-tools.md | 4 ++-- Documentation/filesystem.md | 2 +- Documentation/flexvolume.md | 42 ++++++++++++---------------------- Documentation/helm-operator.md | 6 ++--- Documentation/helm.md | 2 +- Documentation/monitoring.md | 2 +- Documentation/object.md | 4 ++-- Documentation/pool-crd.md | 8 +++---- Documentation/quickstart.md | 8 +++---- Documentation/rbac.md | 18 +++++++-------- Documentation/storage.md | 2 +- Documentation/teardown.md | 8 +++---- Documentation/toolbox.md | 4 ++-- Documentation/tools.md | 2 +- Documentation/upgrade.md | 12 +++++----- 18 files changed, 75 insertions(+), 89 deletions(-) diff --git a/Documentation/README.md b/Documentation/README.md index 5aa3587a0..b279bcd2f 100644 --- a/Documentation/README.md +++ b/Documentation/README.md @@ -6,7 +6,7 @@ Rook turns distributed storage software into a self-managing, self-scaling, and Rook integrates deeply into cloud native environments leveraging extension points and providing a seamless experience for scheduling, lifecycle management, resource management, security, monitoring, and user experience. -Rook is currently in alpha state and has focused initially on orchestrating Ceph on top of Kubernetes. Ceph is a distributed storage system that provides file, block and object storage and is deployed in large scale production clusters. Rook plans to add support for other storage systems beyond Ceph in future releases. +Rook is currently in alpha state and has focused initially on orchestrating Ceph on top of Kubernetes. Ceph is a distributed storage system that provides file, block and object storage and is deployed in large scale production clusters. Rook plans to add support for other storage systems beyond Ceph in future releases. ## Getting Started @@ -30,7 +30,7 @@ and healthy. The Rook operator is a simple container that has all that is needed to bootstrap and monitor the storage cluster. The operator will start and monitor [ceph monitor pods](https://github.com/rook/rook/blob/master/design/mon-health.md) and a daemonset for the OSDs, which provides basic -RADOS storage. The operator manages CRDs for pools, object stores (S3/Swift), and file systems by initializing the pods and other artifacts necessary to +RADOS storage. The operator manages CRDs for pools, object stores (S3/Swift), and file systems by initializing the pods and other artifacts necessary to run the services. The operator will monitor the storage daemons to ensure the cluster is healthy. Ceph mons will be started or failed over when necessary, and diff --git a/Documentation/common-issues.md b/Documentation/common-issues.md index 06354de89..5ef01bc97 100644 --- a/Documentation/common-issues.md +++ b/Documentation/common-issues.md @@ -25,7 +25,7 @@ There are two main categories of information you will need to investigate issues ## Kubernetes Tools Kubernetes status is the first line of investigating when something goes wrong with the cluster. Here are a few artifacts that are helpful to gather: -- Rook pod status: +- Rook pod status: - `kubectl get pod -n rook -o wide` - `kubectl get pod -n rook-system -o wide` - Logs for Rook pods @@ -38,12 +38,12 @@ Kubernetes status is the first line of investigating when something goes wrong w - Other Rook artifacts: - The monitors that are expected to be in quorum: `kubectl -n rook get configmap rook-ceph-mon-endpoints -o yaml | grep data` - More artifacts in the `rook` namespace: `kubectl -n rook get all` - + ## Ceph Tools After you verify the basic health of the running pods, next you will want to run Ceph tools for status of the storage components. There are two ways to run the Ceph tools, either in the Rook toolbox or inside other Rook pods that are already running. ### Tools in the Rook Toolbox - The [rook-tools pod](./toolbox.md) is a one-stop shop for both Ceph tools and other troubleshooting tools. Once the pod is up and running one connect to the pod to execute Ceph commands to evaluate that current state of the cluster. + The [rook-tools pod](./toolbox.md) is a one-stop shop for both Ceph tools and other troubleshooting tools. Once the pod is up and running one connect to the pod to execute Ceph commands to evaluate that current state of the cluster. ```bash kubectl exec -it rook-tools bash ``` @@ -276,12 +276,12 @@ If the first mon is not detected healthy, the operator will continue to check un - The mon pod is failing to start ### Operator fails to connect to the mon -First look at the logs of the operator to confirm if it is able to connect to the mons. +First look at the logs of the operator to confirm if it is able to connect to the mons. ``` $ kubectl -n rook-system logs -l app=rook-operator ``` -Likely you will see an error similar to the following that the operator is timing out when connecting to the mon. The last command is `ceph mon_status`, +Likely you will see an error similar to the following that the operator is timing out when connecting to the mon. The last command is `ceph mon_status`, followed by a timeout message five minutes later. ``` 2018-01-21 21:47:32.375833 I | exec: Running command: ceph mon_status --cluster=rook --conf=/var/lib/rook/rook/rook.config --keyring=/var/lib/rook/rook/client.admin.keyring --format json --out-file /tmp/442263890 @@ -295,12 +295,12 @@ followed by a timeout message five minutes later. The error would appear to be an authentication error, but it is misleading. The real issue is a timeout. ### Solution -If you see the timeout in the operator log, verify if the mon pod is running (see the next section). -If the mon pod is running, check the network connectivity between the operator pod and the mon pod. +If you see the timeout in the operator log, verify if the mon pod is running (see the next section). +If the mon pod is running, check the network connectivity between the operator pod and the mon pod. A common issue is that the CNI is not configured correctly. ### Failing mon pod -Second we need to verify if the mon pod started successfully. +Second we need to verify if the mon pod started successfully. ``` $ kubectl -n rook get pod -l app=rook-ceph-mon @@ -317,15 +317,15 @@ $ kubectl -n rook describe pod -l mon=rook-ceph-mon0 ... Last State: Terminated Reason: Error - Message: The keyring does not match the existing keyring in /var/lib/rook/rook-ceph-mon0/data/keyring. + Message: The keyring does not match the existing keyring in /var/lib/rook/rook-ceph-mon0/data/keyring. You may need to delete the contents of dataDirHostPath on the host from a previous deployment. ... ``` ### Solution -This is a common problem reinitializing the Rook cluster when the local directory used for persistence has **not** been purged. -This directory is the `dataDirHostPath` setting in the cluster CRD and is typically set to `/var/lib/rook`. -To fix the issue you will need to delete all components of Rook and then delete the contents of `/var/lib/rook` (or the directory specified by `dataDirHostPath`) on each of the hosts in the cluster. +This is a common problem reinitializing the Rook cluster when the local directory used for persistence has **not** been purged. +This directory is the `dataDirHostPath` setting in the cluster CRD and is typically set to `/var/lib/rook`. +To fix the issue you will need to delete all components of Rook and then delete the contents of `/var/lib/rook` (or the directory specified by `dataDirHostPath`) on each of the hosts in the cluster. Then when the cluster CRD is applied to start a new cluster, the rook-operator should start all the pods as expected. **Important: Deleting the `dataDirHostPath` folder is destructive to the storage. Only delete the folder if you are trying to permanently purge the Rook cluster.** @@ -351,7 +351,7 @@ old state. Looking at the OSD pod logs you will see an error about the file alre ``` $ kubectl -n rook logs rook-ceph-osd-fl8fs ... -2017-10-31 20:13:11.187106 I | mkfs-osd0: 2017-10-31 20:13:11.186992 7f0059d62e00 -1 bluestore(/var/lib/rook/osd0) _read_fsid unparsable uuid +2017-10-31 20:13:11.187106 I | mkfs-osd0: 2017-10-31 20:13:11.186992 7f0059d62e00 -1 bluestore(/var/lib/rook/osd0) _read_fsid unparsable uuid 2017-10-31 20:13:11.187208 I | mkfs-osd0: 2017-10-31 20:13:11.187026 7f0059d62e00 -1 bluestore(/var/lib/rook/osd0) _setup_block_symlink_or_file failed to create block symlink to /dev/disk/by-partuuid/651153ba-2dfc-4231-ba06-94759e5ba273: (17) File exists 2017-10-31 20:13:11.187233 I | mkfs-osd0: 2017-10-31 20:13:11.187038 7f0059d62e00 -1 bluestore(/var/lib/rook/osd0) mkfs failed, (17) File exists 2017-10-31 20:13:11.187254 I | mkfs-osd0: 2017-10-31 20:13:11.187042 7f0059d62e00 -1 OSD::mkfs: ObjectStore::mkfs failed with error (17) File exists @@ -359,9 +359,9 @@ $ kubectl -n rook logs rook-ceph-osd-fl8fs ``` ## Solution -If the error is from the file that already exists, this is a common problem reinitializing the Rook cluster when the local directory used for persistence has **not** been purged. -This directory is the `dataDirHostPath` setting in the cluster CRD and is typically set to `/var/lib/rook`. -To fix the issue you will need to delete all components of Rook and then delete the contents of `/var/lib/rook` (or the directory specified by `dataDirHostPath`) on each of the hosts in the cluster. +If the error is from the file that already exists, this is a common problem reinitializing the Rook cluster when the local directory used for persistence has **not** been purged. +This directory is the `dataDirHostPath` setting in the cluster CRD and is typically set to `/var/lib/rook`. +To fix the issue you will need to delete all components of Rook and then delete the contents of `/var/lib/rook` (or the directory specified by `dataDirHostPath`) on each of the hosts in the cluster. Then when the cluster CRD is applied to start a new cluster, the rook-operator should start all the pods as expected. # Node hangs after reboot @@ -373,7 +373,7 @@ Then when the cluster CRD is applied to start a new cluster, the rook-operator s ## Solution The node needs to be [drained](https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/) before reboot. After the successful drain, the node can be rebooted as usual. -Because `kubectl drain` command automatically marks the node as unschedulable (`kubectl cordon` effect), the node needs to be uncordoned once it's back online. +Because `kubectl drain` command automatically marks the node as unschedulable (`kubectl cordon` effect), the node needs to be uncordoned once it's back online. Drain the node: ``` diff --git a/Documentation/crds.md b/Documentation/crds.md index ab0ade4a2..79bb51753 100644 --- a/Documentation/crds.md +++ b/Documentation/crds.md @@ -10,4 +10,4 @@ has its own CRD defined. - [Cluster](cluster-crd.md): A Rook cluster provides the basis of the storage platform to serve block, object stores, and shared file systems. - [Pool](pool-crd.md): A pool manages the backing store for a block store. Pools are also used internally by object and file stores. - [Object Store](object-store-crd.md): An object store exposes storage with an S3-compatible interface. -- [File System](filesystem-crd.md): A file system provides shared storage for multiple Kubernetes pods. \ No newline at end of file +- [File System](filesystem-crd.md): A file system provides shared storage for multiple Kubernetes pods. diff --git a/Documentation/direct-tools.md b/Documentation/direct-tools.md index cf6d6d3dc..c9247372e 100644 --- a/Documentation/direct-tools.md +++ b/Documentation/direct-tools.md @@ -6,7 +6,7 @@ indent: true # Direct Tools -Rook is designed with Kubernetes design principles from the ground up. This topic is going to escape the bounds of Kubernetes storage and show you how to +Rook is designed with Kubernetes design principles from the ground up. This topic is going to escape the bounds of Kubernetes storage and show you how to use block and file storage directly from a pod without any of the Kubernetes magic. The purpose of this topic is to help you quickly test a new configuration, although it is not meant to be used in production. All of the benefits of Kubernetes storage including failover, detach, and attach will not be available. If your pod dies, your mount will die with it. @@ -59,7 +59,7 @@ rbd unmap /dev/rbd0 ## Shared Filesystem Tools -After you have created a file system as described in the [Shared Filesystem](filesystem.md) topic, you can mount the filesystem from multiple pods. +After you have created a file system as described in the [Shared Filesystem](filesystem.md) topic, you can mount the filesystem from multiple pods. The the other topic you may have mounted the filesystem already in the registry pod. Now we will mount the same file system in the toolbox pod. This is just a simple way to validate the Ceph file system and is not recommended for production Kubernetes pods. diff --git a/Documentation/filesystem.md b/Documentation/filesystem.md index 238a2a2c6..020908209 100644 --- a/Documentation/filesystem.md +++ b/Documentation/filesystem.md @@ -62,7 +62,7 @@ $ ceph status ## Consume the Shared File System: K8s Registry Sample -As an example, we will start the kube-registry pod with the shared file system as the backing store. +As an example, we will start the kube-registry pod with the shared file system as the backing store. Save the following spec as `kube-registry.yaml`: ```yaml diff --git a/Documentation/flexvolume.md b/Documentation/flexvolume.md index 10b2b2dfc..da4af0fd6 100644 --- a/Documentation/flexvolume.md +++ b/Documentation/flexvolume.md @@ -3,50 +3,36 @@ title: Flex Volume Configuration weight: 12 indent: true --- - # Flex Volume Configuration - Rook uses [FlexVolume](https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md) to integrate with Kubernetes for performing storage operations. In some operating systems where Kubernetes is deployed, the [default Flexvolume plugin directory](https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md#prerequisites) (the directory where flexvolume drivers are installed) is **read-only**. -This is the case for Kubernetes deployments on CoreOS and Rancher. -In these environments, the Kubelet needs to be told to use a different flexvolume plugin directory that is accessible and writeable. -To do this, you will need to first add the `--volume-plugin-dir` flag to the Kubelet and then restart the Kubelet process. +This is the case for Kubernetes deployments on: +* [ContainerLinux](https://coreos.com/os/docs/latest/) (previously named CoreOS) +* [Rancher](http://rancher.com/) +* [Atomic](https://www.projectatomic.io/) + +In these environments, the Kubelet needs to be told to use a different flexvolume plugin directory that is accessible and writeable (`rw`). +To do this, you will need to first add the `--volume-plugin-dir` flag to the Kubelet and then restart the Kubelet process. These steps need to be carried out on **all nodes** in your cluster. -## CoreOS Container Linux - -In CoreOS, our recommendation is to specify the flag as shown below: +Please refer to the below sections for more information on your specific platform and/or Kubernetes version used. ```bash --volume-plugin-dir=/var/lib/kubelet/volumeplugins ``` - Restart Kubelet in order for this change to take effect. -In Kubernetes 1.9.x, you must provide this directory when deploying the [rook-operator](/cluster/examples/kubernetes/rook-operator.yaml) by setting the environment variable `FLEXVOLUME_DIR_PATH`. For example: +## For Kubernetes >= 1.9.x +In Kubernetes 1.9.x, you must provide the above set Flexvolume plugin directory when deploying the [rook-operator](/cluster/examples/kubernetes/rook-operator.yaml) by setting the environment variable `FLEXVOLUME_DIR_PATH`. For example: ```yaml - name: FLEXVOLUME_DIR_PATH value: "/var/lib/kubelet/volumeplugins" ``` +(In the `rook-operator.yaml` manifest replace `` with the path) -## Rancher +## For Rancher Rancher provides an easy way to configure Kubelet. This flag can be provided to the Kubelet configuration template at deployment time or by using the `up to date` feature if Kubernetes is already deployed. -To configure Flexvolume in Rancher, specify this Kubelet flag as shown below: - -```bash ---volume-plugin-dir=/var/lib/kubelet/volumeplugins -``` - -Restart Kubelet in order for this change to take effect. - -In Kubernetes 1.9.x, you must provide this directory when deploying the [rook-operator](/cluster/examples/kubernetes/rook-operator.yaml) by setting the environment variable `FLEXVOLUME_DIR_PATH`. For example: -```yaml -- name: FLEXVOLUME_DIR_PATH - value: "/var/lib/kubelet/volumeplugins" -``` - -## Tectonic - -Follow [these instructions](tectonic.md) to configure Rook on Tectonic. +## For Tectonic +Follow [these instructions](tectonic.md) to configure the Flexvolume plugin for Rook on Tectonic during ContainerLinux node ignition file provisioning. diff --git a/Documentation/helm-operator.md b/Documentation/helm-operator.md index 3be0a59b2..73178ce18 100644 --- a/Documentation/helm-operator.md +++ b/Documentation/helm-operator.md @@ -33,7 +33,7 @@ kubectl -n kube-system patch deploy/tiller-deploy -p '{"spec": {"template": {"sp ## Installing -The Rook Operator helm chart will install the basic components necessary to create a storage platform for your Kubernetes cluster. +The Rook Operator helm chart will install the basic components necessary to create a storage platform for your Kubernetes cluster. After the helm chart is installed, you will need to [create a Rook cluster](quickstart.md#create-a-rook-cluster). The `helm install` command deploys rook on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. @@ -41,7 +41,7 @@ The `helm install` command deploys rook on the Kubernetes cluster in the default Rook currently publishes builds to the `alpha` and `master` channels. In the future, `beta` and `stable` will also be available. ### Alpha -The alpha channel is the most recent release of Rook that is considered ready for testing by the community. +The alpha channel is the most recent release of Rook that is considered ready for testing by the community. ```console helm repo add rook-alpha https://charts.rook.io/alpha helm install rook-alpha/rook @@ -102,7 +102,7 @@ The following tables lists the configurable parameters of the rook-operator char ### Command Line -You can pass the settings with helm command line parameters. Specify each parameter using the +You can pass the settings with helm command line parameters. Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example, the following command will install rook where RBAC is not enabled. ```console diff --git a/Documentation/helm.md b/Documentation/helm.md index a4e4882e1..ac3f7f04c 100644 --- a/Documentation/helm.md +++ b/Documentation/helm.md @@ -5,7 +5,7 @@ weight: 50 # Helm Charts -Rook has published a Helm chart for the [operator](helm-operator.md). Other Helm charts will also be developed for each of the +Rook has published a Helm chart for the [operator](helm-operator.md). Other Helm charts will also be developed for each of the Pool, Object Store, and Filesystem CRDs. - [Rook Operator](helm-operator.md): Installs the Rook Operator and Agents necessary to run a storage cluster diff --git a/Documentation/monitoring.md b/Documentation/monitoring.md index 6bccf3802..ffcf55f05 100644 --- a/Documentation/monitoring.md +++ b/Documentation/monitoring.md @@ -70,7 +70,7 @@ Click on the `Execute` button. Below the `Execute` button, ensure the `Graph` tab is selected and you should now see a graph of your chosen metric over time. -![Prometheus Execute Metric](media/prometheus-metric-cursor-graph.png) +![Prometheus Execute Metric](media/prometheus-metric-cursor-graph.png) ## Prometheus Consoles diff --git a/Documentation/object.md b/Documentation/object.md index 91aeb2627..6ab676365 100644 --- a/Documentation/object.md +++ b/Documentation/object.md @@ -33,7 +33,7 @@ spec: codingChunks: 1 gateway: type: s3 - sslCertificateRef: + sslCertificateRef: port: 80 securePort: instances: 1 @@ -68,7 +68,7 @@ The object store is now available by using the creds of `rook-user`. Take note o ## Consume the Object Storage -Use an S3 compatible client to create a bucket in the object store. +Use an S3 compatible client to create a bucket in the object store. This section will allow you to test connecting to the object store and uploading and downloading from it. The `s3cmd` tool is included in the [Rook toolbox](toolbox.md) pod to simplify your testing. Run the following commands after you have connected to the toolbox. diff --git a/Documentation/pool-crd.md b/Documentation/pool-crd.md index 22078e5c7..e6bec7747 100644 --- a/Documentation/pool-crd.md +++ b/Documentation/pool-crd.md @@ -40,8 +40,8 @@ spec: - `erasureCoded`: Settings for an erasure-coded pool. If specified, `replicated` settings must not be specified. - `dataChunks`: Number of data chunks per object in an erasure coded storage pool - `codingChunks`: Number of coding chunks per object in an erasure coded storage pool -- `failureDomain`: The failure domain across which the replicas or chunks of data will be spread. Possible values are `osd` or `host`, -with the default of `host`. For example, if you have replication of size `3` and the failure domain is `host`, all three copies of the data will be -placed on osds that are found on unique hosts. In that case you would be guaranteed to tolerate the failure of two hosts. If the failure domain were `osd`, +- `failureDomain`: The failure domain across which the replicas or chunks of data will be spread. Possible values are `osd` or `host`, +with the default of `host`. For example, if you have replication of size `3` and the failure domain is `host`, all three copies of the data will be +placed on osds that are found on unique hosts. In that case you would be guaranteed to tolerate the failure of two hosts. If the failure domain were `osd`, you would be able to tolerate the loss of two devices. Similarly for erasure coding, the data and coding chunks would be spread across the requested failure domain. -- `crushRoot`: The root in the crush map to be used by the pool. If left empty or unspecified, the default root will be used. Creating a crush hierarchy for the OSDs currently requires the Rook toolbox to run the Ceph tools described [here](http://docs.ceph.com/docs/master/rados/operations/crush-map/#modifying-the-crush-map). +- `crushRoot`: The root in the crush map to be used by the pool. If left empty or unspecified, the default root will be used. Creating a crush hierarchy for the OSDs currently requires the Rook toolbox to run the Ceph tools described [here](http://docs.ceph.com/docs/master/rados/operations/crush-map/#modifying-the-crush-map). diff --git a/Documentation/quickstart.md b/Documentation/quickstart.md index 41c17cd07..bd47cb35d 100644 --- a/Documentation/quickstart.md +++ b/Documentation/quickstart.md @@ -9,7 +9,7 @@ Welcome to Rook! We hope you have a great experience installing the Rook storage in your cluster. If you have any questions along the way, please don't hesitate to ask us in our [Slack channel](https://Rook-io.slack.com). This guide will walk you through the basic setup of a Rook cluster. This will enable you to consume block, object, and file storage -from other pods running in your cluster. +from other pods running in your cluster. ## Minimum Version @@ -37,7 +37,7 @@ After the cluster is running, you can create [block, object, or file](#storage) ## Deploy the Rook Operator -The first step is to deploy the Rook system components, which include the Rook agent running on each node in your cluster as well as Rook operator pod. +The first step is to deploy the Rook system components, which include the Rook agent running on each node in your cluster as well as Rook operator pod. ```bash cd cluster/examples/kubernetes @@ -64,8 +64,8 @@ For Kubernetes 1.6, it is also necessary to pass the `--enable-controller-attach ## Create a Rook Cluster -Now that the Rook operator and agent pods are running, we can create the Rook cluster. For the cluster to survive reboots, -make sure you set the `dataDirHostPath` property. For more settings, see the documentation on [configuring the cluster](cluster-crd.md). +Now that the Rook operator and agent pods are running, we can create the Rook cluster. For the cluster to survive reboots, +make sure you set the `dataDirHostPath` property. For more settings, see the documentation on [configuring the cluster](cluster-crd.md). Save the cluster spec as `rook-cluster.yaml`: diff --git a/Documentation/rbac.md b/Documentation/rbac.md index 0b06dbcda..8e4d71e9a 100644 --- a/Documentation/rbac.md +++ b/Documentation/rbac.md @@ -20,12 +20,12 @@ Binding individual certificate to ClusterRole `cluster-admin` is revocable by de ## RBAC for PodSecurityPolicies If you have activated the [PodSecurityPolicy Admission Controller](https://kubernetes.io/docs/admin/admission-controllers/#podsecuritypolicy) and thus are -using [PodSecurityPolicies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/), you will require additional `(Cluster)RoleBindings` +using [PodSecurityPolicies](https://kubernetes.io/docs/concepts/policy/pod-security-policy/), you will require additional `(Cluster)RoleBindings` for the different `ServiceAccounts` Rook uses to start the Rook Storage Pods. - + **Note**: You do not have to perform these steps if you do not have the `PodSecurityPolicy` Admission Controller activated! -##### PodSecurityPolicy +##### PodSecurityPolicy You need one `PodSecurityPolicy` that allows privileged `Pod` execution. Here is an example: @@ -54,7 +54,7 @@ spec: ``` **Hint**: Allowing `hostNetwork` usage is required when using `hostNetwork: true` in the Cluster `CustomResourceDefinition`! -You are then also required to allow the usage of `hostPorts` in the `PodSecurityPolicy`. The given port range is a minimal +You are then also required to allow the usage of `hostPorts` in the `PodSecurityPolicy`. The given port range is a minimal working recommendation for rook: ```yaml hostPorts: @@ -86,9 +86,9 @@ rules: - privileged verbs: - use - + ``` -and +and ```yaml apiVersion: v1 kind: Namespace @@ -108,13 +108,13 @@ subjects: - kind: ServiceAccount name: rook-agent namespace: rook-system -``` +``` Save these definitions to one or multiple yaml files and create them by executing `kubectl apply -f .yaml` You will also require two more `RoleBindings` for each Rook Cluster you deploy: Create these two `RoleBindings` in the Namespace you plan to deploy your Rook Cluster into (default is "rook" namespace): - + ```yaml apiVersion: v1 kind: Namespace @@ -150,4 +150,4 @@ subjects: - kind: ServiceAccount name: rook-ceph-osd namespace: rook -``` +``` diff --git a/Documentation/storage.md b/Documentation/storage.md index aa4225864..6fb2f52dd 100644 --- a/Documentation/storage.md +++ b/Documentation/storage.md @@ -8,4 +8,4 @@ weight: 20 Rook provides three types of storage to the Kubernetes cluster. - [Block Storage](block.md): Mount storage to a single pod - [Object Storage](object.md): Expose an S3 API to the storage cluster for applications to put and get data that is accessible from inside or outside the Kubernetes cluster -- [Shared File System](filesystem.md): Mount a file system that can be shared across multiple pods +- [Shared File System](filesystem.md): Mount a file system that can be shared across multiple pods diff --git a/Documentation/teardown.md b/Documentation/teardown.md index 67b405d69..d8a326b3b 100644 --- a/Documentation/teardown.md +++ b/Documentation/teardown.md @@ -17,7 +17,7 @@ If you see issues tearing down the cluster, see the [Troubleshooting](#troublesh If you are tearing down a cluster frequently for development purposes, it is instead recommended to use an environment such as Minikube that can easily be restarted without worrying about any of these steps. ## Delete the Block and File artifacts -First you will need to clean up the resources created on top of the Rook cluster. +First you will need to clean up the resources created on top of the Rook cluster. These commands will clean up the resources from the [block](block.md#teardown) and [file](filesystem.md#teardown) walkthroughs (unmount volumes, delete volume claims, etc). If you did not complete those parts of the walkthrough, you can skip these instructions: ```console @@ -40,7 +40,7 @@ Verify that the cluster CRD has been deleted before continuing to the next step. kubectl -n rook get cluster ``` -## Delete the Operator and Agent +## Delete the Operator and Agent This will begin the process of all cluster resources being cleaned up, after which you can delete the rest of the deployment with the following: ```console kubectl delete thirdpartyresources cluster.rook.io pool.rook.io objectstore.rook.io filesystem.rook.io volumeattachment.rook.io # ignore errors if on K8s 1.7+ @@ -57,7 +57,7 @@ kubectl delete namespace rook ``` ## Delete the data on hosts -IMPORTANT: The final cleanup step requires deleting files on each host in the cluster. All files under the `dataDirHostPath` property specified in the cluster CRD will need to be deleted. Otherwise, inconsistent state will remain when a new cluster is started. +IMPORTANT: The final cleanup step requires deleting files on each host in the cluster. All files under the `dataDirHostPath` property specified in the cluster CRD will need to be deleted. Otherwise, inconsistent state will remain when a new cluster is started. Connect to each machine and delete `/var/lib/rook`, or the path specified by the `dataDirHostPath`. @@ -96,4 +96,4 @@ This will open a text editor (usually `vi`) to allow you to edit the CRD. Look f - cluster.rook.io ``` -Now save the changes and exit the editor. Within a few seconds you should see that the cluster CRD has been deleted and will no longer block other cleanup such as deleting the `rook` namespace. \ No newline at end of file +Now save the changes and exit the editor. Within a few seconds you should see that the cluster CRD has been deleted and will no longer block other cleanup such as deleting the `rook` namespace. diff --git a/Documentation/toolbox.md b/Documentation/toolbox.md index 10a1a7afd..df21e5fad 100644 --- a/Documentation/toolbox.md +++ b/Documentation/toolbox.md @@ -6,7 +6,7 @@ indent: true # Rook Toolbox The Rook toolbox is a container with common tools used for rook debugging and testing. -The toolbox is based on Ubuntu, so more tools of your choosing can be easily installed with `apt-get`. +The toolbox is based on Ubuntu, so more tools of your choosing can be easily installed with `apt-get`. ## Running the Toolbox in Kubernetes @@ -92,4 +92,4 @@ kubectl -n rook delete pod rook-tools ``` ## Troubleshooting without the Toolbox -The Ceph tools will commonly be the only tools needed to troubleshoot a cluster. In that case, you can connect to any of the rook pods and execute the ceph commands in the same way that you would in the toolbox pod. For example, you can connect to the mon, osd, or even the operator pod to execute commands such as `ceph status`. +The Ceph tools will commonly be the only tools needed to troubleshoot a cluster. In that case, you can connect to any of the rook pods and execute the ceph commands in the same way that you would in the toolbox pod. For example, you can connect to the mon, osd, or even the operator pod to execute commands such as `ceph status`. diff --git a/Documentation/tools.md b/Documentation/tools.md index 6e30032b9..5f0a2e5ef 100644 --- a/Documentation/tools.md +++ b/Documentation/tools.md @@ -8,4 +8,4 @@ weight: 70 Rook provides a number of tools to help you manage your cluster. - [Toolbox](toolbox.md): A pod from which you can run all of the tools to troubleshoot the storage cluster - [Advanced Configuration](advanced-configuration.md): Tips and tricks for configuring for cluster -- [Common Issues](common-issues.md): Common issues and their potential solutions \ No newline at end of file +- [Common Issues](common-issues.md): Common issues and their potential solutions diff --git a/Documentation/upgrade.md b/Documentation/upgrade.md index ed49ed93d..0556ec1f5 100644 --- a/Documentation/upgrade.md +++ b/Documentation/upgrade.md @@ -16,7 +16,7 @@ We welcome feedback and opening issues! ## Supported Versions The supported version for this upgrade guide is **from an 0.7 release to the latest builds**. Until 0.8 is released, the latest builds are labeled such as `v0.7.0-27.gbfc8ec6`. Build-to-build upgrades are not guaranteed to work. This guide is to test upgrades only between the official releases. -For a guide to upgrade previous versions of Rook, please refer to the version of documentation for those releases. +For a guide to upgrade previous versions of Rook, please refer to the version of documentation for those releases. - [Upgrade 0.6 to 0.7](https://rook.io/docs/rook/v0.7/upgrade.html) - [Upgrade 0.5 to 0.6](https://rook.io/docs/rook/v0.6/upgrade.html) @@ -80,12 +80,12 @@ Let's look at some sample output and review some of the details: cluster: id: fe7ae378-dc77-46a1-801b-de05286aa78e health: HEALTH_OK - + services: mon: 3 daemons, quorum rook-ceph-mon0,rook-ceph-mon1,rook-ceph-mon2 mgr: rook-ceph-mgr0(active) osd: 1 osds: 1 up, 1 in - + data: pools: 1 pools, 100 pgs objects: 0 objects, 0 bytes @@ -125,7 +125,7 @@ Any pod that is using a Rook volume should also remain healthy: ## Upgrade Process The general flow of the upgrade process will be to upgrade the version of a Rook pod, verify the pod is running with the new version, then verify that the overall cluster health is still in a good state. -In this guide, we will be upgrading a live Rook cluster running `v0.7.0` to the next available version of `v0.8`. Until the `v0.8` release is completed, we will instead use the latest `v0.7` tag such as `v0.7.0-27.gbfc8ec6`. +In this guide, we will be upgrading a live Rook cluster running `v0.7.0` to the next available version of `v0.8`. Until the `v0.8` release is completed, we will instead use the latest `v0.7` tag such as `v0.7.0-27.gbfc8ec6`. Let's get started! @@ -173,7 +173,7 @@ kubectl -n rook exec -it rook-tools -- ceph status ### Toolbox The toolbox pod runs the tools we will use during the upgrade for cluster status. The toolbox is not expected to contain any state, -so we will delete the old pod and start the new toolbox. +so we will delete the old pod and start the new toolbox. ```bash kubectl -n rook delete pod rook-tools ``` @@ -299,7 +299,7 @@ kubectl -n rook get pod -l app=rook-ceph-mds -o jsonpath='{range .items[*]}{.met At this point, your Rook cluster should be fully upgraded to running version `rook/rook:v0.7.0-27.gbfc8ec6` and the cluster should be healthy according to the steps in the [health verification section](#health-verification). ## Upgrading Kubernetes -Rook cluster installations on Kubernetes prior to version 1.7.x, use [ThirdPartyResource](https://kubernetes.io/docs/tasks/access-kubernetes-api/extend-api-third-party-resource/) that have been deprecated as of 1.7 and removed in 1.8. If upgrading your Kubernetes cluster Rook TPRs have to be migrated to CustomResourceDefinition (CRD) following [Kubernetes documentation](https://kubernetes.io/docs/tasks/access-kubernetes-api/migrate-third-party-resource/). Rook TPRs that require migration during upgrade are: +Rook cluster installations on Kubernetes prior to version 1.7.x, use [ThirdPartyResource](https://kubernetes.io/docs/tasks/access-kubernetes-api/extend-api-third-party-resource/) that have been deprecated as of 1.7 and removed in 1.8. If upgrading your Kubernetes cluster Rook TPRs have to be migrated to CustomResourceDefinition (CRD) following [Kubernetes documentation](https://kubernetes.io/docs/tasks/access-kubernetes-api/migrate-third-party-resource/). Rook TPRs that require migration during upgrade are: - Cluster - Pool - ObjectStore