Files
my-rook-config/Documentation/ceph-prerequisites.md
T
Travis Nielsen a251959061 docs: restructure prerequisites and clarify ceph docs
The docs were in need of some restructuring wrt prerequisites. They
were mingled with general k8s and Ceph requirements. For clarity,
we need to separate the ceph prereqs from general k8s requirements.

There is now only a single prereqs page. All other topics should be
found under other sections, such as specific to Ceph. Having subtopics
under the general prereqs caused users to frequently not even notice
the main prereqs topic.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-16 15:15:44 -06:00

3.2 KiB

title, weight, indent
title weight indent
Prerequisites 2010 true

Ceph Prerequisites

LVM package

Ceph OSDs have a dependency on LVM in the following scenarios:

  • OSDs are created on raw devices or partitions
  • If encryption is enabled (encryptedDevice: true in the cluster CR)
  • A metadata device is specified

LVM is not required for OSDs in these scenarios:

  • Creating OSDs on PVCs using the storageClassDeviceSets

If LVM is required for your scenario, LVM needs to be available on the hosts where OSDs will be running. Some Linux distributions do not ship with the lvm2 package. This package is required on all storage nodes in your k8s cluster to run Ceph OSDs. Without this package even though Rook will be able to successfully create the Ceph OSDs, when a node is rebooted the OSD pods running on the restarted node will fail to start. Please install LVM using your Linux distribution's package manager. For example:

CentOS:

sudo yum install -y lvm2

Ubuntu:

sudo apt-get install -y lvm2

Ceph Flexvolume Configuration

NOTE This configuration is only needed when using the FlexVolume driver (required for Kubernetes 1.12 or earlier). The Ceph-CSI RBD driver or the Ceph-CSI CephFS driver are recommended for Kubernetes 1.13 and newer, making FlexVolume configuration redundant.

If you want to configure volumes with the Flex driver instead of CSI, the Rook agent requires setup as a Flex volume plugin to manage the storage attachments in your cluster. See the Flex Volume Configuration topic to configure your Kubernetes deployment to load the Rook volume plugin.

Extra agent mounts

On certain distributions it may be necessary to mount additional directories into the agent container. That is what the environment variable AGENT_MOUNTS is for. Also see the documentation in helm-operator on the parameter agent.mounts. The format of the variable content should be mountname1=/host/path1:/container/path1,mountname2=/host/path2:/container/path2.

Kernel

RBD

Ceph requires a Linux kernel built with the RBD module. Many Linux distributions have this module, but not all distributions. For example, the GKE Container-Optimised OS (COS) does not have RBD.

You can test your Kubernetes nodes by running modprobe rbd. If it says 'not found', you may have to rebuild your kernel or choose a different Linux distribution.

CephFS

If you will be creating volumes from a Ceph shared file system (CephFS), the recommended minimum kernel version is 4.17. If you have a kernel version less than 4.17, the requested PVC sizes will not be enforced. Storage quotas will only be enforced on newer kernels.

Kernel modules directory configuration

Normally, on Linux, kernel modules can be found in /lib/modules. However, there are some distributions that put them elsewhere. In that case the environment variable LIB_MODULES_DIR_PATH can be used to override the default. Also see the documentation in helm-operator on the parameter agent.libModulesDirPath. One notable distribution where this setting is useful would be NixOS.