forked from rook/rook
Sébastien Han
16729e0e38
We can pass bound_service_account_names with a comma separated list of service accounts. Let's do this instead of remapping new values. Earlier, we thought a single service account could be added per Vault role and we were using other variables like `VAULT_AUTH_KUBERNETES_ROOK_OPERATOR_ROLE` that we were remapping to `VAULT_AUTH_KUBERNETES_ROLE` internal for the API calls to Vault. Signed-off-by: Sébastien Han <seb@redhat.com>
228 lines
7.6 KiB
Markdown
228 lines
7.6 KiB
Markdown
---
|
|
title: Key Management System
|
|
weight: 3650
|
|
indent: true
|
|
---
|
|
|
|
# Key Management System
|
|
|
|
Rook has the ability to encrypt OSDs of clusters running on PVC via the flag (`encrypted: true`) in your `storageClassDeviceSets` [template](#pvc-based-cluster).
|
|
By default, the Key Encryption Keys (also known as Data Encryption Keys) are stored in a Kubernetes Secret.
|
|
However, if a Key Management System exists Rook is capable of using it.
|
|
|
|
The `security` section contains settings related to encryption of the cluster.
|
|
|
|
* `security`:
|
|
* `kms`: Key Management System settings
|
|
* `connectionDetails`: the list of parameters representing kms connection details
|
|
* `tokenSecretName`: the name of the Kubernetes Secret containing the kms authentication token
|
|
|
|
Supported KMS providers:
|
|
|
|
* [Vault](#vault)
|
|
## Vault
|
|
|
|
Rook supports storing OSD encryption keys in [HashiCorp Vault KMS](https://www.vaultproject.io/).
|
|
|
|
### Authentication methods
|
|
|
|
Rook support two authentication methods:
|
|
|
|
* [token-based](#token-based-authentication): a token is provided by the user and is stored in a Kubernetes Secret. It's used to
|
|
authenticate the KMS by the Rook operator. This has several pitfalls such as:
|
|
* when the token expires it must be renewed, so the secret holding it must be updated
|
|
* no token automatic rotation
|
|
* [Kubernetes Service Account](#kubernetes-based-authentication) uses [Vault Kubernetes native
|
|
authentication](https://www.vaultproject.io/docs/auth/kubernetes) mechanism and alleviate some of the limitations from the token authentication such as token automatic renewal. This method is
|
|
generally recommended over the token-based authentication.
|
|
|
|
#### Token-based authentication
|
|
|
|
When using the token-based authentication, a Kubernetes Secret must be created to hold the token.
|
|
This is governed by the `tokenSecretName` parameter.
|
|
|
|
Note: Rook supports **all** the Vault [environment variables](https://www.vaultproject.io/docs/commands#environment-variables).
|
|
|
|
The Kubernetes Secret `rook-vault-token` should contain:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: rook-vault-token
|
|
namespace: rook-ceph
|
|
data:
|
|
token: <TOKEN> # base64 of a token to connect to Vault, for example: cy5GWXpsbzAyY2duVGVoRjhkWG5Bb3EyWjkK
|
|
```
|
|
|
|
You can create a token in Vault by running the following command:
|
|
|
|
```console
|
|
vault token create -policy=rook
|
|
```
|
|
|
|
Refer to the official vault document for more details on [how to create a
|
|
token](https://www.vaultproject.io/docs/commands/token/create). For which policy to apply see the
|
|
next section.
|
|
|
|
In order for Rook to connect to Vault, you must configure the following in your `CephCluster` template:
|
|
|
|
```yaml
|
|
security:
|
|
kms:
|
|
# name of the k8s config map containing all the kms connection details
|
|
connectionDetails:
|
|
KMS_PROVIDER: vault
|
|
VAULT_ADDR: https://vault.default.svc.cluster.local:8200
|
|
VAULT_BACKEND_PATH: rook
|
|
VAULT_SECRET_ENGINE: kv
|
|
VAULT_AUTH_METHOD: token
|
|
# name of the k8s secret containing the kms authentication token
|
|
tokenSecretName: rook-vault-token
|
|
```
|
|
|
|
#### Kubernetes-based authentication
|
|
|
|
In order to use the Kubernetes Service Account authentication method, the following must be run to properly configure Vault:
|
|
|
|
```sh
|
|
ROOK_NAMESPACE=rook-ceph
|
|
ROOK_VAULT_SA=rook-vault-auth
|
|
ROOK_SYSTEM_SA=rook-ceph-system
|
|
ROOK_OSD_SA=rook-ceph-osd
|
|
VAULT_POLICY_NAME=rook
|
|
|
|
# create service account for vault to validate API token
|
|
kubectl -n "$ROOK_NAMESPACE" create serviceaccount "$ROOK_VAULT_SA"
|
|
|
|
# create the RBAC for this SA
|
|
kubectl -n "$ROOK_NAMESPACE" create clusterrolebinding vault-tokenreview-binding --clusterrole=system:auth-delegator --serviceaccount="$ROOK_NAMESPACE":"$ROOK_VAULT_SA"
|
|
|
|
# get the service account common.yaml created earlier
|
|
VAULT_SA_SECRET_NAME=$(kubectl -n "$ROOK_NAMESPACE" get sa "$ROOK_VAULT_SA" -o jsonpath="{.secrets[*]['name']}")
|
|
|
|
# Set SA_JWT_TOKEN value to the service account JWT used to access the TokenReview API
|
|
SA_JWT_TOKEN=$(kubectl -n "$ROOK_NAMESPACE" get secret "$VAULT_SA_SECRET_NAME" -o jsonpath="{.data.token}" | base64 --decode)
|
|
|
|
# Set SA_CA_CRT to the PEM encoded CA cert used to talk to Kubernetes API
|
|
SA_CA_CRT=$(kubectl -n "$ROOK_NAMESPACE" get secret "$VAULT_SA_SECRET_NAME" -o jsonpath="{.data['ca\.crt']}" | base64 --decode)
|
|
|
|
# get kubernetes endpoint
|
|
K8S_HOST=$(kubectl config view --minify --flatten -o jsonpath="{.clusters[0].cluster.server}")
|
|
|
|
# enable kubernetes auth
|
|
vault auth enable kubernetes
|
|
|
|
# To fetch the service account issuer
|
|
kubectl proxy &
|
|
proxy_pid=$!
|
|
|
|
# configure the kubernetes auth
|
|
vault write auth/kubernetes/config \
|
|
token_reviewer_jwt="$SA_JWT_TOKEN" \
|
|
kubernetes_host="$K8S_HOST" \
|
|
kubernetes_ca_cert="$SA_CA_CRT" \
|
|
issuer="$(curl --silent http://127.0.0.1:8001/.well-known/openid-configuration | jq -r .issuer)"
|
|
|
|
kill $proxy_pid
|
|
|
|
# configure a role for rook
|
|
vault write auth/kubernetes/role/"$ROOK_NAMESPACE" \
|
|
bound_service_account_names="$ROOK_SYSTEM_SA","$ROOK_OSD_SA" \
|
|
bound_service_account_namespaces="$ROOK_NAMESPACE" \
|
|
policies="$VAULT_POLICY_NAME" \
|
|
ttl=1440h
|
|
```
|
|
|
|
Once done, your `CephCluster` CR should look like:
|
|
|
|
```yaml
|
|
security:
|
|
kms:
|
|
connectionDetails:
|
|
KMS_PROVIDER: vault
|
|
VAULT_ADDR: https://vault.default.svc.cluster.local:8200
|
|
VAULT_BACKEND_PATH: rook/ver1
|
|
VAULT_SECRET_ENGINE: kv
|
|
VAULT_AUTH_METHOD: kubernetes
|
|
VAULT_AUTH_KUBERNETES_ROLE: rook-ceph
|
|
```
|
|
|
|
### General Vault configuration
|
|
|
|
As part of the token, here is an example of a policy that can be used:
|
|
|
|
```hcl
|
|
path "rook/*" {
|
|
capabilities = ["create", "read", "update", "delete", "list"]
|
|
}
|
|
path "sys/mounts" {
|
|
capabilities = ["read"]
|
|
}
|
|
```
|
|
|
|
You can write the policy like so and then create a token:
|
|
|
|
```console
|
|
vault policy write rook /tmp/rook.hcl
|
|
vault token create -policy=rook
|
|
```
|
|
>```
|
|
>Key Value
|
|
>--- -----
|
|
>token s.FYzlo02cgnTehF8dXnAoq2Z9
|
|
>token_accessor oMo7sAXQKbYtxU4HtO8k3pko
|
|
>token_duration 768h
|
|
>token_renewable true
|
|
>token_policies ["default" "rook"]
|
|
>identity_policies []
|
|
>policies ["default" "rook"]
|
|
>```
|
|
|
|
In the above example, Vault's secret backend path name is `rook`. It must be enabled with the following:
|
|
|
|
```console
|
|
vault secrets enable -path=rook kv
|
|
```
|
|
|
|
If a different path is used, the `VAULT_BACKEND_PATH` key in `connectionDetails` must be changed.
|
|
|
|
### TLS configuration
|
|
|
|
This is an advanced but recommended configuration for production deployments, in this case the `vault-connection-details` will look like:
|
|
|
|
```yaml
|
|
security:
|
|
kms:
|
|
# name of the k8s config map containing all the kms connection details
|
|
connectionDetails:
|
|
KMS_PROVIDER: vault
|
|
VAULT_ADDR: https://vault.default.svc.cluster.local:8200
|
|
VAULT_CACERT: <name of the k8s secret containing the PEM-encoded CA certificate>
|
|
VAULT_CLIENT_CERT: <name of the k8s secret containing the PEM-encoded client certificate>
|
|
VAULT_CLIENT_KEY: <name of the k8s secret containing the PEM-encoded private key>
|
|
# name of the k8s secret containing the kms authentication token
|
|
tokenSecretName: rook-vault-token
|
|
```
|
|
|
|
Each secret keys are expected to be:
|
|
|
|
* VAULT_CACERT: `cert`
|
|
* VAULT_CLIENT_CERT: `cert`
|
|
* VAULT_CLIENT_KEY: `key`
|
|
|
|
For instance `VAULT_CACERT` Secret named `vault-tls-ca-certificate` will look like:
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: vault-tls-ca-certificate
|
|
namespace: rook-ceph
|
|
data:
|
|
cert: <PEM base64 encoded CA certificate>
|
|
```
|
|
|
|
Note: if you are using self-signed certificates (not known/approved by a proper CA) you must pass `VAULT_SKIP_VERIFY: true`.
|
|
Communications will remain encrypted but the validity of the certificate will not be verified.
|