From ceph v16.2.6 onwards the vault TLS suppport in RGW was added,
include similar changes for RGW.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
The rook operator as well as the toolbox pod run with the "rook" user
with UID 2016. The UID was chosen based on the year of the initial
commit in the rook/rook repository.
No more root user running.
Closes: https://github.com/rook/rook/issues/8734
Signed-off-by: Sébastien Han <seb@redhat.com>
We can pass bound_service_account_names with a comma separated list of
service accounts. Let's do this instead of remapping new values.
Earlier, we thought a single service account could be added per Vault
role and we were using other variables like
`VAULT_AUTH_KUBERNETES_ROOK_OPERATOR_ROLE` that we were remapping to
`VAULT_AUTH_KUBERNETES_ROLE` internal for the API calls to Vault.
Signed-off-by: Sébastien Han <seb@redhat.com>
Rook cluster-wide encryption can now use the native Kubernetes
authentication to interact with vault KMS instead of using the token
method.
Signed-off-by: Sébastien Han <seb@redhat.com>
When deploying a cluster-wde encrypted cluster we now use the rook
binary to execute some code to fetch the key encryption key.
Using cURL all the time to fetch the key has its limitations. The
incoming integration with Kubernetes Authentication through service
accounts is leading the usage of cURL to its end.
The logic is really complex and prone to errors. Re-implementing the
Go logic into Bash is not a viable option.
Also, using the lib in a binary allows us to keep a consistent behavior
throughout the life cycle of our code.
Signed-off-by: Sébastien Han <seb@redhat.com>
When TLS is used and includes a caert, client key/cert, we need to copy
the content of the secret to a file in the operator's container
filesystem so that we can build the TLS config and thus the HTTP Client,
which reads those files.
Also, removing the files after each API call so they don't persist on
the filesystem forever.
Signed-off-by: Sébastien Han <seb@redhat.com>
Prior, we were returning a nil map and thus the assignment for forced
deletion was not working since we were trying to assign on a nil map.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:
* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited
As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.
A second new controller for the operator's general config has been
created, it manages:
* the logging level
* the ceph CLI command timeout
* the discovery daemon
The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.
Signed-off-by: Sébastien Han <seb@redhat.com>
Passing struct by value essentially gives you a copy, so when modified
within a function, the scope is then reduced to that function. Using
pointers solves that you mutate the struct as many times as you want from
anywhere.
As a result, the auto-detection of the Vault KV backend was not working
correctly.
Also, added a ton of unit tests for Vault.
Signed-off-by: Sébastien Han <seb@redhat.com>
Rook will now auto detect the kv version of the vault server. This
allows users not having to pass the VAULT_BACKEND configuration in the
CephCluster CR.
Signed-off-by: Sébastien Han <seb@redhat.com>
By default, keys stored in Vault KV Secret Engine are versioned and keys
are soft-deleted, see Vault documentation:
When deleting data the standard vault kv delete command will perform a soft
delete.
It will mark the version as deleted and populate a deletion_time timestamp.
Soft deletes do not remove the underlying version data from storage,
which allows the version to be undeleted.
The vault kv undelete command handles undeleting versions. A version's
data is permanently deleted only when the key has more versions than are
allowed by the max-versions setting, or when using vault kv destroy.
When the destroy command is used the underlying version data will be
removed and the key metadata will be marked as destroyed.
If a version is cleaned up by going over max-versions the version
metadata will also be removed from the key.
Signed-off-by: Sébastien Han <seb@redhat.com>
We don't need to pass the Volume with projection for TLS when TLS is not enabled
Somehow when this happens and we try to update a deployment spec it fails with:
```
ValidationError(Pod.spec.volumes[7].projected): missing required field "sources"
```
Signed-off-by: Sébastien Han <seb@redhat.com>
The ValidateConnectionDetails() contains `clusterSpec` assuming `SecuritySpec` only
part to `cephCluster` CRD. But we may need to validate same for `SecuritySpec` in
`cephObjectStore` CRD. Hence modifying the signature of the api.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
When the cluster is growing, the operator always processes all OSDs and
re-generate their Deployment spec. The generated environment variables
for encryption are coming from a map, which by design is unordered. So
once generated the deployment will see their spec change since the
environment variable position will change.
See from the following diff:
```
env: env:
- name: KMS_SERVICE_NAME <
value: vault <
- name: VAULT_NAMESPACE <
value: ocsns <
- name: VAULT_TLS_SERVER_NAME <
- name: VAULT_BACKEND_PATH <
value: n_ocs <
- name: VAULT_ADDR - name: VAULT_ADDR
value: https://vault.qe.rh-ocs.com:8200 value: https://vault.qe.rh-ocs.com:8200
> - name: VAULT_BACKEND_PATH
> value: n_ocs
> - name: VAULT_TLS_SERVER_NAME
- name: KMS_PROVIDER - name: KMS_PROVIDER
value: vault value: vault
> - name: KMS_SERVICE_NAME
> value: vault
> - name: VAULT_NAMESPACE
> value: ocsns
- name: VAULT_TOKEN - name: VAULT_TOKEN
valueFrom: valueFrom:
secretKeyRef: secretKeyRef:
key: token key: token
name: ocs-kms-token name: ocs-kms-token
```
Signed-off-by: Sébastien Han <seb@redhat.com>
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The vault env variable VAULT_BACKEND is read by the secret library so we
must not change it. This is reverting
ee6a03dfb8 with a few extra fixes for the
rgw code.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the cluster was expanded we would iterate over the entire list of
OSDs again, create a new key and override in Vault... The Secret store
is not impacted since we have a handler for "already exist".
Signed-off-by: Sébastien Han <seb@redhat.com>
The first patch to configure vault for ceph object store. If the `security.kms` configured in
`clusterSpec` CRD, RGW will be configured with vault kms settings to handle SSE request from s3 clients.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Golang maps order is not always guaranteed to be identical, which makes
the table driven test to fail. Because of the nature of the maps, it is
hard to predict the exact position of an element in the map. This
makes the table driven test flappy.
Reworked the unit test using a contains statement which achieves the
same validation.
Closes: https://github.com/rook/rook/issues/6522
Signed-off-by: Sébastien Han <seb@redhat.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
Jiffin Tony Thottan
Sébastien Han
Satoru Takeuchi
Arun Kumar Mohan