Merge pull request #10912 from parth-gr/restricted-upgrade
docs: add upgrade steps for multi-tenant custer
This commit is contained in:
2 files changed
+58
-46
No files matched your search
@@ -28,11 +28,13 @@ The features available from the external cluster will vary depending on the vers
|
||||
|
||||
In order to configure an external Ceph cluster with Rook, we need to extract some information in order to connect to that cluster.
|
||||
|
||||
1. Run the python script [create-external-cluster-resources.py](https://github.com/rook/rook/blob/master/deploy/examples/create-external-cluster-resources.py) for creating all users and keys.
|
||||
### 1. Create all users and keys
|
||||
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --rbd-data-pool-name <pool_name> --cephfs-filesystem-name <filesystem-name> --rgw-endpoint <rgw-endpoint> --namespace <namespace> --format bash
|
||||
```
|
||||
Run the python script [create-external-cluster-resources.py](https://github.com/rook/rook/blob/master/deploy/examples/create-external-cluster-resources.py) for creating all users and keys.
|
||||
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --rbd-data-pool-name <pool_name> --cephfs-filesystem-name <filesystem-name> --rgw-endpoint <rgw-endpoint> --namespace <namespace> --format bash
|
||||
```
|
||||
|
||||
- `--namespace`: Namespace where CephCluster will run, for example `rook-ceph-external`
|
||||
- `--format bash`: The format of the output
|
||||
@@ -54,51 +56,60 @@ In order to configure an external Ceph cluster with Rook, we need to extract som
|
||||
- `--cephfs-data-pool-name`: (optional) Provides the name of the CephFS data pool, used for creating CephFS StorageClass
|
||||
- `--rados-namespace`: (optional) Divides a pool into separate logical namespaces, used for creating RBD PVC in a RadosNamespaces
|
||||
- `--subvolume-group`: (optional) Provides the name of the subvolume group, used for creating CephFS PVC in a subvolumeGroup
|
||||
- `--upgrade`: (optional) Upgrades the 'Ceph CSI keyrings (For example: client.csi-cephfs-provisioner) with new permissions needed for the new cluster version and older permission will still be applied.
|
||||
- `--restricted-auth-permission`: (optional) Restrict cephCSIKeyrings auth permissions to specific pools, and cluster. Mandatory flags that need to be set are `--rbd-data-pool-name`, and `--cluster-name`. `--cephfs-filesystem-name` flag can also be passed in case of CephFS user restriction, so it can restrict users to particular CephFS filesystem.
|
||||
|
||||
!!! note
|
||||
Restricting the csi-users per pool, and per cluster will require creating new csi-users and new secrets for that csi-users, so it can only be used for new deployments.
|
||||
#### Multi-tenancy
|
||||
|
||||
Sample run:
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --cephfs-filesystem-name <filesystem-name> --rbd-data-pool-name <pool_name> --cluster-name <cluster-name> --restricted-auth-permission true --format <bash> --rgw-endpoint <rgw_endpoin> --namespace <rook-ceph-external>
|
||||
```
|
||||
|
||||
- `--upgrade`: (optional) Upgrades the 'Ceph CSI keyrings (For example: client.csi-cephfs-provisioner) with new permissions needed for the new cluster version and older permission will still be applied.
|
||||
|
||||
Sample run:
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --upgrade
|
||||
```
|
||||
This will upgrade all the default csi-users(non-restricted)
|
||||
|
||||
Restricted users created using --restricted-auth-permission flag (For example: client.csi-rbd-node-rookStorage-replicapool) need to pass mandatory flags: '--rbd-data-pool-name(if it is a rbd user), --cluster-name and --run-as-user' flags while upgrading, in case of cephfs users if you have passed --cephfs-filesystem-name flag while creating csi-users then while upgrading it will be mandatory too
|
||||
|
||||
Sample run:
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --upgrade --rbd-data-pool-name replicapool --cluster-name rookstorage --run-as-user client.csi-rbd-node-rookStorage-replicapool
|
||||
```
|
||||
To enable multi-tenancy, run the script with the `--restricted-auth-permission` flag and pass the mandatory flags with it,
|
||||
It will generate the secrets which you can use for creating new `Consumer cluster` deployment using the same `Source cluster`(ceph cluster).
|
||||
So you would be running different isolated consumer clusters on top of single `Source cluster`.
|
||||
|
||||
!!! note
|
||||
An existing non-restricted user cannot be downgraded to a restricted user by upgrading. Admin needs to create a new restricted user for this by re-running the script.The upgrade flag should only be used to append new permissions to users. It shouldn't be used for changing a csi user already applied permissions. For example, you shouldn't change the pool(s) a user has access to.
|
||||
Restricting the csi-users per pool, and per cluster will require creating new csi-users and new secrets for that csi-users.
|
||||
So apply these secrets only to new `Consumer cluster` deployment while using the same `Source cluster`.
|
||||
|
||||
2. Copy the bash output.
|
||||
#### Sample run
|
||||
|
||||
Example Output:
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --cephfs-filesystem-name <filesystem-name> --rbd-data-pool-name <pool_name> --cluster-name <cluster-name> --restricted-auth-permission true --format <bash> --rgw-endpoint <rgw_endpoin> --namespace <rook-ceph-external>
|
||||
```
|
||||
|
||||
```console
|
||||
export ROOK_EXTERNAL_FSID=797f411a-aafe-11ec-a254-fa163e1539f5
|
||||
export ROOK_EXTERNAL_USERNAME=client.healthchecker
|
||||
export ROOK_EXTERNAL_CEPH_MON_DATA=ceph-rados-upstream-w4pdvq-node1-installer=10.0.210.83:6789
|
||||
export ROOK_EXTERNAL_USER_SECRET=AQAdm0FilZDSJxAAMucfuu/j0ZYYP4Bia8Us+w==
|
||||
export ROOK_EXTERNAL_DASHBOARD_LINK=https://10.0.210.83:8443/
|
||||
export CSI_RBD_NODE_SECRET=AQC1iDxip45JDRAAVahaBhKz1z0WW98+ACLqMQ==
|
||||
export CSI_RBD_PROVISIONER_SECRET=AQC1iDxiMM+LLhAA0PucjNZI8sG9Eh+pcvnWhQ==
|
||||
export MONITORING_ENDPOINT=10.0.210.83
|
||||
export MONITORING_ENDPOINT_PORT=9283
|
||||
export RBD_POOL_NAME=replicated_2g
|
||||
export RGW_POOL_PREFIX=default
|
||||
```
|
||||
#### Upgrade Sample Run
|
||||
|
||||
1) If consumer cluster doesn't have restricted caps, this will upgrade all the default csi-users (non-restricted):
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --upgrade
|
||||
```
|
||||
|
||||
2) If the consumer cluster has restricted caps:
|
||||
Restricted users created using `--restricted-auth-permission` flag need to pass mandatory flags: '`--rbd-data-pool-name`(if it is a rbd user), `--cluster-name` and `--run-as-user`' flags while upgrading, in case of cephfs users if you have passed `--cephfs-filesystem-name` flag while creating csi-users then while upgrading it will be mandatory too. In this example the user would be `client.csi-rbd-node-rookstorage-replicapool` (following the pattern `csi-user-clusterName-poolName`)
|
||||
|
||||
```console
|
||||
python3 create-external-cluster-resources.py --upgrade --rbd-data-pool-name replicapool --cluster-name rookstorage --run-as-user client.csi-rbd-node-rookstorage-replicapool
|
||||
```
|
||||
|
||||
!!! note
|
||||
An existing non-restricted user cannot be converted to a restricted user by upgrading.
|
||||
The upgrade flag should only be used to append new permissions to users. It shouldn't be used for changing a csi user already applied permissions. For example, you shouldn't change the pool(s) a user has access to.
|
||||
|
||||
### 2. Copy the bash output
|
||||
|
||||
Example Output:
|
||||
|
||||
```console
|
||||
export ROOK_EXTERNAL_FSID=797f411a-aafe-11ec-a254-fa163e1539f5
|
||||
export ROOK_EXTERNAL_USERNAME=client.healthchecker
|
||||
export ROOK_EXTERNAL_CEPH_MON_DATA=ceph-rados-upstream-w4pdvq-node1-installer=10.0.210.83:6789
|
||||
export ROOK_EXTERNAL_USER_SECRET=AQAdm0FilZDSJxAAMucfuu/j0ZYYP4Bia8Us+w==
|
||||
export ROOK_EXTERNAL_DASHBOARD_LINK=https://10.0.210.83:8443/
|
||||
export CSI_RBD_NODE_SECRET=AQC1iDxip45JDRAAVahaBhKz1z0WW98+ACLqMQ==
|
||||
export CSI_RBD_PROVISIONER_SECRET=AQC1iDxiMM+LLhAA0PucjNZI8sG9Eh+pcvnWhQ==
|
||||
export MONITORING_ENDPOINT=10.0.210.83
|
||||
export MONITORING_ENDPOINT_PORT=9283
|
||||
export RBD_POOL_NAME=replicated_2g
|
||||
export RGW_POOL_PREFIX=default
|
||||
```
|
||||
|
||||
## Commands on the K8s consumer cluster
|
||||
|
||||
|
||||
@@ -332,11 +332,12 @@ class RadosJSON:
|
||||
common_group.add_argument(
|
||||
"--restricted-auth-permission",
|
||||
default=False,
|
||||
help="Restricted cephCSIKeyrings auth permissions to specific pools, cluster."
|
||||
help="Restrict cephCSIKeyrings auth permissions to specific pools, cluster."
|
||||
+ "Mandatory flags that need to be set are --rbd-data-pool-name, and --cluster-name."
|
||||
+ "--cephfs-filesystem-name flag can also be passed in case of cephfs user restriction, so it can restrict user to particular cephfs filesystem"
|
||||
+ "sample run: `python3 /etc/ceph/create-external-cluster-resources.py --cephfs-filesystem-name myfs --rbd-data-pool-name replicapool --cluster-name rookStorage --restricted-auth-permission true`"
|
||||
+ "Note: Restricting the users per pool, and per cluster will require to create new users and new secrets for that users.",
|
||||
+ "sample run: `python3 /etc/ceph/create-external-cluster-resources.py --cephfs-filesystem-name myfs --rbd-data-pool-name replicapool --cluster-name rookstorage --restricted-auth-permission true`"
|
||||
+ "Note: Restricting the csi-users per pool, and per cluster will require creating new csi-users and new secrets for that csi-users."
|
||||
+ "So apply these secrets only to new `Consumer cluster` deployment while using the same `Source cluster`.",
|
||||
)
|
||||
|
||||
output_group = argP.add_argument_group("output")
|
||||
@@ -439,8 +440,8 @@ class RadosJSON:
|
||||
+ "For restricted users(For example: client.csi-cephfs-provisioner-openshift-storage-myfs), users created using --restricted-auth-permission flag need to pass mandatory flags"
|
||||
+ "mandatory flags: '--rbd-data-pool-name, --cluster-name and --run-as-user' flags while upgrading"
|
||||
+ "in case of cephfs users if you have passed --cephfs-filesystem-name flag while creating user then while upgrading it will be mandatory too"
|
||||
+ "Sample run: `python3 /etc/ceph/create-external-cluster-resources.py --upgrade --rbd-data-pool-name replicapool --cluster-name rookStorage --run-as-user client.csi-rbd-node-rookStorage-replicapool`"
|
||||
+ "PS: An existing non-restricted user cannot be downgraded to a restricted user by upgrading. Admin need to create a new restricted user for this by re-running the script."
|
||||
+ "Sample run: `python3 /etc/ceph/create-external-cluster-resources.py --upgrade --rbd-data-pool-name replicapool --cluster-name rookstorage --run-as-user client.csi-rbd-node-rookstorage-replicapool`"
|
||||
+ "PS: An existing non-restricted user cannot be converted to a restricted user by upgrading."
|
||||
+ "Upgrade flag should only be used to append new permissions to users, it shouldn't be used for changing user already applied permission, for example you shouldn't change in which pool user has access",
|
||||
)
|
||||
|
||||
|
||||
Reference in new issue
Block a user