Commit Graph
105 Commits
Author SHA1 Message Date
parth-gr ec2681f82b external: fix rgw check if no multisite flag is passed
if there is no multisite config pass, it will still checks for
the zones and zones group and not able to query anything

Signed-off-by: parth-gr <paarora@redhat.com>
2023-05-16 20:10:29 +05:30
parth-grandSergio Pérez Fernández c50c308abe external: fix rgw multisite config check
the check validate_rgw_multisite was always checking for realm
updated it to check the specific config

fixed validate_rgw_endpoint pool validation

added missing realm zonegroup and zone while interacting with user resources

Co-authored-by: Sergio Pérez Fernández <sergioperez794@gmail.com>
Signed-off-by: parth-gr <paarora@redhat.com>
2023-05-05 00:56:18 +05:30
parth-gr e02942ab77 external: check for pool and cluster name is provided
While adding the support for dot and special rbd pools
the check for pool and cluster name check was removed,
Adding it back as it is needed
https://github.com/rook/rook/pull/12056

Signed-off-by: parth-gr <paarora@redhat.com>
2023-04-25 18:48:27 +05:30
Travis Nielsen eaf3e3662d Merge pull request #12037 from parth-gr/external-multisite-zone
external: add support for multisite in external cluster script
2023-04-13 07:55:44 -06:00
Travis Nielsen 6103cb93dd Merge pull request #12061 from neoaggelos/feat/external
external: do not enforce need for monitoring endpoint
2023-04-12 16:08:54 -06:00
Angelos Kolaitis 9e4a6ddee5 external: do not enforce need for monitoring endpoint
Add --skip-monitoring-endpoint to create-external-cluster-resources.py
to allow the script to work with Ceph clusters without an enabled
prometheus module.

Document the new flag in external-cluster.md and add relevant unit tests

Signed-off-by: Angelos Kolaitis <neoaggelos@gmail.com>
2023-04-12 21:04:48 +03:00
parth-gr 6c3dabcf5f external: add alias rbd pool name to support . pools name
if restricted permission was on the . was appended to k8s secret and
failed to create the secret so added a alias name is user wan't to suuport
pool with . in name

Signed-off-by: parth-gr <paarora@redhat.com>
2023-04-12 19:26:20 +05:30
parth-gr b60da0b8c1 external: add support for multisite in external cluster
Add support for realm, zone and zonegroup

Signed-off-by: parth-gr <paarora@redhat.com>
2023-04-10 20:03:16 +05:30
parth-gr 626d1c44ff external: name correction of validate_rgw_metadata_ec_pool_name function
The function is used for rbd so it should be name to validate_rbd_metadata_ec_pool_name

Signed-off-by: parth-gr <paarora@redhat.com>
2023-04-05 20:54:13 +05:30
parth-gr e44df84687 external: make rgw call separate from cephfs and rbd
if the rge endint is not reachable don't block the creation
of rbd and cephfs sc, just print the error instead of raising exception

Signed-off-by: parth-gr <paarora@redhat.com>
2023-03-30 19:54:33 +05:30
Sheetal Pamecha 6dbbc9d71c external: use f-strings for formatting
Replace the old formatting method with the f-string
and fix few pylint errors

Signed-off-by: Sheetal Pamecha <spamecha@redhat.com>
2023-03-17 11:20:26 +05:30
Matt 8bdabf85a8 external: pool and metadata ec pools were reversed in scripts
Pool and Metadata Pool Appear to be reversed.

dataPool needs to contain the EC Data pool
pool needs to contain the Metadata Pool
Closes: https://github.com/rook/rook/issues/11918

Signed-off-by: Matt P <dragon@fbdragon.co.uk>
2023-03-16 22:33:23 +00:00
parth-gr 96ea7817f8 external: add realm support for external cluster
if a cluster won't use the deafult realm and try to
execute the python script it will fail to vaildate rgw
as the rgw-admin-user would be created on default realm
So creating user in specific realm if the realm name is passed

Signed-off-by: parth-gr <paarora@redhat.com>
2023-03-01 20:55:48 +05:30
parth-gr def9fd9056 external: dont do rgw validation if python version < 3
Signed-off-by: parth-gr <paarora@redhat.com>
2023-01-23 20:38:49 +05:30
parth-gr fb7b67c3f4 external: fix endpoint_dial check for rgw endpoint
the validate rgw endpoint tls cert returns the certificate instead
of returning whether it exist or not,
So updatd the flow how to use it

Closes: https://github.com/rook/rook/issues/11060
Signed-off-by: parth-gr <paarora@redhat.com>
2022-10-14 19:16:30 +05:30
Travis Nielsen eaee34872b Merge pull request #10912 from parth-gr/restricted-upgrade
docs: add upgrade steps for multi-tenant custer
2022-09-14 07:44:07 -06:00
parth-gr 0ad1b52005 docs: add upgrade steps for multi-tenant custer
Currently for external clustr multitenancy is only applicable for new
deployments, but adding doc change so multi-tenancy can be achieved for
upgraded cluster also

Closes: https://github.com/rook/rook/issues/10678
Signed-off-by: parth-gr <paarora@redhat.com>
2022-09-14 12:48:18 +05:30
Blaine Gardner 2b0eaa974f Merge pull request #10909 from subhamkrai/remove-import-error
external: fix import errors for python2
2022-09-13 08:49:12 -06:00
subhamkrai 9a952df419 external: fix import errors for python2
remove duplicate import urllib.parse which is handled below
with try and except no need to import separately. Also, change
the way to get the python version.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-09-13 10:59:18 +05:30
parth-gr 54425803b6 external: do not add info=read cap if ceph version doesn't support it
The rgw ip validation feature was needing to update the rgw user
cap to add info=read permission, but this permission is only backported
till ceph pacific, so we added a check that only add this cap
when ceph version support it,
But this cap was also adding with the other caps, so removed it from there
Now it will only add this cap if ceph version support it

Signed-off-by: parth-gr <paarora@redhat.com>
2022-09-09 15:46:18 +05:30
Josh Soref 6e7b8767f3 core: fix spelling
* another
* are
* availability
* available
* bootstrap
* boundaries
* ceph
* certificate
* class
* codifies
* consuming
* corrupted
* createor
* csi
* deployments
* exceeded
* execute
* filesystem
* healthiness
* heuristics
* immediately
* insecure
* installed
* isolated
* maintained
* maximum
* minute
* monitor
* new
* nginx
* nonexistent
* not
* occurs
* omitempty
* operator
* orchestration
* persistentvolumes
* placement
* preexisting
* prometheus
* protecting
* provisioner
* purposes
* reconcile
* regex
* related
* requests
* returns
* rubbish
* running
* schedulable
* schedule
* serviceaccount
* simulating
* snapshots
* statement
* static
* tenants
* the
* unavailable
* volumeattachment
* waiting
* with
* wrapper
* zonegroup

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-07-07 18:10:47 -04:00
parth-gr c9a864ccdb external: add support for subvolumegroup and rados namespace
Using subvolume group and rados namespace for external deployment
was complicated and not documented,
Automating the process for the users, so they just need to pass
the respective names by cli flag for creation

Closes: rook#10427

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-28 21:57:54 +05:30
subhamkrai fa1eed1d4e core: fix lint issue in python files
Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-27 16:53:01 +05:30
parth-gr b7c165b411 csi: make storage class updated with csi-users in external cluster
For using restricted auth users we need to update the
restricted users in the Storage class, so updating
it in the json output for external cluster

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-08 14:49:46 +05:30
parth-gr dcba689129 csi: update restricted auth flag
remove the restriction of per rados namespace as
there can be multiple namespaces and the csi-users
can only have 1 type in a storage class

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-08 14:21:49 +05:30
subhamkrai 00f21c0866 external: format python files using tool black
`black` is python formatter tool which
automatically formats python files without
manual changing. just run `black <python-file>`
and done.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-06 20:01:27 +05:30
Sébastien Han eb08db40e9 Merge pull request #10350 from subhamkrai/support-py2
external-script: remove signature import
2022-06-03 08:49:58 +02:00
parth-gr fed91ddb81 rgw: make fqdn rgw converted to ip everywhere
During the recent addtions of rgw validation
changes there are too many places where we use
rgwendpoint, so making fqdn to ip conversion globally

Signed-off-by: parth-gr <paarora@redhat.com>
2022-05-31 15:11:36 +05:30
subhamkrai c8f11f3e62 core: clean the import section
import all the libraries at top.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-05-30 16:04:30 +05:30
subhamkrai 60bf1f6e1c core: remove signature import
remove signature import as it is not required.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-05-30 16:04:12 +05:30
parth-gr cbda9f2f7d csi: update cephfs osd caps
Currently cephfs caps are data=filesystemname,
which only have write access, but not the read access
changing it to *=filesystemname so we can read data also

Signed-off-by: parth-gr <paarora@redhat.com>
2022-05-25 18:56:54 +05:30
Sébastien Han 9b6143baa7 Merge pull request #10006 from parth-gr/rgw-external
rgw: validate rgw endpoint ip
2022-05-13 14:28:10 +02:00
parth-gr 624400ff93 rgw: support FQDN for RGW endpoint
Now rgw can be pass by FQDN too,
to avoid problems where it is required
The format should be similar to
Ip(<IP>:<PORT> format), <FQDN>:<PORT> format

Signed-off-by: parth-gr <paarora@redhat.com>
2022-05-12 18:31:35 +05:30
parth-gr b97bc74f45 csi: update the upgrade flag description
Currently it was mention we are updating the csi-user
But more precisely we go and update the keyrings
which auto update users so replacing it

Signed-off-by: parth-gr <paarora@redhat.com>
2022-05-11 19:52:38 +05:30
parth-gr cf29043f4d rgw: donot validate prefix rgw pools when prefix is default
Signed-off-by: parth-gr <paarora@redhat.com>
2022-05-11 19:52:38 +05:30
parth-gr 09d52511ad rgw: validate rgw endpoint ip
The ceph-external-cluster-details-exporter.py
generates the json output even with
rgw-endpoint IP from different cluster
Added a check validate_rgw_endpoint which will validate
the above rgw_endpoint Ip by matching the cluster fsid

Signed-off-by: parth-gr <paarora@redhat.com>
2022-05-11 19:52:30 +05:30
parth-gr 63945df72d rgw: change name of rgw admin keys
Corrected RGW_ADMIN_OPS_USER_ACCESS_KEY  and
RGW_ADMIN_OPS_USER_SECRET_KEY so it should be
output correctely when called script with
bash format output

Signed-off-by: parth-gr <paarora@redhat.com>
2022-04-05 19:33:36 +05:30
parth-gr 79db113aab csi: rbd node secret was misspelled
Corrected CSI_RBD_NODE_SECRET name so it should be
output correctely when called script with
bash format output

Signed-off-by: parth-gr <paarora@redhat.com>
2022-03-29 16:56:42 +05:30
parth-gr 339855f172 ci: fixed pylint failure
pylint was failing with used-before-assignment error
fixed it by making the variables assignmet before it was used
Also backward compatibility is removed from the ModuleNotFoundError import,
as we auto installed python3 version on ceph standalone

Signed-off-by: parth-gr <paarora@redhat.com>
2022-03-29 12:09:32 +05:30
vavuthu b9d8df9ec9 core: avoid creating json_out incase of dry-run
When using dry-run option with external script, out_map dictionary
doesn't contain keys ( ACCESS_KEY and SECRET_KEY ) which leads to KeyError.
This fix to avoid creating json_out incase of dry-run since we will not use
json_out for dry-run.

Signed-off-by: vavuthu <vavuthu@redhat.com>
2022-03-10 14:25:52 +05:30
vavuthu 149beedf20 core: assign ACCESS_KEY and SECRET_KEY values only in case of actual run
When using dry-run option with external script, function create_rgw_admin_ops_user
is not returning expected values which leads to TypeError. This fix is to append
values to out_map dictionary only in case of actual run.

Signed-off-by: vavuthu <vavuthu@redhat.com>
2022-03-10 14:16:13 +05:30
vavuthu ce98b17bf7 core: fix incorrect join command
Fixing typo in join command
joing -> join

Signed-off-by: vavuthu <vavuthu@redhat.com>
2022-03-10 13:56:37 +05:30
subhamkrai 3e6aa9e556 core: split external script unit test and main code in diff file
split the external script main code and unit test in different file.

Closes: https://github.com/rook/rook/issues/9588
Signed-off-by: subhamkrai <srai@redhat.com>
2022-02-24 18:23:21 +05:30
parth-gr 3e64525b62 csi: make use of common user creation method
refactor the code so we can use common csi user
creation method

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-16 10:29:34 +05:30
parth-gr 9b20b475cf csi: modify upgrade flag in external cluster
The upgrade function doesn't seem to be that smart
for now so it can update the new auth caps listed
with the existing one, it only compares the value of
the current cap with the MIN_USER_CAP_PERMISSIONS,

Modified the upgrade flag functionality so we can use
this while upgrading the cluster clients

Closes: https://github.com/rook/rook/issues/9589

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-16 10:29:34 +05:30
parth-gr 4f43f2d93e csi: make create and get process separate for csi users
Currently if we run external-script and user already exit,
it fails to create/run same user again if the caps are upgraded,
so we need to check first if user is already present.
And for upgrading user we can do upgrade after this
PR: https://github.com/rook/rook/pull/9609 will get merged.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2048458

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-04 20:08:31 +05:30
parth-gr 7e851f15c6 csi: add osd blocklist capabilities to the external cephcluster
The CSI users created by the external script(both for rbd and cephfs)
need the blocklist capability so the CSI pods can execute the
`osd blocklist` commands for metro DR
update the mon caps of csi users with `profile simple-rados-client-with-blocklist`
so they have privileges to run osd blocklist cmds

Closes: https://issues.redhat.com/browse/RHSTOR-2451

Signed-off-by: parth-gr <paarora@redhat.com>
2022-01-18 18:01:01 +05:30
parth-gr 4a3088c580 security: make cephfs_filesystem_name flag optional
cephfs_filesystem_name flag isn't mandatory if restricted_auth_permission
is on.

Signed-off-by: parth-gr <paarora@redhat.com>
2022-01-07 20:27:05 +05:30
parth-gr 6ed7423bac security: update auth permissions for external cluster
Added a clusterName, poolName , filesystemName, radosNamespace
as a suffix on ceph auth principals for external cluster so
every csi authentication would be unique if
restricted_auth_permission flag is specified.

Closes: https://github.com/rook/rook/issues/9192
Signed-off-by: parth-gr <paarora@redhat.com>
2022-01-05 13:51:41 +05:30
subhamkrai c891ddb57d security: add dry run mode for external cluster script
Adding dry run mode for external cluster script.
This will add cli argument `--dry-run`. By default
`dry-run` option will be `False`
which means it will only print something like below.

```
Execute: 'ceph fs ls'
Execute: 'ceph fsid'
Execute: 'ceph quorum_status'
Execute: 'ceph auth get-or-create client.healthchecker mon allow r, allow command quorum_status,
	allow command version mgr allow command config osd allow rwx pool=default.rgw.meta, allow r pool=
	.rgw.root, allow rw pool=default.rgw.control, allow rx pool=default.rgw.log, allow x
	pool=default.rgw.buckets.index'
Execute: 'ceph mgr services'
Execute: 'ceph auth get-or-create client.csi-rbd-node mon profile rbd osd profile rbd'
Execute: 'ceph auth get-or-create client.csi-rbd-provisioner mon profile rbd mgr allow rw osd
	profile rbd'
Execute: 'ceph status'

```

Signed-off-by: subhamkrai <srai@redhat.com>
2021-12-16 13:05:39 +05:30