Add a design for CephObjectStore to allow users to pass through Ceph
configs (e.g., `rgw_enable_apis`, `debug_rgw`) to the object store. This
design will help users set configs that they want for their environments
that aren't covered under Rook's umbrella of managed configs. This will
benefit users by allowing them to make use of rare RGW features that
Rook devs don't have ability to prioritize, as well as newly-released
RGW features that Rook devs haven't had time to plan API interfaces for.
This frees up Rook developers to focus effort on implementing APIs that
have integration benefits with Kubernetes or other resources and less
effort on simple configs.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For the specification see:
<https://github.com/rook/rook/blob/master/design/ceph/object/swift-and-keystone-integration.md>
* extend the API object specs for swift and keystone integration
* adapt rgw to the new go-ceph version
- The parameter lists of the API call have changes, as parameters
ignored by the RGW Admin Ops API are no longer serialized, therefore
the mock has to be adapted.
- There is now validation for the user keys that are passed to the
User get API, therefore things failed when we had empty keys in our
User proxy object.
* expand the reconcile loop for the swift and keystone integration
* fix minor mistakes in design document
* add env var to pass extra args to minikube
Minikube decides CPU cores and memory automatically based on the
available resources on the machine which may be insufficient to
run rook. This commit adds an environment variable to add arbitrary
arguments to the minikube command, so both can be specified if
desired.
* integration tests for swift and keystone
The new integration of swift or s3 and keystone support by rook
does not have any integration tests yet.
This commit introduces integration tests for swift and keystone. The
tests are done against a minimal keystone setup (keystone container
image from Yaook-project (https://yaook.cloud), sqlite as database
backend, cert-manager and trust-manager for test certificate setup).
To prevent hardcoded credentials, passwords are generated
by the tests. The integration tests use the openstack client
(keystone- and swift-functionality) (https://docs.openstack.org/
python-openstackclient/ latest/). This was a concious design decision
to use client tooling as close as possible to the end user instead of
using other go-libraries (such as gophercloud).
* add documentation on swift and keystone
Currently there is no documentation on the use of Swift to access
an object store as well as the use of OpenStack keystone for
authentication.
This commit adds documentation on the use of Swift and OpenStack
keystone, as well as CRD-related documentation and an example setup.
* add integration tests for S3 via keystone
This commit introduces integration tests for s3 and keystone. The
tests are run against the same minimal keystone setup that the tests
for swift and keystone use.
The integration tests use the aws s3 client to use client tooling as
close as possible to the end user instead of using other go-libraries.
Co-authored-by: Jan Klippel <jan.klippel@uhurutec.com>
Co-authored-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Signed-off-by: Jan Klippel <jan.klippel@uhurutec.com>
Signed-off-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
RGW can only serve a single certificate. This limitation means that the
prior behavior of using the default service for admin ops when TLS is
enabled may mean it requires additional complex certificate management
to make sure the object store uses a certificate valid for Rook internal
admin ops and user connections.
This is needlessly complex for users. Instead, change Rook's behavior
and documentation to clarify that it will use the same endpoint intended
for S3 client applications. This means that users have a more
straightforward path to enabling both Rook and consuming applications.
More info: https://github.com/rook/rook/issues/14530
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Update the vhost-style S3 bucket access design doc to reflect recent
user feedback and configuration issues. The primary goal is to add a
field to disambiguate for users and Rook which endpoint should be used
for CephObjectStoreUsers, OBCs, and COSI buckets.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Until now, an object store would create all the necessary
metadata pools and the data pool that were exclusively
for its own object store. When isolation between object
stores is necessary, this would cause many pools and
PGs to be created in the cluster, which was not
manageable.
Now one set of pools can be created to be shared
by any number of object stores. The metadata and data
between each object store is isolated by
RADOS namespaces, which by design will keep the
data safe for multi-tenancy.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When CPU requests and limits are assigned to a pod,
the pod will be guaranteed the requests, up to the limits.
Even if there are spare CPU cycles, the pod cannot use
them. Thus, pods can be unnecessarily denied compute
when they need to burst if the limits are set.
Therefore, it is not recommended to set CPU limits
since the CPU requests are already guaranteeing that
no pod will be starved at least for its requests.
Signed-off-by: travisn <tnielsen@redhat.com>
The PR #11091 add support for LDAP integration with RGW, this contains
breif overview of design specific details.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
User can define his desired endpoint list in Zone CR so that it will
overwrite the default service name for rgw.
Resolves#6432
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
The colon character is not allowed in secret names, therfore the secret
with the subuser credentials must use another character to join the
user and the subuser name.
This in turn makes the mapping non-injective (which has security
implications).
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Make explicity how the operator acts when the subuser list in the
CephObjectStoreUser CR is changed.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Since only OpenStack Identity API version 3 is supported the
API version need not be configurable in the resource.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
As proposed in the review the natural target version for this feature is
1.9 which is to be released around March 2022.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
The open questions have been resolved in the discussion and the chosen
solution has been added to the main text. So the Open Questions section
is obsolete now.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
The goals/non-goals were mixed up to a degree. This
cleanly separates them to their respective sections.
In the course of that the term "Keystone API" was replaced by the more
correct "OpenStack Identity API" and the documentation of the API was
linked.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Instead of mixing configuration for different protocols in the gateway
section, depending on the type option, introduce a new section protocols for
API specific configuration.
This allows configuring s3 and swift in one RGW and at the same time
simplifies validation of the CRD and allows for a clean separation of
concerns.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
* add annotations to the examples of new CR fields
* add links to external documentation
* clarify language issues raised in review
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
The prior formulation which openrc variables take effect was
ambiguous/contradictory.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Access to the Swift API is granted via subusers. The configuration
of the subusers is added to the cephobjectstoreuser CR.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
The integration of Keystone includes HTTPS API calls to Keystone. We
document why the CA bundle used to verify those connections,
can't be explicitly configured and how it can be indirectly configured.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
It is not clear at this moment whether anyone has a use case for this.
As a workaround you can explicitly generate a user and subuser that
can access the bucket created via OBC.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Ceph RGW supports integrating with OpenStack clouds by providing
the Swift object store API and using Keystone for authentication.
The design document describes how the required configuration shall
be mapped to the Object Store CRD.
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
If the admin wants to use insecure TLS to validate connections to rgw
internally, the TLS secret can have another entry "insecureSkipVerify"
and set it to "true".
Signed-off-by: Sébastien Han <seb@redhat.com>
`CephObjectStore->gateway->type` is not used. Rook has only supported s3-like
interface and hasn't had no code which handles `type` field.
In addition, this field was removed from CRD in the following commit.
ceph: auto-gen crds
31db03fece
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This documents the Ceph topics and buckets notification management by rook through CRs.
Signed-off-by: Guillaume Moutier <guillaume.moutier@gmail.com>
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
Santosh
Blaine Gardner
Travis Nielsen
Artem Torubarov
Jiffin Tony Thottan
Josh Soref
Jiffin Tony Thottan
subhamkrai
Sébastien Han
Yuval Lifshitz
Denis Egorenko
Satoru Takeuchi
Guillaume Moutier
Mateusz Gozdek