the rgw operations for s3 can now be accessible using sidecar
rgw-ops-log availabe in json form that can be further filtered logging
for observability, this will set the rgw_enable_ops_log setting
Signed-off-by: Deepika Upadhyay <deepika.upadhyay@clyso.com>
Implement #15119
Allow users to override RGW configurations by specifying Ceph config
options in the CephObjectStore. For configurations that require RGW to
be restarted when the config is applied, allow configs to be specified
as CLI arguments to the RGW as well.
This is an advanced option and is documented as such. Users should be
careful to understand the values they are setting, as there is no
validation to prevent the object store from breaking when these configs
are used.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For the specification see:
<https://github.com/rook/rook/blob/master/design/ceph/object/swift-and-keystone-integration.md>
* extend the API object specs for swift and keystone integration
* adapt rgw to the new go-ceph version
- The parameter lists of the API call have changes, as parameters
ignored by the RGW Admin Ops API are no longer serialized, therefore
the mock has to be adapted.
- There is now validation for the user keys that are passed to the
User get API, therefore things failed when we had empty keys in our
User proxy object.
* expand the reconcile loop for the swift and keystone integration
* fix minor mistakes in design document
* add env var to pass extra args to minikube
Minikube decides CPU cores and memory automatically based on the
available resources on the machine which may be insufficient to
run rook. This commit adds an environment variable to add arbitrary
arguments to the minikube command, so both can be specified if
desired.
* integration tests for swift and keystone
The new integration of swift or s3 and keystone support by rook
does not have any integration tests yet.
This commit introduces integration tests for swift and keystone. The
tests are done against a minimal keystone setup (keystone container
image from Yaook-project (https://yaook.cloud), sqlite as database
backend, cert-manager and trust-manager for test certificate setup).
To prevent hardcoded credentials, passwords are generated
by the tests. The integration tests use the openstack client
(keystone- and swift-functionality) (https://docs.openstack.org/
python-openstackclient/ latest/). This was a concious design decision
to use client tooling as close as possible to the end user instead of
using other go-libraries (such as gophercloud).
* add documentation on swift and keystone
Currently there is no documentation on the use of Swift to access
an object store as well as the use of OpenStack keystone for
authentication.
This commit adds documentation on the use of Swift and OpenStack
keystone, as well as CRD-related documentation and an example setup.
* add integration tests for S3 via keystone
This commit introduces integration tests for s3 and keystone. The
tests are run against the same minimal keystone setup that the tests
for swift and keystone use.
The integration tests use the aws s3 client to use client tooling as
close as possible to the end user instead of using other go-libraries.
Co-authored-by: Jan Klippel <jan.klippel@uhurutec.com>
Co-authored-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Signed-off-by: Jan Klippel <jan.klippel@uhurutec.com>
Signed-off-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
Some users want to deploy two CephObjectStores for a single Zone. The first
configures RGWs to process the synchronization of the data, while the second
CephObjectStore configures the client RGWs.
Currently, this can be implemented by setting the RGW option
'rgw_run_sync_thread' in the 'rook-config-override' ConfigMap, though it is not
really user friendly.
Ref: https://docs.ceph.com/en/latest/radosgw/config-ref/#confval-rgw_run_sync_thread
This commit adds a new option in the CephObjectStore CRD as defined in issue
https://github.com/rook/rook/issues/12272. The new option
'disableMultisiteSyncTraffic' determine whether the operator should disable the
multisite sync threads for the RGWs.
If the option is set to 'false', or if the option is not specified, the operator
does nothing. This ensures that the multisite sync threads will not be enabled
for users that disabled explicitely the multisite sync threads either manually
or with the 'rook-config-override' ConfigMap.
This commit also recommends to use two objectstore when scaling Ceph Objectstore
Multisite replication, with one objectstore configured with disabled replication
traffic.
Signed-off-by: Lucas Henry <polyedre@disroot.org>
use Zone and ZoneGroup instead of storename for rgw_zone and rgw_zonegroup
Signed-off-by: Olivier Bouffet <olivier.bouffet@infomaniak.com>
(cherry picked from commit c92270cd66)
From ceph v16.2.6 onwards the vault TLS suppport in RGW was added,
include similar changes for RGW.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
For the TLS communication for AdminOps Api, httpclient is required and
filled with TLS certs, currently it is set to nil pointer in
`buckethealthchecker` and `cephobjectstoreuser`.
Thanks @Krast76 finding the issue even proposing the fix.
Fixes: #8132
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Service serving certificates are intended to applications that require
encryption in openshift. These certificates are issued as TLS web server
certificates. Currently RGW supports TLS authentication with help of
certs passed as secrets, in this case we add following details as
`service.annotations` in the Objectstore Gateway Spec :
```
service:
annotations:
service.beta.openshift.io/serving-cert-secret-name: <name for
autogenerated secret>
```
More details about service serving cert can be found at :
https://docs.openshift.com/container-platform/4.6/security/certificates/service-serving-certificate.html
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Remove specific rgw config added to the mon configuration database when deleting a CephObjectstore
or when scaling down number of rgw
Signed-off-by: n.fraison <n.fraison@criteo.com>
When running on SDN, the container is not privileged and the network
stack is not exposed so running the gw on port 80 will not work (not
enough privileged).
Closes: https://github.com/rook/rook/issues/5106
Signed-off-by: Sébastien Han <seb@redhat.com>
The CRD watcher has been replaced by the new controller-runtime
framework.
This brings robustness in our operator, meaning that any resources that
are modified will be reconciled into the desired state.
Closes: https://github.com/rook/rook/issues/4937
Signed-off-by: Sébastien Han <seb@redhat.com>
rgw needs the fronted on its cli line to select the fronted as well the
port. This option is not supported in the mon config store.
Closes: https://github.com/rook/rook/issues/4471
Signed-off-by: Sébastien Han <seb@redhat.com>
The Ceph Mimic release introduced the monitor config store which allows
us to set configuration flag per daemon in a centralized fashion. Let's
use it for Mimic and newer and keep the CLI flag of daemon startup for
older versions.
The main benefit is that the config of a daemon can now be overridden.
Closes: https://github.com/rook/rook/issues/4307
Signed-off-by: Sébastien Han <seb@redhat.com>
Failure to create a keyring prior to creating a deployment
dependent on that keyring created a race conditition resulting
in an intermittent and unnecessary pod failure. This change
creates a keyring prior to any RGW deployment.
Partially fixes: #4089
Signed-off-by: egafford <egafford@redhat.com>
Allow users to specify overrides in the CephCluster CRD. The override
ConfigMap still exists for emergency situations and is mounted into
daemon pods directly instead of being merged into a Rook-created config
file.
Rook Ceph no longer uses a config file for managing daemons with the
exception of the OSDs which still generate a config in an init
container.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
The beast fronted configuration for rgw is quite different than civetweb
so we adapted the construct of the arg line based on that configuration
requirement.
Closes: https://github.com/rook/rook/issues/3411
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit does multiple things:
* remove support for AllNodes where we would deploy one rgw per node on
all the nodes.
* a transition path is implemented in the code so that if someone has an
existing deployment, daemonsets will be removed and replaced by an
deployments.
* when using "instances", each rgw deployed has its own key which makes
Ceph reporting the exact number of rgw running, see:
```
[root@rook-ceph-operator-775cf575c5-bh4sr /]# ceph -s
cluster:
id: 611fcf39-0669-4864-9a12-debb35c0397a
health: HEALTH_OK
services:
mon: 3 daemons, quorum a,b,c (age 12h)
mgr: a(active, since 12h)
osd: 3 osds: 3 up (since 12h), 3 in (since 12h)
rgw: 3 daemons active (my.store.a, my.store.b, my.store.c)
data:
pools: 6 pools, 600 pgs
objects: 235 objects, 3.8 KiB
usage: 3.0 GiB used, 84 GiB / 87 GiB avail
pgs: 600 active+clean
```
Closes: https://github.com/rook/rook/issues/2474, https://github.com/rook/rook/issues/2957 and https://github.com/rook/rook/issues/3245
Signed-off-by: Sébastien Han <seb@redhat.com>
As per: ceph/ceph#26599, Beast is now the
default fronted for rados gateway.
Newly created cluster as of Nautilus will use it by default.
Re-added version of 03587352d5Resolves: #2707
Signed-off-by: Sébastien Han <seb@redhat.com>
Configure the Ceph rgw daemon completely from the operator a la the
recent changes to the Ceph mon, mgr, and mds operators.
Create the rgw deployment or daemonset first, and then create the
keyring secret for the object store with its owner reference as the
corresponding deployment or daemonset. When the replication controller
is deleted, the secret is also deleted.
The RGW's mime.types file is now stored in a configmap with a different
file created for each object store. This is primarily just a means to
get the mime.types file into the rgw pod, but the added benefit is that
the administrator can modify the configmap, which could reduce
susceptibility to file type execution vulnerabilities (worst case).
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Deepika Upadhyay
Blaine Gardner
Artem Torubarov
Travis Nielsen
Lucas Henry
zhucan
Olivier
Jiffin Tony Thottan
Denis Egorenko
Satoru Takeuchi
n.fraison
Sébastien Han
Mateusz Los
egafford
Blaine Gardner