External CephObjectStores already have endpoints defined by
spec.gateway.externalRgwEndpoints, and if the external store is
configured with TLS (HTTPS), the store's certificates will likely not
accept connections intended for the Service endpoint Rook creates. Some
users might not be able to easily add the service endpoint to their
certificates. Therefore, don't even bother creating a Service for
external clusters.
This does introduce a few issues. The Service seems to have been
initially created to allow multiple external RGW endpoints to be
addressable via a single address in Rook. For all connections to an
external CephObjectStore with multiple endpoints, simply choose an
endpoint at random. Random selection will prevent Rook from failing to
create buckets or users on an external store if one of the external
store's endpoints fails.
The latest OBC library (lib-bucket-provisioner) allows updating the
endpoints on ObjectBuckets after they are created. This allows Rook
users to change endpoints on external CephObjectStores without breaking
all existing OBCs. It requires implementation of the new GetUserID()
library call, requires updating Provision() and Grant() calls to be
idempotent, and it requires removing the Update() call.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The `periodWillChange()` implementation to detect map keys to ignore.
Previously, we relied on the `GoString()` method to output the same
format always to determine the current path to the
map[string]interface{} node, but that may change depending on the underlying
implementation of go-cmp.
Instead, parse the go-cmp `Path` step by step and construct a JSON path
to the node in a stable format.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Remove the health checker for CephObjectStore. The liveness and
readiness probes go through the same code paths in RGW as creating
buckets without as much affect on the storage backend.
Full discussion: https://github.com/rook/rook/issues/11031
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Introduce a new env variable ROOK_CSI_IMAGE_PULL_POLICY in rook operator configmap which should be used to
customize the imagePullPolicy for the csi driver and imagePullPolicy property in cephVersionSpec for ceph pods.
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
if Multus is enabled the clusterinfo should be updated with
network as multus as to run the ceph cmds in remote
executor
Signed-off-by: parth-gr <paarora@redhat.com>
OBC provisioner ignores insecure skip flag even if it is declared for
TLS auth, hence bucket creation was failing
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Codespell had its first release in a year yesterday and added some
dictionary rules that are now being flagged in our code. Only one
issue required changes to our code, so modify the codespell config to
allow the words we want to allow.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
User can define his desired endpoint list in Zone CR so that it will
overwrite the default service name for rgw.
Resolves#6432
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
The tool gofmt in go 1.19 requires certain formatting
in the comments section for better rendering.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The RGW support server side encryption with help of s3 protocol, till
now the `sse:kms` was support in which keys will be provided by the user
and but it will be saved in external management service like vault. Now
the support for `sse:s3` is added so the entire encryption key
management is performed by RGW itsels.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
In object package code, external rgw server check is whether ceph cluster
is external instead of rgw. Correcting such scenarios in the code base.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Return the error instead of hiding it, err could be non-nil for a lot of
reasons, eg: a canceled context or a non-existing object.
Signed-off-by: Sébastien Han <seb@redhat.com>
Check the radosgw-admin realm user list per object store instead of relying
on the ceph dashboard get-rgw-api- command.
Resolves#9099
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The CSI package needs to load clusterInfo, today this code is in the mon
package which makes the call of LoadClusterInfo impossible without
having a circular import.
Signed-off-by: Sébastien Han <seb@redhat.com>
The access/secrets keys for the user struct need to allocate only if
ceph user creation succeeds.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
This removes double package imports. Example:
```
"github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
```
Only one is now being used as shown in go-staticcheck ST1019
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
For supporting features like service account authentication for vault
KMS , a service account account need to attach with pod.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Previously, the struct maintaining the list of cluster was still
initialized with a cluster item. Then the monitoring check will see that
the cluster is part of the struct already and thus won't run the
monitoring go routine again.
Now each time we cancel the context, we also remove the cluster item
from the map so that when the controller runs again, the monitoring
struct is re-populated and the go routine runs and statuses are updated.
Closes: https://github.com/rook/rook/issues/9911
Signed-off-by: Sébastien Han <seb@redhat.com>
The aws go lang sdk needs value for region, it is set differently in
various part of current code. With PR the value is always `us-east-1` so
that it will work RGW server without any issues.
This reverts commit 280c29f330.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
adding observedGeneration field in the cephcluster cr
status for having better control on reconciling,
as observedGeneration field will be updated by the controller
Closes: https://github.com/rook/rook/issues/9673
Signed-off-by: parth-gr <paarora@redhat.com>
Added few test cases to check for the following:
* creation of an external object store
* deletion of an external object store
* creation of an external object store with a missing secret
* creation of an external object store with ExternalRGWEndpoints set to
nil.
Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>
ReportReconcileResult should never be given an object that is nil. It is
impossible to force compile-time checking for this because the object's
type is an interface. We can't force this, but we can rework the
function definition to handle more cases where the object may not be
returned fully-complete, and we can rework callers to return structs
(not pointers-to-structs) so it is less likely a caller will pass nil
to the function.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Added unit test for probes changes and
did re-naming of liveness probes file
as it will contain multiple types of probes
Signed-off-by: parth-gr <paarora@redhat.com>
currently if multisite is not configured Rook sets values for
zone/zonegroup/realm as object store name which is only valid for
internal ceph object store. This may not exists for rgw running in
external cluster.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Jiffin Tony Thottan
Blaine Gardner
Travis Nielsen
subhamkrai
Avan Thakkar
parth-gr
zhucan
Joseph Lee
Eng Zer Jun
Josh Soref
Sébastien Han
Alexander Trost
Divyansh Kamboj
Pranshu Srivastava