The canary that checks every RGW zone.json *_pool field is covered by
Rook's zonePoolNSSuffix map was inline in runObjectE2ETest, running
against the legacy per-pass store before the shared store existed.
Move it into tests/integration/object/zonepools as a standalone
shared-store consumer. It now validates the shared store's zone, which
carries the real shared-pool placements the canary is meant to guard,
and runs first among the shared-store packages so it still sees a fresh
zone.
Add a Sharedstore.Installer() accessor so the packaged canary can run
radosgw-admin inside the cluster while keeping the uniform
(t, k8sh, store) package entry signature.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add tests/integration/object/bucket/lifecycle, converting the bucket
lifecycle slice of the legacy testObjectStoreOperations onto the shared
store. An OBC is created with a bucketLifecycle additionalConfig, the
rules are verified on the rgw bucket over the S3 API, then updated and
removed with the bucket polled until it reflects each change. The
lifecycleCmpOpts comparison options move out of the suite dispatcher into
the package, and the Ceph v19.2.3 removal-regression skip now discovers
the CephCluster by listing the store's namespace. It runs against the
shared store in both the TLS and non-TLS passes.
Drop the converted lifecycle block from the legacy test.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Generalize the object suite's sharedstore.Create to accept the target
namespace, store name, and RGW instance count instead of the hardcoded
object-ns / sharedstore / single instance, then use it to stand up the
SmokeSuite object store.
TestObjectStorage_SmokeTest previously created its store with
runObjectE2ETestLite, which applies a CephObjectStore manifest and polls
for health on fixed intervals. It now builds the store through the
typed-client, watch-based sharedstore fixture and tears it down with
Destroy. lite-store is the only object store in the smoke namespace, so
the fixture safely owns the cluster-global .rgw.root realm pool that
Destroy deletes.
The object suite's own call is updated to pass its namespace,
"sharedstore", and one instance, preserving its existing behavior.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add tests/integration/object/bucket/policy, converting the bucket policy
slice of the legacy testObjectStoreOperations onto the shared store: an OBC
created with a bucketPolicy additionalConfig, the policy verified verbatim
on the rgw bucket over the S3 API, then updated and removed with the bucket
polled until it reflects each change. The policy JSON is generated from the
bucket name, which the legacy test hardcoded in the Resource ARN. It runs
against the shared store in both the TLS and non-TLS passes.
Drop the converted policy block from the legacy test.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add tests/integration/object/bucket/quota, covering both quota behaviors
from the legacy testObjectStoreOperations on OBCs of its own: user quota
via the OBC maxObjects additionalConfig (enforced, then raised and
re-enforced, synced on the ObjectBucket reflecting the new limit) and
bucket quota via bucketMaxObjects switched to bucketMaxSize. It runs
against the shared store in both the TLS and non-TLS passes.
Drop the converted S3-access and bucket-quota blocks from the legacy test;
the main OBC now exists only as a dependents fixture for the store
deletion checks.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add obc.Update, a get/mutate/update helper for live ObjectBucketClaims,
generalized from the notification suite's label updater, which becomes a
thin wrapper over it. The bucket quota conversion needs the same shape for
additionalConfig updates.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The README still described the util layout from before the package
consolidation, with ready, admin, sns, s3, and tls as standalone packages
and a code sample calling ready.ObjectStoreUser. The readiness predicates
live in wait4 and the client builders in client; update the package list,
the code sample, and the playbook references to match.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add tests/integration/object/bucket/rw, the first slice carved from the
legacy testObjectStoreOperations. It provisions an OBC on the shared store,
writes, reads back, and deletes an object over S3, and checks the OBC stays
Bound. It runs against the shared store in both the TLS and non-TLS passes.
Drop the converted put/get and OBC-revert subtests from the legacy test;
the user quota subtest now seeds both of the objects it needs. The
remaining quota, policy, lifecycle, and dependents slices convert in later
PRs.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Introduce util/obc as the home for ObjectBucketClaim test helpers: the
provisioner StorageClass constructor (moved out of util/fixture, where it
was a misplaced pure constructor), the create/bound and delete/absent
lifecycle waiters, and a per-OBC S3 client. The lifecycle waiters and S3
client are promoted from the notification suite, which built them inline as
their only consumer.
Retarget the StorageClass callers (topic/kafka, bucket/owner) and rework
the notification suite onto the shared helpers.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add client.NewS3Agent, which builds an rgw S3 agent for an object store
from a set of credentials and the store's endpoint. The object tests build
S3 clients from several credential sources (the store's dashboard admin
user, an OBC's provisioned secret); this factors out the endpoint and agent
construction they share.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Rework the bucket notification integration test onto the shared
CephObjectStore fixture and the wait4 toolkit, following the object
suite conversion playbook. The old test stood up its own object store
and drove everything through tests/framework/clients and kubectl with
fixed-interval polling.
The new tests/integration/object/notification package is an ordered
t.Run script on typed clients and watch-based waits. It deploys an HTTP
sink, drives a CephBucketTopic (HTTP endpoint), CephBucketNotification,
and a notification-labelled OBC, and verifies both end-to-end delivery
(matched in the sink's logs) and the notification configured on the rgw
bucket (read over the S3 API), covering the label add/remove and
notification-before-topic orderings.
Running against the shared store, it no longer creates a dedicated
object store and now exercises both the TLS and non-TLS suite passes.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Rework the COSI driver integration test onto the shared-store object test
toolkit: an ordered t.Run script with the (t, k8sh, store) signature, typed
clients, and watch-based waits, replacing kubectl-string manifests and
fixed-interval polling.
Drive the COSI bucket resources (BucketClass, BucketClaim, Bucket) with the
upstream sigs.k8s.io/container-object-storage-interface typed client, exposed
as k8sh.COSIClientset alongside the existing OBC client. Create the
CephCOSIDriver and its privileged user through typed clients, wait for the
driver Deployment with wait4, and verify the provisioned bucket through the
shared store's rgw admin client.
Install the COSI CRDs and central controller from the consolidated upstream
repo pinned to v0.2.2 (the former -api and -controller repos are retired) via
a kubectl -k fixture that is removed with t.Cleanup. The driver cannot trust
a TLS RGW endpoint, so the suite skips itself in the TLS pass rather than
being special-cased in the dispatcher.
Retire the now-unused COSIOperation client and the GetCOSIDriver,
GetBucketClass, and GetBucketClaim manifest helpers.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add a conventions doc and conversion playbook so the remaining old-style
object tests (store lifecycle, notifications, COSI, bucket rw/quota/policy/
lifecycle) can be converted to this pattern mechanically.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The object store user (caps/keys/opmask), bucket-owner, and bucket topic
tests already shared a CephObjectStore but polled with count-based
utils.Retry behind a wide entry signature. Move them onto the wait4 toolkit
and a slim (t, k8sh, store) signature: each is an ordered t.Run script using
typed clients and watch-based waits, with per-package check helpers for
repeated verification. Extend the shared store fixture to provision the
realm/zonegroup/zone pools and serve TLS, build its rgw admin and SNS clients
through the consolidated util/client package (replacing the separate
admin/sns/s3 packages), and run every package in both the TLS and non-TLS
passes (now separate parallel jobs).
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add the shared helpers the object test packages build on:
- wait4: watch-based Assert/Require waiters (Create/Delete/Condition/Absent)
that block on Kubernetes watch events rather than fixed-interval polling,
plus Eventually for non-k8s state (rgw admin/S3/SNS), a pod-log waiter, and
the readiness predicates used with them.
- secrets: verification of the Secret references CRDs publish in status.
- fixture: create-with-t.Cleanup helpers for pure-cleanup resources, and
constructors for ObjectBucketClaim test resources.
- client: rgw admin and SNS client builders, S3 credentials/endpoint, and the
store's TLS cert plus a verification-skipping HTTP client.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
This factors out the creation of a single CephObjectStore instance to be
shared between the individually namespaced object suite packages.
This reduces the object suite runtime by the cost of starting up and
tearing down an object store three times.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
This change cleans up and standardizes import statements across the Ceph operator code. It removes redundant or duplicate imports and reorganizes alias names for improved clarity and consistency. Additionally, the ST1019 exception was removed from .golangci.yaml now that the code complies with the rule.
Signed-off-by: Carlos Barria <cbarria@yahoo.com>
userSecretRef and passwordSecretRef fields are added to allow the Kafka
endpoint username and password to be supplied from a Kubernetes Secret,
rather than exposed as plaintext as part of the endpoint URI. If the
endpoint URI has HTTP basic auth user-id and user-pass components, they
are overridden by userSecretRef and passwordSecretRef.
Squid added bucket topic attributes for configuring the user-name and
password for pushing notifications to Kafka as an alternative to
encoding credentials into the URI. However, these attributes are not
supported under Reef. Thus, this initial implementation relies on URI
mangling for compatible with both Reef and Squid. Future work could
using Ceph version detection and switch to the new attributes to be used
with Squid and/or the implementation could be converted exclusively to
use the new attributes once Rook has dropped support for Reef.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Joshua Hoblitt
Joshua Hoblitt
Travis Nielsen
Carlos Barria