This adds markdown link checking in th documentation sources
to the daily-nightly CI jobs.
The purpose of this is to catch breaking external links to help
keep out documentation clean.
Catching breaking external links is best done in the daily CI runs
instead of checking links in PRs. link checking in PRs would prevent
changes in PRs from newly introducing broken links.
Signed-off-by: Michael Adam <obnox@samba.org>
After some markdown link fixes,
make docs-build failed due to some non-existing anchors in documentation
sources.
This changes fixes the docs-build by rectifying anchors.
Assisted-by: IBM-Bob
Signed-off-by: Michael Adam <obnox@samba.org>
This change fixes some of the broken links in the markdown
documentatoion sources found by the markdown link checker.
Assisted-by: IBM Bob
Signed-off-by: Michael Adam <obnox@samba.org>
This adds a make target lint.markdown-links
for checking forbroken links in the markdown
documentation sources.
Thid is intended to be used in the CI as well as
locally by developers
The link checking is implemented using the markdown-link-check tool
but it is self-contained in that it does not require the
tool to be installed on the system.
The only prerequisite for using this target is to have
a working docker or podman available.
Assisted-by GitHub copilot
Assisted-by: Google Antigravity/Gemini
Assisted-by: IBM Bob
Signed-off-by: Michael Adam <obnox@samba.org>
The go.mod.check target generates pkg/apis/go.mod from the root go.mod
and must remove the root-only replace directive that points
github.com/rook/rook/pkg/apis to ./pkg/apis. The sed pattern expected
the line to start with "replace", but the actual line is tab-indented
inside the replace block, so the directive was never removed.
Update the sed pattern to match optional leading whitespace and remove
the incorrect self-replace from pkg/apis/go.mod.
Signed-off-by: Malay Kumar Parida <mparida@redhat.com>
The minikube-to-kind conversion moved the master/release integration
tests to kind, which selects the Kubernetes version via the
kindest/node:<version> image tag. There is no published
kindest/node:v1.32.13 image (Docker Hub returns 404); the latest 1.32
node image kind ever built is v1.32.11, since later kind releases
advanced to 1.33+ and never produced a newer 1.32 patch.
Pin the on-release matrix's 1.32 entry to v1.32.11 so the 1.32-line
coverage keeps a node image that actually exists. The other matrix
versions (v1.31.14, v1.34.8, v1.36.1) already have published images.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
currently, drbd hostname was ready using command
but now the change we use kubernetes built-in api
to read hostname from `spec.nodeName`.`
Signed-off-by: subhamkrai <srai@redhat.com>
shouldReconcileChangedNode only diffed the node spec, so changes to node
labels, including rook's own topology.rook.io/* labels, were never detected.
Relabeling a node therefore did not trigger a CephCluster reconcile until some
other condition eventually did.
Trigger a reconcile from the node update predicate when one of the OSD topology
labels rook cares about changes. The whitelist reuses the canonical label set
from topology.GetDefaultTopologyLabels() (the hostname, the Kubernetes
region/zone labels, and the topology.rook.io/* labels), so a reconcile is
scoped to topology changes instead of firing on any label or annotation change.
The UpdateFunc returns true directly for these changes because onK8sNode()
returns false for a node that is already an OSD host, which would otherwise
swallow the relabel.
Resolves#17652
Signed-off-by: Anas Khan <anas@anaskhan.me>
Bump the most recent kubernetes version exercised by the integration and
canary matrices from v1.35.5 to v1.36.1. The kindest/node image for 1.36 is
provided by the kind v0.32.0 pinned by the kind-conversion commit. The older
matrix versions are unchanged.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Converting the bucket quota tests to typed clients left the framework
helpers and package globals they were the last consumers of unused. Remove
UpdateObc, CheckOBMaxObject, GetAccessKey, and GetSecretKey from the bucket
client, GetEndPointUrl from the object client, and the orphaned object-test
globals.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add tests/integration/object/bucket/quota, covering both quota behaviors
from the legacy testObjectStoreOperations on OBCs of its own: user quota
via the OBC maxObjects additionalConfig (enforced, then raised and
re-enforced, synced on the ObjectBucket reflecting the new limit) and
bucket quota via bucketMaxObjects switched to bucketMaxSize. It runs
against the shared store in both the TLS and non-TLS passes.
Drop the converted S3-access and bucket-quota blocks from the legacy test;
the main OBC now exists only as a dependents fixture for the store
deletion checks.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add obc.Update, a get/mutate/update helper for live ObjectBucketClaims,
generalized from the notification suite's label updater, which becomes a
thin wrapper over it. The bucket quota conversion needs the same shape for
additionalConfig updates.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The integration and canary suites ran on a single-node minikube
`driver: none` cluster, where the kubelet ran directly on the GitHub
runner, so the host docker daemon doubled as the cluster runtime and
host block devices and host paths were directly visible to pods. Replace
that with a kind cluster.
Every suite creates its cluster through the shared
integration-test-setup-cluster-resources composite action, so the
conversion is centralized there and converts the smoke, object, helm,
keystone, multi-cluster, upgrade, on-release, nightly, encryption-KMS and
all canary jobs at once.
- Replace the setup-minikube step with helm/kind-action, selecting the
kubernetes version via the kindest/node image tag and creating a
single-node cluster from a new kind config (kind pinned to v0.32.0 for
reproducibility).
- Drop the cri-dockerd install; kind nodes use their built-in containerd.
- Add a kind config that bind-mounts the host /dev, /var/lib/rook and
/run/udev into the node so the existing host-based disk-prep helpers
(use_local_disk*, create_partitions_for_osds, blockDevicePV.sh,
localPathPV.sh, ...) keep working unchanged: devices and partitions
created on the host appear in the node and in the OSD pods that
hostPath-mount the node /dev, and ceph-volume can read the host udev
database it needs to inventory disks.
- Prepare the kind node for the host-level operations rook runs against the
underlying host: remount /sys read-write so CSI's kernel RBD mapping
(`rbd map --device-type krbd`, which writes /sys/bus/rbd) works, and install
lvm2 and cryptsetup, which rook runs in the node's mount namespace to
provision LVM- and encryption-backed OSDs. kindest/node images provide none
of this; the minikube driver:none runner host did.
- Route the Service and pod CIDRs from the runner to the kind node so
host-side tests (the `go test` process runs on the runner) can reach
in-cluster ClusterIPs, e.g. an S3 request to the RGW service. With minikube
driver:none the runner already shared the cluster network.
- Load locally built images into the cluster. Under minikube `driver: none`
the built image was already in the cluster runtime; under kind it must be
imported, so build_rook and create_helm_tag now import their images into
each node's containerd through a new load_image_into_cluster helper (via the
node's ctr, which avoids the kind/kindest-node containerd-config version
skew that breaks `kind load docker-image`).
- Point Vault's kubernetes-auth at the in-cluster API endpoint
(kubernetes.default.svc) instead of the kubeconfig server URL: kind exposes
that as https://127.0.0.1:<port>, unreachable from the in-cluster Vault pod,
so OSD encryption-key retrieval via k8s-auth failed.
- Replace the remaining direct minikube references in the canary workflow: a
`minikube kubectl` call and the external-cluster topology values.
- Adapt host-name assumptions that only held under driver:none: resolve the
disk-cleanup job by the k8s node name rather than the runner hostname, and
let kind-action ignore post-job cluster-teardown failures (the runner is
ephemeral; nvme/multus devices can wedge `docker rm` of the node).
- Update stale comments that described the CI environment as minikube.
- Move the multus integration test's kind config under tests/config too, so
both kind cluster configs live in one place.
create-dev-cluster.sh and other local-dev tooling are intentionally left on
minikube.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The README still described the util layout from before the package
consolidation, with ready, admin, sns, s3, and tls as standalone packages
and a code sample calling ready.ObjectStoreUser. The readiness predicates
live in wait4 and the client builders in client; update the package list,
the code sample, and the playbook references to match.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add tests/integration/object/bucket/rw, the first slice carved from the
legacy testObjectStoreOperations. It provisions an OBC on the shared store,
writes, reads back, and deletes an object over S3, and checks the OBC stays
Bound. It runs against the shared store in both the TLS and non-TLS passes.
Drop the converted put/get and OBC-revert subtests from the legacy test;
the user quota subtest now seeds both of the objects it needs. The
remaining quota, policy, lifecycle, and dependents slices convert in later
PRs.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Introduce util/obc as the home for ObjectBucketClaim test helpers: the
provisioner StorageClass constructor (moved out of util/fixture, where it
was a misplaced pure constructor), the create/bound and delete/absent
lifecycle waiters, and a per-OBC S3 client. The lifecycle waiters and S3
client are promoted from the notification suite, which built them inline as
their only consumer.
Retarget the StorageClass callers (topic/kafka, bucket/owner) and rework
the notification suite onto the shared helpers.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Add client.NewS3Agent, which builds an rgw S3 agent for an object store
from a set of credentials and the store's endpoint. The object tests build
S3 clients from several credential sources (the store's dashboard admin
user, an OBC's provisioned secret); this factors out the endpoint and agent
construction they share.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
find_extra_block_dev piped find_extra_block_devs into head -1. Since
find_extra_block_devs prints every eligible disk and ends with echo
"$devs", head closes the pipe after the first line and the producer's
echo races into SIGPIPE (echo: write error: Broken pipe), failing the
command substitution under pipefail/errexit.
The race only fires reliably now that the runners present two data disks
(sdb and sdc), so find_extra_block_devs emits multiple lines; with a
single disk the one-line output almost never triggered it.
Read the producer to completion with mapfile via process substitution
and return the first element, mirroring find_second_block_dev, so there
is no downstream head to close the pipe early.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Fix duplicate words, incorrect articles (a/an), it's/its, and other small
grammar mistakes in Go comments and user-facing messages across pkg/, cmd/,
and tests/.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Converting the bucket notification test to typed clients left the
framework notification plumbing it was the last consumer of unused.
Remove the NotificationOperation client, TopicClient.CreateHTTPServer,
BucketClient.CreateObcNotification and CheckBucketNotificationSetonRGW
(and the now-dead UpdateObcNotification helpers), and the
GetBucketNotification and GetOBCNotification manifest builders.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Rework the bucket notification integration test onto the shared
CephObjectStore fixture and the wait4 toolkit, following the object
suite conversion playbook. The old test stood up its own object store
and drove everything through tests/framework/clients and kubectl with
fixed-interval polling.
The new tests/integration/object/notification package is an ordered
t.Run script on typed clients and watch-based waits. It deploys an HTTP
sink, drives a CephBucketTopic (HTTP endpoint), CephBucketNotification,
and a notification-labelled OBC, and verifies both end-to-end delivery
(matched in the sink's logs) and the notification configured on the rgw
bucket (read over the S3 API), covering the label add/remove and
notification-before-topic orderings.
Running against the shared store, it no longer creates a dedicated
object store and now exercises both the TLS and non-TLS suite passes.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Akanksha Trehun
Travis Nielsen
Michael Adam
Malay Kumar Parida
Parth Arora
Joshua Hoblitt
Joshua Hoblitt
subhamkrai
Anas Khan
weifanglab
subham rai
dependabot[bot]