Added failureDomains and osdsPerDomain fields to the
ErasureCodedSpec, which maps to the crush-num-failure-domains and
crush-osds-per-failure-domain Ceph EC profile parameters. This enables
creating EC pools that distribute chunks across fewer, larger hosts
without needing one host per data chunk.
Signed-off-by: Prabhala Tara Aasrita <taraprabhala@Prabhalas-MacBook-Pro.local>
(cherry picked from commit 715f38de91)
Add coverage for GetPort defaults and custom values, Service and probe
port wiring, Ganesha NFS_Port config, and create-or-update when
spec.server.port changes on an existing Service.
Signed-off-by: raaizik <raaizik@yahoo.com>
(cherry picked from commit b97296c47c)
When NFS runs with host networking, port 2049 may already be in use.
Add CephNFS spec.server.port (default 2049), wire it through Ganesha
config, the operator Service, and probes, and regenerate CRDs.
Signed-off-by: raaizik <raaizik@yahoo.com>
(cherry picked from commit 5e36fa4bc3)
When the active MGR IP already exists at a non-zero position in the
externalMgrEndpoints list, the rotation logic that replaces endpoints[0]
creates duplicate IPs. Also guards against mgr map failure which could
overwrite endpoints[0] with an empty string.
Fixes: #17761
Signed-off-by: beyondcloud-co <87993136+beyondcloud-co@users.noreply.github.com>
(cherry picked from commit 8134d683e3)
The go text package v0.37 has a vulnerability, thus update
to the latest version of the package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 8c397f6381)
assignMons schedules each mon in its own goroutine and shares a
failedMonSchedule bool to record whether any of them failed. The
resultLock mutex already guards the shared c.mapping.Schedule map
write, but the three failedMonSchedule = true assignments were done
without holding the lock. When two or more mons fail to schedule at
the same time (for example when waitForMonitorScheduling errors, the
node choice is nil, or getNodeInfoFromNode fails) their goroutines
write the flag concurrently, which is a write-write data race.
Guard the failedMonSchedule writes with the existing resultLock, the
same lock the goroutines already use for the map update. The post-Wait
read is left as is since the WaitGroup orders it after every write.
Add a regression test that fails scheduling for multiple mons at once
and confirms assignMons returns an error; run it with -race to catch
the race.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
(cherry picked from commit 0bbd5de84c)
decodePeerToken logged the whole decoded PeerToken with "%+v" at debug
level. PeerToken embeds Key, the peer cluster's cephx auth secret, so
running the operator with ROOK_LOG_LEVEL=DEBUG wrote that credential to
the operator logs in plaintext (CWE-532).
Log only the non-sensitive fields (fsid, client id, mon host, namespace),
which keep the message useful for diagnostics, and drop Key. Add a
regression test that captures the debug output and asserts the key is
never present.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
(cherry picked from commit 1013a8effc)
The Placement field comments on RBDMirroringSpec and
FilesystemMirroringSpec both read "The affinity to place the rgw pods",
copied from GatewaySpec. Neither spec manages RGW pods: RBDMirroringSpec
configures the rbd mirror daemon and FilesystemMirroringSpec the
cephfs-mirror daemon.
Update the comments to name the correct pods, matching the wording
already used by the sibling Resources and PriorityClassName fields in
each struct, and regenerate Documentation/CRDs/specification.md so the
published CRD API reference matches.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
(cherry picked from commit b26cad41ef)
currently there was a bug in the code where it didnt removed
the account from the ceph cluster during intial intialize
Signed-off-by: parth-gr <partharora1010@gmail.com>
(cherry picked from commit cca7020f0b)
CephFS reconciliation loop on Filesystem creation was swallowing errors.
The err variable which contained the error was replaced with the resulted error from the
kubernetes update status call on the CRD.
This commit fixes that and aggregate both errors in case there is both a kubernetes status update failure and a creation error.
Signed-off-by: Maxime Bertin <mbertin@luccasoftware.com>
(cherry picked from commit 0696b24d9c)
currently, drbd hostname was ready using command
but now the change we use kubernetes built-in api
to read hostname from `spec.nodeName`.`
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit c49ef5f079)
shouldReconcileChangedNode only diffed the node spec, so changes to node
labels, including rook's own topology.rook.io/* labels, were never detected.
Relabeling a node therefore did not trigger a CephCluster reconcile until some
other condition eventually did.
Trigger a reconcile from the node update predicate when one of the OSD topology
labels rook cares about changes. The whitelist reuses the canonical label set
from topology.GetDefaultTopologyLabels() (the hostname, the Kubernetes
region/zone labels, and the topology.rook.io/* labels), so a reconcile is
scoped to topology changes instead of firing on any label or annotation change.
The UpdateFunc returns true directly for these changes because onK8sNode()
returns false for a node that is already an OSD host, which would otherwise
swallow the relabel.
Resolves#17652
Signed-off-by: Anas Khan <anas@anaskhan.me>
(cherry picked from commit 47b4167b70)
Fix duplicate words, incorrect articles (a/an), it's/its, and other small
grammar mistakes in Go comments and user-facing messages across pkg/, cmd/,
and tests/.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit c0eb360041)
Several godoc comments led with a stale or incorrect identifier, left
over from renames, exported/unexported changes, copy-paste between
sibling declarations, or plain typos. As a result the documented name no
longer matched the function, method, type, or var it describes. Correct
each leading word to the name of the declaration it documents.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 49612461a4)
this commit implement api to configure
ceph health warning mute/unmute.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 9a6c5842bc)
adding api changes to mute/unmute ceph warnings
based on user configuration. The field is map[string]string
key is ceph warning and value is duration how longs
the warnings will be muted.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit ff5f3baa58)
When the operator starts, there is no need to handle the
node watcher immediately while the controller runtime
cache is being initialized. The cluster will anyway be
fully reconciled and create new OSDs if needed. The node
watcher is only intended for triggering a reconcile when
a node is added or updated at a later time after a sync
would anyway be triggered.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 7803fac10d)
currently k8s add the default eviction time as 300 sec
setting it to 5 sec for the quick re-schedule
Signed-off-by: parth-gr <partharora1010@gmail.com>
(cherry picked from commit 9e0b6e5bbe)
When the OSD activate init container cannot match its configured
device, it falls back to a bare 'ceph-volume raw list', which
enumerates every block device on the node and opens each one to read
bluestore labels. Opening a krbd device whose backing cluster is
unhealthy can block in uninterruptible sleep; when that device's I/O
depends on the OSD being activated, activation deadlocks.
Build the fallback scan list explicitly with lsblk and exclude the
network-backed device types that can hang when their backing storage
is unavailable (rbd, nbd, drbd) along with zram (volatile RAM). Rook
never provisions OSDs on any of these, so the fallback can never
legitimately need to find an OSD on them. loop devices are kept,
since OSDs on loop devices are supported for CI and local testing.
Fail fast if the filtered list is empty rather than degenerating back
to a full scan.
Note this hardens only the rename-fallback path. The deadlock observed
in CI (kernel hung tasks under 'ceph-volume raw activate' opening a
stalled /dev/rbd0; actions runs 27325963851, 27333888237, and
27378983111, which reproduced it on this PR's own head) ultimately
originates inside ceph-volume: 'raw activate' runs an unconditional
LUKS2/TPM2 discovery sweep over all block devices even when given an
explicit --device. That requires a ceph-volume fix; this change makes
the Rook-controlled scan safe regardless of ceph version.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit da07f0c476)
The realm system user's access and secret keys are generated by
GeneratePassword() and then wrapped in base64. GeneratePassword()
deliberately excludes '/' from the access key character set, but the
base64.StdEncoding wrap reintroduces it: its alphabet contains '/' and
'+', and the 14-character input always produces trailing '='. The
encoded string is the literal key used in S3 requests.
An access key containing '/' breaks AWS SigV4 credential scope parsing
("<access-key>/<date>/<region>/<service>/aws4_request" is split on
'/'), so "radosgw-admin realm pull" against the realm endpoint fails
permanently with "request failed: (22) Invalid argument" (HTTP 400),
and a CephObjectRealm pulling that realm can never reconcile.
This is the dominant cause of the "deploy second cluster rook"
failures in the rgw-multisite-testing canary job: every sampled failure
had a generated access key containing '/' and looped on EINVAL for the
whole 600s wait window, while runs with slash-free keys pulled the
realm successfully.
Encode both keys with base64.RawURLEncoding instead, whose alphabet
(A-Za-z0-9-_, unpadded) is safe in credential scopes, URLs, and shell
arguments. Only newly created realm secrets are affected; existing
secrets are not modified by the reconciler.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 573e2f81e7)
Remove dead code from the k8sutil package; none of these symbols had any
callers anywhere in the repo (verified with deadcode, staticcheck U1000,
and repo-wide grep, tests included):
- patcher.go: delete file (Patcher type, NewPatcher, Patch)
- replicaset.go: delete file (DeleteReplicaSet)
- customresource.go: WatchCR (the CustomResource type remains in use)
- deployment.go: WaitForDeploymentImage
- job.go: WaitForJobCompletion
- k8sutil.go: GetK8SVersion
- pod.go: ConfigOverrideMount, ConfigOverrideVolume, GetPodLog
- prometheus.go: DeleteServiceMonitor
- service.go: DeleteService
- volume.go: NodeConfigURI, BinariesMountInfo, YamlToVolumes,
YamlToVolumeMounts, and the BinariesMountPath const (only used by the
removed BinariesMountInfo)
Now-unused imports are dropped from each file. The shared
deleteResourceAndWait helper remains (still used by DeleteDeployment).
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit b29e89ef67)
The OSDPerfStats struct had no references anywhere in the repo (verified
with repo-wide grep and an exported-symbol audit); the code that consumed
this unmarshal target was removed long ago.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit ef9761d4df)
NewSharedStorageAndWorkerNodesValidationTestConfig was a wrapper that
simply returned NewDefaultValidationTestConfig() and had no callers
anywhere in the repo (verified with deadcode and repo-wide grep, tests
included). The sibling constructors are all referenced by the multus
validation CLI and remain.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 2433565238)
WriteFile and PathToProjectRoot had no callers anywhere in the repo
(verified with deadcode, staticcheck U1000, and repo-wide grep, tests
included). WriteFile has long been superseded by direct os.WriteFile
usage. Drop the now-unused bytes, fmt, and runtime imports; the file's
remaining helpers (WriteFileToLog, CreateTempFile) are still in use.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 8df364df98)
Remove four dead functions from pkg/daemon/ceph/client; none had any
callers anywhere in the repo (verified with deadcode, staticcheck U1000,
and repo-wide grep, tests included):
- auth.go: AuthGetOrCreate
- command.go: ExecuteRBDCommandWithTimeout
- filesystem.go: MarkFilesystemAsDown
- status.go: IsClusterCleanError (IsClusterClean itself remains in use)
No imports become unused by these removals.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 278453e10c)
kmod.go was entirely dead: IsBuiltinKernelModule, LoadKernelModule,
CheckKernelModuleParam, and the getKernelVersion helper had no callers
anywhere in the repo (verified with grep, deadcode, and staticcheck).
Remove the file.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 0b6d81d83a)
daemonset.go was entirely dead: CreateDaemonSet, DeleteDaemonset,
AddRookVersionLabelToDaemonSet, and GetDaemonsets had no callers anywhere
in the repo (verified with grep, deadcode, and staticcheck). Rook does not
manage any DaemonSets via these helpers. Remove the file.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit c8ad81a3cf)
Make the NVMe-oF gateway image optional in the CR spec.
When not specified, the operator fetches the image from
the Ceph mon config store using the key
mgr/cephadm/container_image_nvmeof. Falls back to the
hardcoded default if the config is unavailable.
Signed-off-by: Oded Viner <oviner@redhat.com>
(cherry picked from commit 8b4fd67e17)
The Clear method on updateQueue had no callers anywhere in the repo
(verified with grep, deadcode, and staticcheck). The queue is emptied via
Remove/Pop in practice. Remove the dead method.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 50c95d343e)
When RGW and MDS intances are reduce to 1, the older RGW and MDS PDB
instance (with maxUnavailabe=1) are still available. This blocks node
drain when the node, on which the single RGW and MDS instances are
running, is drained.
This PR deletes any stale MDS and RGW PDB when the only 1 MDS and 1 RGW
instances are available.
Signed-off-by: Santosh <sapillai@redhat.com>
(cherry picked from commit 3396198e9c)
VerifyPodLabels had no callers anywhere in the repo, and it was the only
caller of VerifyAppLabels, which in turn was the only caller of the
checkLabel and combineErrors helpers. Remove the whole dead cluster (and
the now-unused strings, github.com/pkg/errors, and ceph/controller
imports). AssertLabelsContainCephRequirements in the same file remains and
is unaffected. Verified with grep, deadcode, and staticcheck.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit ac75488621)
container.go in the ceph operator test-helper package was entirely dead:
ContainerTestDefinition, its TestContainer method, and the logCommandWithArgs
helper had no references anywhere in the repo (no test or non-test callers,
verified with grep, deadcode, and staticcheck). The package doc comment is
also present on info.go, so removing this file leaves the package documented.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 9efb7f351f)
IsModuleInSpec has no callers anywhere in the repository (verified with
deadcode and grep). Remove the dead code.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit f8c0d36a50)
Remove the following unreferenced symbols:
- (*S3Agent).CreateBucketNoInfoLogging
- (*S3Agent).DeleteBucket (wrapper; callers use the SDK client directly)
- GetBucketsStats
- ObjectBuckets type and its Len/Less/Swap sort.Interface methods
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit ae481f0f43)
Remove (*BucketPolicy).EjectPrincipals and (*PolicyStatement).EjectPrincipals,
which had no callers outside policy.go. Also drop the duplicate PutBucketVersioning
entry in AllowedActions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit df30bf66e8)