if the csi secret is updated or a client profile is updated
we need a re-reconile of rns
So added the watcher for it
Signed-off-by: parth-gr <partharora1010@gmail.com>
Changes:
- Use a dedicated .nvmeof pool (via CephBlockPool CR named
builtin-nvmeof) for the NVMe-oF gateway internal state.
- Use a separate nvmeof pool for the StorageClass data.
- Remove the pool field from the CephNVMeOFGateway CRD.
The gateway now always uses the .nvmeof pool, hardcoded
in the operator.
- Add .nvmeof to the allowed CephBlockPool name overrides.
- Create a production example nvmeof.yaml (instances: 2,
replicas: 3) and a CI-only nvmeof-test.yaml (instances: 1,
replicas: 1).
- Update documentation and CI test script accordingly.
Signed-off-by: Oded Viner <oviner@redhat.com>
archiveCrash is meant to silence the RECENT_CRASH health warning for a
removed OSD by archiving its crash, but the early-return guard was
inverted. GetCrashList unmarshals `ceph crash ls` into a non-nil slice on
every success, so `if crash != nil` was always true. The function always
logged "no ceph crash to silence" and the archive loop was unreachable, so
OSD removal never archived the crash and the warning lingered. Guard on
`len(crash) == 0` so only a genuinely empty list short-circuits.
Reviving the loop surfaced two latent bugs it had always masked. On a
non-empty list with no entry for the removed OSD the crash id stayed empty
and the code ran `ceph crash archive ""`, which the mgr rejects with EINVAL
and logs as a spurious error during routine OSD removal, so skip the
archive when no crash matches. The loop also broke on the first match, but
`ceph crash ls` is oldest-first and includes already-archived entries, so
an OSD with several crashes had only its oldest one archived while
RECENT_CRASH persisted; archive every matching entry instead of just the
first.
Extend TestArchiveCrash to cover all four cases: a single matching crash, a
list with several matching crashes that are all archived, an empty list,
and a non-empty list with no matching entry where nothing is archived.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
In createOrUpdateCephUser, when the desired user carries no explicit
keys the reconciler falls back to the live RGW user's keys. If that live
user also has zero keys the code intends to fail, but it built the error
with errors.Wrapf(err, ...) at a point where err is already nil (the
prior SetUserQuota error was handled and returned just above).
errors.Wrapf(nil, ...) returns nil, so the failure was swallowed and
createOrUpdateCephUser returned success on a user the operator itself
flagged as broken.
The reconcile then continued to generateCephUserSecret, which indexes
userConfig.Keys[0] to populate the Kubernetes secret and panicked on the
empty key slice. The reconciler's deferred RecoverAndLogException caught
and logged that panic, so the reconcile was abandoned before it reached
the Ready status update; and because the recovered Reconcile returns a
zero Result with a nil error, the request was not requeued either. The
user was left neither marked Ready nor retried.
Construct the error with errors.Errorf so the intended failure is
surfaced instead of being swallowed. Add a regression test that returns
a keyless live user and asserts a non-nil "no keys set" error.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
this commit add support for updating rgw caps with
`user-info-without-key` and `accounts` to cephobjectstoreuser crd
Adding the check for min ceph version from which these caps
are available.
Co-Authoured-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Signed-off-by: subhamkrai <srai@redhat.com>
Ceph reports bytes_quota in `ceph fs subvolumegroup info` as the JSON
string "infinite" when no quota is set, and as a number otherwise. The
subvolumeGroupInfo.BytesQuota field is typed int64, so json.Unmarshal
fails with "cannot unmarshal string into Go value of type int64" for any
subvolume group without a quota (the default).
That unmarshal error is returned from getCephFSSubVolumeGroupInfo and
swallowed in CreateCephFSSubVolumeGroup: exec.ExitStatus reports ok=false
for the wrapped error, so the non-ENOENT guard does not fire and the
dedicated resize path (which enforces --no-shrink) is skipped.
Add an UnmarshalJSON on subvolumeGroupInfo that accepts either a number
or the "infinite" sentinel, mapping "infinite" to 0 (unlimited) so the
existing CmpInt64 comparison keeps working. Add a table-driven test
covering the finite and infinite cases.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Add annotations to RBD node secrets in
import-external-cluster.sh so that externally
created secrets with
custom names can be discovered at runtime.
Update CreateUpdateClientProfileRadosNamespace to
look up secrets by annotation
instead of using hardcoded names.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
kubernetes/code-generator is versioned alongside the core k8s.io modules, so codegen should logically stay in sync with the Kubernetes version in go.mod.
So we now resolve KUBE_CODEGEN_TAG from k8s.io/apimachinery with go list instead of maintaining a separate
pin that can silently fall behind dependency bumps. This removes manual version upkeep entirely. Bumping k8s modules in
go.mod automatically picks the matching code-generator release on the next make codegen.
Also rename CODE_GENERATOR_VERSION to KUBE_CODEGEN_TAG to align with the kube_codegen.sh tooling and the CODE_GENERATOR path used by codegen.sh.
Add the generated files for latest codegen run as well.
Signed-off-by: Malay Kumar Parida <mparida@redhat.com>
assignMons schedules each mon in its own goroutine and shares a
failedMonSchedule bool to record whether any of them failed. The
resultLock mutex already guards the shared c.mapping.Schedule map
write, but the three failedMonSchedule = true assignments were done
without holding the lock. When two or more mons fail to schedule at
the same time (for example when waitForMonitorScheduling errors, the
node choice is nil, or getNodeInfoFromNode fails) their goroutines
write the flag concurrently, which is a write-write data race.
Guard the failedMonSchedule writes with the existing resultLock, the
same lock the goroutines already use for the map update. The post-Wait
read is left as is since the WaitGroup orders it after every write.
Add a regression test that fails scheduling for multiple mons at once
and confirms assignMons returns an error; run it with -race to catch
the race.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
decodePeerToken logged the whole decoded PeerToken with "%+v" at debug
level. PeerToken embeds Key, the peer cluster's cephx auth secret, so
running the operator with ROOK_LOG_LEVEL=DEBUG wrote that credential to
the operator logs in plaintext (CWE-532).
Log only the non-sensitive fields (fsid, client id, mon host, namespace),
which keep the message useful for diagnostics, and drop Key. Add a
regression test that captures the debug output and asserts the key is
never present.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Port 1024 is the lowest non-privileged port on Linux (privileged ports
are 0-1023), so it should be honored as-is like any port >= 1024. The
dashboardInternalPort guard used `<=` instead of `<`, which pushed a
requested Port of 1024 into the "privileged" branch and reset the
container/targetPort back to the default (7000/8443). That left the
Service Port at 1024 but its targetPort at the default, a mismatch that
contradicts the constant's comment and the identical Port 1025 case.
Change the comparison to `<` and add a boundary case to
TestStartSecureDashboard asserting both Port and targetPort are 1024.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
currently there was a bug in the code where it didnt removed
the account from the ceph cluster during intial intialize
Signed-off-by: parth-gr <partharora1010@gmail.com>
The Placement field comments on RBDMirroringSpec and
FilesystemMirroringSpec both read "The affinity to place the rgw pods",
copied from GatewaySpec. Neither spec manages RGW pods: RBDMirroringSpec
configures the rbd mirror daemon and FilesystemMirroringSpec the
cephfs-mirror daemon.
Update the comments to name the correct pods, matching the wording
already used by the sibling Resources and PriorityClassName fields in
each struct, and regenerate Documentation/CRDs/specification.md so the
published CRD API reference matches.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
testPodSpecPlacement accepts a pref parameter but never asserts the
preferred anti-affinity count, so the PreferredDuringScheduling branch
of SetNodeAntiAffinityForPod went unverified. The assertion was dropped
in b91f4211c9 when the separate preferred flag was removed from
SetNodeAntiAffinityForPod, leaving pref unused.
Restore the missing assertion so the helper validates both the required
and preferred term counts. Drop the third empty-placement case, which
became an exact duplicate of the second once the two-boolean API was
collapsed and now adds no coverage.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
The capacity-retention case in TestCephStatus asserted
formatTime(now-1m) against formatTime(now-1m), comparing an
expression to itself. That assertion can never fail and never
looks at the value under test, so the branch in
toCustomResourceStatus that keeps the previous capacity when
newStatus.PgMap.TotalBytes is 0 was left unverified.
Seed a known LastUpdated on the current status alongside the
existing TotalBytes, then assert the aggregated status carries
that timestamp through. This checks that the prior capacity,
including its LastUpdated, is retained when total bytes drop to 0.
The sibling cases already assert Capacity.LastUpdated the same way.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
The compareJSON test helper unmarshaled the expected JSON into both
the expected and actual variables, so assert.Equal always compared the
expected value against itself. The actualJSON argument was never read,
which made the assertion a tautology that could never fail and silently
disabled validation of updateCsiClusterConfig's output.
Parse actualJSON for the actual value. Enabling the check surfaced a
stale expectation in the multus sub-test: updateNetNamespaceFilePath
clears rbd.netNamespaceFilePath (holder pods are no longer used with the
ceph-csi-operator), so the expected literal is updated to drop that
field and match the real output.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Test_cleanCSIDirs created both the rbd and cephfs CSI driver
directories but only ever checked that the rbd directory was
removed. The second os.Stat call re-checked rbdDriverDir instead
of cephFSDriverDir, so removal of the cephfs CSI directory by
cleanCSIDirs was never verified even though the test set it up
for exactly that purpose.
Point the second assertion at cephFSDriverDir so both directories
that cleanCSIDirs is expected to remove are actually asserted.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
The success log in removeSnapshotSchedule passed its arguments in the
wrong order for the format string "successfully removed snapshot
schedule %q for pool %q". The arguments were poolName followed by
snapScheduleResponse.Interval, so the schedule slot printed the pool
name and the pool slot printed the interval.
Swap the two arguments so the interval prints in the schedule slot and
the pool name in the pool slot. This matches the sibling log in
enableSnapshotSchedule and the order used to build the remove command.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Add annotations to CephFS and RBD provisioner secrets in
import-external-cluster.sh so that externally created secrets with
custom names can be discovered at runtime.
Update CreateUpdateClientProfileRadosNamespace and
generateProfileSubVolumeGroupSpec to look up secrets by annotation
instead of using hardcoded names.
closes: https://github.com/rook/rook/issues/16956
Signed-off-by: parth-gr <partharora1010@gmail.com>
CephFS reconciliation loop on Filesystem creation was swallowing errors.
The err variable which contained the error was replaced with the resulted error from the
kubernetes update status call on the CRD.
This commit fixes that and aggregate both errors in case there is both a kubernetes status update failure and a creation error.
Signed-off-by: Maxime Bertin <mbertin@luccasoftware.com>
Travis Nielsen
Chiman Jain
Parth Arora
Oded Viner
Joshua Hoblitt
subham rai
Anas Khan
dependabot[bot]
Malay Kumar Parida
Blaine Gardner
Michael Adam
subhamkrai
Madhu Rajanna
Artem Torubarov
Maxime Bertin