Added failureDomains and osdsPerDomain fields to the
ErasureCodedSpec, which maps to the crush-num-failure-domains and
crush-osds-per-failure-domain Ceph EC profile parameters. This enables
creating EC pools that distribute chunks across fewer, larger hosts
without needing one host per data chunk.
Signed-off-by: Prabhala Tara Aasrita <taraprabhala@Prabhalas-MacBook-Pro.local>
A state machine for osd destroy phase of osd-replacement flow.
The state machine is implemented in OSD health goroutine, where each
step is handled on a separate tick. State machine is responsible
for draining, destroying OSD, and reserving its CRUSH position,
downscalling its deployment, and adding "readyForSwap" annotation.
Destroy phase does not zap the device on purpose. On disk signature
is kept to avoid automatic reprovisioning of destroyed OSD and to
detect physical swap.
Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
contains osd prepare job edits to support osd replacement flow:
- removes destroyed OSD IDs from node status CM to stop cluster
controller from reprovisioning destroyed OSD
- adds provisioning logic for OSD replacement to match available
blank devices to destroyed OSD slots
Signed-off-by: Artem Torubarov <artem.torubarov@sap.com>
the .nvmeof pool was incorrectly tagged with the rbd
application and initialized as an rbd pool. add .nvmeof
to the built-in pool switch so it gets the nvmeof-meta
application tag instead, and skip rbd pool init.
Signed-off-by: Oded Viner <oviner@redhat.com>
Ceph reports bytes_quota in `ceph fs subvolumegroup info` as the JSON
string "infinite" when no quota is set, and as a number otherwise. The
subvolumeGroupInfo.BytesQuota field is typed int64, so json.Unmarshal
fails with "cannot unmarshal string into Go value of type int64" for any
subvolume group without a quota (the default).
That unmarshal error is returned from getCephFSSubVolumeGroupInfo and
swallowed in CreateCephFSSubVolumeGroup: exec.ExitStatus reports ok=false
for the wrapped error, so the non-ENOENT guard does not fire and the
dedicated resize path (which enforces --no-shrink) is skipped.
Add an UnmarshalJSON on subvolumeGroupInfo that accepts either a number
or the "infinite" sentinel, mapping "infinite" to 0 (unlimited) so the
existing CmpInt64 comparison keeps working. Add a table-driven test
covering the finite and infinite cases.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
The success log in removeSnapshotSchedule passed its arguments in the
wrong order for the format string "successfully removed snapshot
schedule %q for pool %q". The arguments were poolName followed by
snapScheduleResponse.Interval, so the schedule slot printed the pool
name and the pool slot printed the interval.
Swap the two arguments so the interval prints in the schedule slot and
the pool name in the pool slot. This matches the sibling log in
enableSnapshotSchedule and the order used to build the remove command.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Fix duplicate words, incorrect articles (a/an), it's/its, and other small
grammar mistakes in Go comments and user-facing messages across pkg/, cmd/,
and tests/.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The //nolint directives at these sites omit the linter name, so each
suppresses every linter on its line rather than the one check it needs.
That hides any unrelated errcheck/gosec/govet finding later introduced
on the same line. Name the specific linter for each:
- staticcheck for the two operator sites: SA4004 (the intentional
single-iteration loop in the OSD PVC host lookup) and SA1019 (the
deliberate read of the deprecated S3.Enabled field in the RGW
API-enable builder).
- errcheck for the rbd-mirror deferred token-file cleanup and the
test-framework logging helpers (WriteString / writeHeader).
No behavior change.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Several godoc comments led with a stale or incorrect identifier, left
over from renames, exported/unexported changes, copy-paste between
sibling declarations, or plain typos. As a result the documented name no
longer matched the function, method, type, or var it describes. Correct
each leading word to the name of the declaration it documents.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
The OSDPerfStats struct had no references anywhere in the repo (verified
with repo-wide grep and an exported-symbol audit); the code that consumed
this unmarshal target was removed long ago.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
Remove four dead functions from pkg/daemon/ceph/client; none had any
callers anywhere in the repo (verified with deadcode, staticcheck U1000,
and repo-wide grep, tests included):
- auth.go: AuthGetOrCreate
- command.go: ExecuteRBDCommandWithTimeout
- filesystem.go: MarkFilesystemAsDown
- status.go: IsClusterCleanError (IsClusterClean itself remains in use)
No imports become unused by these removals.
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
createREplicatedPoolForApp creates a base Crush rule (named after the
pool) on every reconcile, even if the pool already exists and uses a
suffixed rule ( eg pool_host_ssd). This causes orphaned rules to
reappear after every operator restart, immediately after
CleanupUnusedCrushRules deletes them.
This PR ensures that the base rule is created only if the pool does not
exist yet.
Signed-off-by: Santosh <sapillai@redhat.com>
this commit adds flag stripe_unit for the command
ceph osd erasure-code-profile set [flags]. This key
increase perfomance for ec pool. The default value
is 4kib/4096 bytes.
Signed-off-by: subhamkrai <srai@redhat.com>
During the initial cluster setup, CleanUnusedCrushRules() runs after MGR
startup but before any pools are created (if user does not create the
.mgr pool along with the cephCluster manifest). It deletes the default
replicated_pool since no pool references it.
This leaves the cluster in a state if Zero crush rules. When the MGR
tries to start the device_health_metris during this window, thne it
fails to find the crush rule and crashes.
Skip the cleanup when no pools exists yet, since there is nothing
meaninful to cleanup.
Signed-off-by: Santosh <sapillai@redhat.com>
Clean up stale CRUSH rules after the Ceph mgr starts so rules left behind by pool failure-domain or device-class changes do not accumulate indefinitely.
The cleanup is guarded by a package-level RWMutex. Pool create and update paths hold the read lock while creating and assigning CRUSH rules, while cluster-wide cleanup holds the write lock before listing pools and deleting unused rules. This keeps pool reconciles parallel with each other while preventing cleanup from deleting a rule that another reconcile has just created but not yet attached to a pool.
Keep direct pool-delete cleanup for the pool's current CRUSH rule, make the cluster-wide cleanup best-effort across all unused rules, and add an operator-level ROOK_DELETE_UNUSED_CRUSH_RULES setting for clusters that need to leave unused custom rules in place.
Document the operator and Helm settings, regenerate the Helm chart docs, and add a pending release note for the default cleanup behavior.
Signed-off-by: Asish Kumar <officialasishkumar@gmail.com>
Ceph incorrectly raises the MON_NETSPLIT health warning in stretch cluster configurations. Added a call to MuteHealthWarning() in postMonStartupActions() which runs `ceph health mute MON_NETSPLIT --sticky` to suppress the warning during reconcile. All errors are logged as warnings and the reconcile proceeds.
Signed-off-by: Prabhala Tara Aasrita <taraasrita@ibm.com>
Remove helper methods for querying and configuring
various osd settings that are no longer in use anywhere
in Rook
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
deleting an erasure coded CephBlockPool left the ec profile
orphaned. recreating with different settings failed because
the old profile could not be overridden.
- delete ec profile during CephBlockPool deletion
- disable --format json on ec profile set command
- fix ec profile name mismatch in CephObjectStore deletion
Signed-off-by: Oded Viner <oviner@redhat.com>
For Rook v1.19,removing support for Ceph Reef. It is at end of life.
Users on Reef can continue to use Rook v1.18.x or older.
The latest two Ceph versions Squid and Tentacle are supported.
Signed-off-by: subhamkrai <srai@redhat.com>
If the ceph status is "active+lean+snaptrimp" or
"ative+clean+snatrip_wait" , the the PDB controller should treat this
ceph staus as healthy and don't block the node drain
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
currently, in RNS crd while updating status we don't
have retryOnConflicts, similar to other CRD due to
this RNS status was not updated succesfully and to
fix this we have restart the rook operator pod.
Signed-off-by: subhamkrai <srai@redhat.com>
The target size ratio is currently always being set to 0, when
neither the legacy targetSizeRatio nor the parameter for
target_size_ratio are set. In this case, we should not set it to 0,
but rather should skip the command to set the target size ratio.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
blocklisting IP can cause problems when the cleanup
job pod runs on the same host or gets the same ip
as that of the rbd client ip that is present in
the rbd status/watchers. instead of blocklisting
the whole ip of the node or the client blocklist
only the unique ip:nonce to allow the node to not
have any effect because of the cleanup job.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Enabling crush updates is false by default, and can be enabled
if set to true. The setting needs to be nullable in case
we need to differentiate between the value being set
to false or just not set, to know if the default should be
different. This is necessary for rook consumers who are
refererencing the pool struct and want to change the default,
rather than using rook's default value of false.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
when the osd is created there is no need for
osd bootstrap keyring, so remove the keyring
and keyring file,
New keyring will be created if
new osd need to be provisoned
Signed-off-by: parth-gr <partharora1010@gmail.com>
Implement CephX key rotation for Rook's client.admin user.
Admin user rotation is risky, so this has been tested extensively both
in unit tests as well as by manually injecting failures during runtime.
In testing, all failures were able to be recovered by the recovery
routine.
A mutex is also added to help ensure that two simultaneous admin key
rotation processes cannot be running simultaneously for any given
namespace. The mutex is tested in unit tests, and it was verified during
runtime via manual testing.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Travis Nielsen
Prabhala Tara Aasrita
Artem Torubarov
Oded Viner
Joshua Hoblitt
Anas Khan
Joshua Hoblitt
Blaine Gardner
pengqima
daixihegu
subhamkrai
Sunnatillo
Santosh Pillai
immanuwell
Asish Kumar
Denis Egorenko
Prabhala Tara Aasrita
huyejia
parth-gr
Madhu Rajanna