OSDs are created with `ceph-volume lvm batch` if `metadataDevice`
field is specified in CephCluster CR. Since this command does not
support kernel names (e.g. `/dev/dm-X`) for multipath devices,
we should use real names (e.g. `/dev/mapper/mpathX`) instead.
NOTE:
I intentionally keep raw mode OSD creation code as is
because `ceph-volume raw prepare` supports kernel names
for any devices.
Closes: https://github.com/rook/rook/issues/17057
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
when the osd is created there is no need for
osd bootstrap keyring, so remove the keyring
and keyring file,
New keyring will be created if
new osd need to be provisoned
Signed-off-by: parth-gr <partharora1010@gmail.com>
During the OSD preparation step, if the encrypted OSD disk belongs to a
different cluster, then clean the disk if `wipeDeviceFromOtherClusters`
is set to true.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Allow force creation of OSD on a device previously configured in a different ceph cluster.
The OSD prepare pod job will zap the OSD disk using ceph-volume lvm zap
if that disk has metadata from a different ceph cluster.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Currently, when rook provisions OSDs(in the OSD prepare job), rook effectively run a
c-v command such as the following.
```console
ceph-volume lvm batch --prepare <deviceA> <deviceB> <deviceC> --db-devices <metadataDevice>
```
but c-v lvm batch only supports disk and lvm, instead of disk partitions.
We can resort to `ceph-volume lvm prepare` to implement it.
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
The validation logic of checking the number of devices is
wrong when `metadataDevice` is set and `osdsPerDevice` > 1.
`len(cvReports)` is the expected number of OSDs and is
the number of specified data devices multiplied
by `osdsPerDevice`.
Closes: https://github.com/rook/rook/issues/13637
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This reverts commit 093125904b.
Once we introduced encrypted OSD on partition in #12924.
However, even though `encryptedDevice` field is `true`,
an non-encrypted OSD is created. To prevent the creation
of misconfigured OSD, it's better to revert #12924 for now.
Initially, I intended to correct this problem. However, I found
it will take long time because we need to change the KMS-related
code, init containers of both OSD pod and OSD prepare pod.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk
Signed-off-by: sp98 <sapillai@redhat.com>
few functions got change as they were deprecated
for ex: ioutil.Readfile change to os.Readfile
ioutil.TempFile change to os.CreateTemp
And fixed golang-ci-lint-issues
Signed-off-by: parth-gr <paarora@redhat.com>
It removes the condition that select the devices name prefixed with /dev/mapper as it is causing the ceph-volume command to fail.
Closes: #11301
Signed-off-by: ausias-armesto <ausiasarmesto@gmail.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a specific disk is configured to have more than one OSD per device,
or if it has a metadata device specified, exclude it from raw mode on an
individual basis.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The OSD prepare job was succeeding even if the OSDs failed to be listed
by ceph-volume. These errors should be returned and fail the job, resulting
in a retry to retrieve the OSDs. Otherwise, the failure is masked and
the admin needs to dig through logs to find why the OSDs were not configured
on that node.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We can eliminate the OSD creation code for Ceph < v15.2.0 because Nautilus(v14.2.z)
is no longer supported. In other words, we can now expect that all newly-created
OSDs on PVC are raw mode.
Please note that the existing lvm mode OSDs on PVCs still be supported after this change.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:
* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited
As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.
A second new controller for the operator's general config has been
created, it manages:
* the logging level
* the ceph CLI command timeout
* the discovery daemon
The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.
Signed-off-by: Sébastien Han <seb@redhat.com>
Both `ExecuteCommandWithOutputFileTimeout()` and
`ExecuteCommandWithOutputFile()` generate unnecessary system calls by
creating/reading/removing files where the stream output of the command
can simply be used. So sticking with `ExecuteCommandWithOutput()` and
`ExecuteCommandWithCombinedOutput()` for reading outputs is sufficient.
Closes: https://github.com/rook/rook/issues/8343
Signed-off-by: Sébastien Han <seb@redhat.com>
Even if we can use raw mode, do NOT use raw mode on disks. Ceph bluestore disks can
sometimes appear as though they have "phantom" Atari (AHDI) partitions created on them
when they don't in reality. This is due to a series of bugs in the Linux kernel when it
is built with Atari support enabled. This behavior does not appear for raw mode OSDs on
partitions, and we need the raw mode to create partition-based OSDs. We cannot merely
skip creating OSDs on "phantom" partitions due to a bug in `ceph-volume raw inventory`
which reports only the phantom partitions (and malformed OSD info) when they exist and
ignores the original (correct) OSDs created on the raw disk.
Resolves https://github.com/rook/rook/issues/7940
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Currently it is not possible to set resource limits based on device classes
for different OSDs. Now adding an ability to use predefined keys for main
cluster spec Resource section to reflect resource limits for different OSDs
with different device classes.
Closes: https://github.com/rook/rook/issues/8007
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
The unit test TestConfigureCVDevices was failing intermittently
due to comparing the first element of the slice of two elements.
The slice ordering is not guaranteed, which caused a failure
intermittently. The test only compares the critical values in the test
and no need to check elements of the slice.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The deviceClass was only being set for LVM mode OSDs and was
missed for the raw mode for non-PVC. Now the deviceClass will be set
for all expected scenarios.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This mops up the last of the multiple-imports within ceph-related
packages, in pkg/daemon/ceph/agent/flexvolume/manager/ceph and
pkg/daemon/ceph/osd.
Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
When c-v runs after the end of the prepare sequence, it will look for
lvm and raw-based OSDs. The raw mode detects LVM OSDs so we must trim
the list of OSD, only to report the LVM ones if their ID exists.
Closes: https://github.com/rook/rook/issues/7359
Signed-off-by: Sébastien Han <seb@redhat.com>
For simple OSD scenario that don't require the following:
* an encrypted osd
* an osd with a backing device
* multiple osd on the same block device
The OSD will be prepared using the ceph-volume raw mode which does
not use LVM on non-pvc case.
Closes: https://github.com/rook/rook/issues/4768
Signed-off-by: Sébastien Han <seb@redhat.com>
The code was assuming that devices were passed by the user as
"/dev/sda", this is bad! We all know people should be using paths like
/dev/disk/by-id so we must support them.
Closes: https://github.com/rook/rook/issues/6685
Signed-off-by: Sébastien Han <seb@redhat.com>
this commit will enable one more linter ineffassign
in golangci-lint.
This linter throws an error when variable is assigned and never used.
`golangci-lint run --disable-all -E ineffassign` is used detects ineffassign
errors only.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
callCephVolume() was using our exec interface with CombinedOutput which
means return both stdout and stderr. This is fine when preparing OSDs
since we read back the stdout and also want any errors if any (stderr).
However when listing OSD we only want the output since ceph-volume tends
to mix stdout and stderr in the output and does not always silence the
warning types/harmless errors. If the listing has a problem we report it
by printing the ceph-volume log file.
Signed-off-by: Sébastien Han <seb@redhat.com>
Satoru Takeuchi
Jongwoo Han
Santosh Pillai
parth-gr
Joshua Hoblitt
Liang Zheng
Travis Nielsen
guoguangwu
parth-gr
ausias-armesto
Eng Zer Jun
Josh Soref
Blaine Gardner
subhamkrai
Sébastien Han
Jonas Zeiger
Denis Egorenko
Lars Lehtonen
shenjiatong
Daichi Sakaue
subhamkrai