Commit Graph
551 Commits
Author SHA1 Message Date
Travis Nielsen d1c4667c8b Merge pull request #13225 from ushitora-anqou/use-regex-match-for-pdb-reset
core: add pgHealthyRegex to DisruptionManagementSpec
2023-11-27 14:23:25 -07:00
Ryotaro Banno 0ad88e9d68 core: add pgHealthyRegex to DisruptionManagementSpec
This patch adds `pgHealthyRegex` field to DisruptionManagementSpec.
`pgHealthyRegex` is a regular expression that is used to determine which
PG states should be considered healthy. The default value of
`pgHealthyRegex` is:

        ^(active\+clean|active\+clean\+scrubbing|active\+clean\+scrubbing\+deep)$

which is effectively the same as before.

Signed-off-by: Ryotaro Banno <ryotaro.banno@gmail.com>
2023-11-22 00:19:04 +00:00
gauravsitlani 3d0049c547 core: operator to skip reconcile of mgr, rgw, mds and rbd-mirror daemons in debug
During certain maintenance tasks the admin will own running
operations on the ceph mgr, rgw, mds and rbd-mirror daemons
and the operator should not interfere with those operations.

Co-authored-by: gauravsitlani <gaurav.sitlani@live.com>
Signed-off-by: subhamkrai <srai@redhat.com>
2023-11-21 20:14:16 +05:30
travisn 03d077aa6b core: remove support for ceph pacific
Pacific is end of life and no longer necessary to
support in Rook with v1.13.

Signed-off-by: travisn <tnielsen@redhat.com>
2023-11-14 17:07:03 -07:00
sp98 ffd69291f1 osd: use bash script if restart interval is > 0
If `FlappingRestartIntervalHours` for OSD is > 0 in the cephCluster CR,
then start OSD inside a bash script.

Signed-off-by: sp98 <sapillai@redhat.com>
2023-10-11 11:38:26 +05:30
Travis Nielsen cb9ffacce5 Merge pull request #12909 from testwill/pkg-import
core: import packages only once
2023-09-21 15:30:04 -06:00
guoguangwu 235ac293ff core: import packages only once
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-16 13:40:17 +08:00
sp98 4eb9f62205 osd: make osd pod to sleep when osds are flapping
When OSDs flap, ceph stops the OSD daemon if its marked down greater than
5 times in 600 seconds. But OSD pod restarts and marks the OSD `up` again.
This causes the PGs mapped to these OSDs to peer. While the PGs are peering,
IO to these PGs are blocked.

So we need to ensure that if ceph is marking OSD `down` due to flapping, OSD pod
should not restart to mark the OSDs `up` again.

This PR adds a sleep to the OSD pod if the container returned with a 0 exit code
Default behavior is to sleep for 6 hrs. But user can configure it from the
ceph cluster spec.

Signed-off-by: sp98 <sapillai@redhat.com>
2023-09-15 22:59:46 +05:30
Blaine Gardner 17f0072d9d Merge pull request #12778 from BlaineEXE/multus-allow-cidr-spec
multus: allow using NADs without inspectable CIDRs
2023-09-07 13:42:41 -06:00
Blaine Gardner 3c43268d0a multus: detect network CIDRs via canary
Change how Rook detects network CIDRs for Multus networks. The IPAM
configuration is only defined as an arbitrary string JSON blob with a
"type" field and nothing more. Rook's detection of CIDRs for whereabouts
had already grown out of date since the initial implementation.
Additionally, Rook did not support DHCP IPAM, which is a reasonable
choice for users. And more, Rook did not support CNI plugin chaining,
which further complicates NADs. Based on the CNI spec, network chaning
can result in any changes to network CIDRs from the first-given plugin.

All these problems make it more and more difficult for Rook to support
Multus by inspecting the NAD itself to predict network CIDRs. Instead,
it is better for Rook to treat the CNI process as a black box. To
preserve legacy functionality of auto-detecting networks and to make
that as robust as possible, change to a canary-style architecture like
that used for Ceph mons, from which Rook will detect the network CIDRs
if possible.

Also allow users to specify overrides for CIDR ranges. This allows Rook
to still support esoteric and unexpected NAD or network configurations
where a CIDR range is not detectable or where the range detected would
be incomplete. Because it may be impossible for Rook to understand the
network CIDRs wholistically while residing only on a portion of the
network, this feature should have been present from Multus's inception.

Improving CIDR auto-detection and allowing users to specify overrides
for auto-detected CIDRs rounds out Rook's Multus support for CephCluster
(core/RADOS) installations. No further architectural changes should be
needed for CephClusters as regards application of public/cluster network
CIDRs for Multus networks.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2023-09-07 10:12:55 -06:00
Travis Nielsen 0ae3800929 Merge pull request #12770 from sp98/osd-migration-part2
osd: replace existing OSDs to use new store
2023-09-05 08:57:20 -06:00
sp98 11b8d10a5a osd: replace existing OSDs to use new store
This follow up the #12507
- fixes replacing of encrypted OSDs.
- Updates OSD status at the end of reconcile

Signed-off-by: sp98 <sapillai@redhat.com>
2023-09-01 21:07:38 +05:30
subhamkrai 29d2b6a071 core: restart ceph daemons when network updated
We need to restart all the ceph daemons whenever
cephCluster network settings are modified like
requiremsgr2, encryption and compression. This
required for Ceph to consider the new settings
it require new ceph daemons all over.

Signed-off-by: subhamkrai <srai@redhat.com>
2023-09-01 11:30:48 +05:30
sp98 886bb357ef osd: replace osd to use new backend store
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk

Signed-off-by: sp98 <sapillai@redhat.com>
2023-08-17 21:40:55 +05:30
Sheetal Pamecha d86fa2abbc osd: use cp -a command for copying init-containers
remove copy-binaries cmd and related code

Signed-off-by: Sheetal Pamecha <spamecha@redhat.com>
2023-07-31 09:31:38 +05:30
sp98 c13d34f8d3 osd: configure backend store
Configure osd backend store from the cephCluster spec

Signed-off-by: sp98 <sapillai@redhat.com>
2023-07-24 09:16:30 +05:30
YZ775 ed9da20d68 operator: add ceph image version label to PVC
This PR makes Rook operator to add a label on PVC that contains an image version of Ceph when creating an OSD.

Signed-off-by: YZ775 <yuzuki-mimura@cybozu.co.jp>
2023-06-19 08:19:46 +09:00
avanthakkar 541d091f9c core: use ROOK_CEPH_MON_HOST from config store in OSD pods too
Signed-off-by: avanthakkar <avanjohn@gmail.com>

Volume "ceph-daemons-sock-dir" is coming empty in case if dataDirHostPath,
which is the case for osd onPVC. Fix the volume creation by using the
ceph cluster spec dataDirHostPath, which allows to run socket commands
on osd containers.
2023-06-01 21:09:58 +05:30
Satoru Takeuchi 13e4464fed osd: supoprt expand lvm osd on pvc
The expansion of OSD on PVC is only supported in raw mode OSD.
It's nice to support lvm mode osd too.

Closes: https://github.com/rook/rook/issues/10835

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2023-05-16 00:23:09 +00:00
travisn db04b127b0 core: remove obsolete journal size config value
The journal size was only applicable to the filestore OSD
format which has not been supported by rook since v1.2.
Remove the remaining obsolete setting from the examples and
code.

Signed-off-by: travisn <tnielsen@redhat.com>
2023-04-24 15:54:53 -06:00
sp98 9e8e85d68b core: use clusterID for service nslookup
use ClusterID of the k8s cluster while doing nslookup of the
exported mon or osd service IP

Signed-off-by: sp98 <sapillai@redhat.com>
2023-04-11 15:24:08 +05:30
travisn cd94ed7ddc core: set key rotation default in code instead of crds
The defaults in CRDs cause an update and trigger a new
reconcile if the value is not set. For the key rotation
setting, the default was only being updated when the kms
feature was enabled. This update was causing the reconcile
to fail in the kms tests. With the default now in code,
the default can remain empty and no cr update will trigger
a new reconcile unexpectedly.

Signed-off-by: travisn <tnielsen@redhat.com>
2023-03-21 11:03:56 -06:00
Travis Nielsen fd2f5c8abe Merge pull request #11904 from parth-gr/golang-ci-lint-update
ci: update golangci-lint version as it fails for some k8s version in …
2023-03-16 10:52:13 -06:00
parth-gr 8e317ee074 ci: update golangci-lint version as it fails for some k8s version in 1.10
Closes: https://github.com/rook/rook/issues/11896

Signed-off-by: parth-gr <paarora@redhat.com>
2023-03-16 20:59:22 +05:30
Travis Nielsen c0fdcd3f40 Merge pull request #11823 from parth-gr/topology
osd: validate and remove duplicate topology
2023-03-15 12:39:17 -06:00
parth-gr 89764f0371 osd: remove deprecated k8s topology label
Signed-off-by: parth-gr <paarora@redhat.com>
2023-03-14 17:28:19 +05:30
parth-gr c22f5f36cb osd: validate and remove duplicate topology
remove the duplicate topology added to the node,
and the priority to keep duplicate would be give to
the one who is lowest in the hierarchy

Closes: https://github.com/rook/rook/issues/11800

Signed-off-by: parth-gr <paarora@redhat.com>
2023-03-09 21:34:38 +05:30
Rakshith R f39a32fcdb osd: add capability to reconcile key rotation cron jobs
This commits adds code to reconcile key rotation cron jobs.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-03-08 12:07:26 +05:30
Rakshith R ef38aea2a4 osd: add util function to create key rotation cron job
This commit adds util function to create key rotation cron job
such as getKeyRotationContainer, getKeyRotationCronPodTemplateSpec
and makeKeyRotationCronJob.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-03-08 12:07:26 +05:30
Rakshith R 71e011f731 osd: add rotate-key functionality to rook's key-management cmd
This commit adds functionality to be able to rotate
key encryption key of encrypted PVC backed OSDs.
Necessary changes such as adding update functionality
to kms and rbac changes are made as well.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-03-08 12:07:26 +05:30
sp98 12bbc9b1a8 osd: add container ports only if mcs is enabled
Add container ports for OSD container only when multi cluster service is
enabled in the cluster spec.

Signed-off-by: sp98 <sapillai@redhat.com>
2023-03-01 09:07:08 +05:30
Travis Nielsen 6bcf9004a0 Merge pull request #11567 from cybozu-go/fix-10860
osd: handle device name change and device removel correctly
2023-02-27 11:54:30 -07:00
sp98 7292ac5927 core: export mon and OSD services
Signed-off-by: sp98 <sapillai@redhat.com>
2023-02-27 21:36:59 +05:30
Travis Nielsen d655c86b2f osd: create a service for each osd
The osd needs to bind to an ip that will allow both internal
and external access for rbd mirroring scenarios.
THe operator will create a service for each osd and
instruction the osd to bind to the pod ip while
publishing the clusterIP of the service as the
public IP.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 14e7225e0a18aff85ede1d476fcaddbbfbcbb732)
2023-02-27 21:05:48 +05:30
Satoru Takeuchi 3a44a394b5 osd: handle device name change and device removel correctly
If a kernel device name change happens and a block device file
in the OSD directory becomes dangling link, this OSD fails
to start continuously. This problem can be resolved by confirming
the validity of the device file and recreating it if necessary.

The original problem happened in host-based clusters. A similar
problem exists in PVC-based clusters. In this case, if a PV,
corresponds to an existing OSD, and points to a missing block
device file, the OSD pod fails to consume this PV. Although this
behavior is undesirable, I don't think this problem should be
handled in Rook. If doing so, we must re-create the existing PV.
Rook shouldn't do such work.

Closes: https://github.com/rook/rook/issues/10860

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2023-02-24 05:29:14 +00:00
parth-grandBlaine Gardner 69ae9569cd build: update k8s version to 1.26.1
Update the Kubernetes API version used to 1.26.1, and start testing
against Kubernetes version 1.26.1 in CI.

Co-authored-by: parth-gr <paarora@redhat.com>
Co-authored-by: Blaine Gardner <blaine.gardner@redhat.com>

Signed-off-by: parth-gr <paarora@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2023-02-23 20:18:47 -07:00
parth-gr a84daf9bf0 core: change io/ioutil package to use io and os package
few functions got change as they were deprecated
for ex: ioutil.Readfile change to os.Readfile
ioutil.TempFile change to os.CreateTemp
And fixed golang-ci-lint-issues

Signed-off-by: parth-gr <paarora@redhat.com>
2023-02-17 20:38:29 +05:30
Rakshith R 8b9ded1f77 csi: add option to enable read affinity for rbd volumes
This commit adds code to supply variables required
to enable read affinity for rbd volumes.
It leverages `read_from_replica=localize,crush_locations=key:value`
krbd map option from ceph to serve reads from the most local OSD.
refer:
https://docs.ceph.com/en/latest/man/8/rbd/#kernel-rbd-krbd-options

Signed-off-by: Rakshith R <rar@redhat.com>
2023-02-16 19:09:39 +05:30
Travis Nielsen 06eb431525 Merge pull request #11674 from travisn/osd-version-label
osd: Ensure rook version label is not set on pod
2023-02-15 12:21:09 -07:00
Avan Thakkar 460900756c core: add service monitor for ceph-exporter service
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2023-02-15 15:44:56 +05:30
Travis Nielsen 1ba93b4047 osd: ensure rook version label is not set on pod
The rook-version tag must not be set on the pod spec labels
since it will result in the ceph daemons being restarted every
time there is a rook version update, even if the ceph version
or pod spec was not otherwise updated.

The rook-version tag was being added to the OSD pod spec
due to a shared pointer to the labels. The code intended
to only add the version label to the deployment labels,
but since the pod labels shared the map variable, the
pod unintentionally also had the version label added.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2023-02-14 16:10:53 -07:00
Avan Thakkar 2f8ee60374 core: introduce ceph-exporter
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2023-02-06 02:47:09 -05:00
Yuma Ogami 7599f1bc6b osd: add expand-bluefs init container for host-based cluster
OSD is not resized in host-based clusters even though expanding
the backing block device. It's due to the lack of bluefs-bdev-expand
init container.

Closes: https://github.com/rook/rook/issues/10930
Signed-off-by: Yuma Ogami <yuma-ogami@cybozu.co.jp>
2023-01-03 05:03:03 +00:00
Yuma Ogami c6c264df84 osd: add unit test for chown-container-data-dir init container
Signed-off-by: Yuma Ogami <yuma-ogami@cybozu.co.jp>
2022-12-22 07:43:15 +00:00
Travis Nielsen 403335177a core: read mon secret from file instead of env var
Environment variables are not recommended for secrets in pods since
they can be easily leaked if the environemnt variables are logged.
By mounting the mon secret as a file, the mgr and osd prepare pods
can read the mon secret from a file for better security.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-12-14 16:24:44 -07:00
Travis Nielsen 4a70d3489d osd: replace dot with dash in pvc names
The volume names of a pod must follow the standard naming convention
which does not allow periods in the name. Since the device set name
is used as the pvc name and then the volume name to attach to the osd
prepare job, we must sanitize the name for the invalid character
and replace it.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-12-09 12:43:17 -07:00
Satoru Takeuchi 0e10d97431 Merge pull request #11331 from travisn/remove-env-vars
core: Remove unnecessary env vars from pod specs
2022-12-06 08:28:06 +09:00
Rakshith R bde286ef49 osd: re-open encrypted disk during osd-prepare-job if closed
This commit implements this corner case during osd-prepare job.
```
The encrypted block is not opened, this is an extreme corner case
The OSD deployment has been removed manually AND the node rebooted
So we need to re-open the block to re-hydrate the OSDInfo.

Handling this case would mean, writing the encryption key on a
temporary file, then call luksOpen to open the encrypted block and
then call ceph-volume to list against the opened encrypted block.
We don't implement this, yet and return an error.
```
When underlying PVC for osd are CSI provisioned, the encrypted device
is closed when PVC is unmounted due to osd pod being deleted.
Therefore, this may occur more frequently and needs to be handled.
This commit implements the fix for the same.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-11-30 18:54:23 +05:30
Travis Nielsen 33e824a323 core: remove unnecessary env vars from pod specs
The fsid, username, and user secrets were from legacy and no
longer needed on the osd daemon or mon pods. This removes the
unnecessary env vars from the pod specs.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-11-18 16:39:22 -07:00
Shinya Hayashi 05875a3f4f osd: support loop devices for test clusters
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.

This feature is intended to be used for testing purposes only.

Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
2022-11-09 06:45:41 +00:00