Commit Graph
4320 Commits
Author SHA1 Message Date
Travis Nielsen 22d4139b74 core: add support for ceph squid
With the release of the first squid RC, we add squid
to the supported versions and add tests to run
Rook against the squid release.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-07-15 10:04:32 -06:00
Travis Nielsen 689b39de4f Merge pull request #14057 from travisn/deviceclass-log
pool: Return error if device class update fails
2024-07-12 08:50:35 -06:00
Travis Nielsen da73421d7e Merge pull request #14396 from parth-gr/csi-logrotate-file
csi: add log rotation for csi cephfs nfs pod containers
2024-07-12 08:41:45 -06:00
parth-gr 4ca8b12e12 csi: add log rotation for csi cephfs nfs pod containers
Use the same logrotate flow of rbd, as merged in #14305

1) Make the csi cephfs and nfs container logs persisted in a file

2) Use the cephcluster api specs to configure the log rotate

3) Add log rotation to rotate the log file and
Add a sidecar log collector container

And some fixes on the implementation:

1) Add a operator namespace in the log file path

2) Only add volume to the container if logrotate is enabled

Closes: https://github.com/rook/rook/issues/12809, https://github.com/rook/rook/issues/14429

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-07-12 12:50:35 +05:30
Blaine Gardner 67772a296b Merge pull request #14451 from BlaineEXE/multus-reset-test-line
multus: reset validation tool debounce time to 30
2024-07-11 14:53:12 -06:00
Blaine Gardner 5884f1c900 multus: reset validation tool debounce time to 30
Reset the Multus validation tool debounce time to its intended 30 second
value. It was changed to 5 for testing, and the change was accidentally
committed.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2024-07-11 11:27:33 -06:00
Blaine Gardner 33f5407dd4 multus: add host checking to validation tool
In order to help users check that they have implemented the newly-added
Multus host configuration prerequisites, add a check to the validation
tool to verify connectivity.

Because users who are already running clusters with Multus enabled, add
a flag that allows users to only check for host configuration
prerequisites. This mode will not start the large number of clients that
would normally be started because those clients could disrupt a running
Rook cluster negatively.

Host checking pods require host network access. Many Kubernetes
distributions have pod security features enabled. In order to allow
non-Vanilla distros to run this tool, allow specifying a service account
that pods will run as, which can be configured by the admin to allow
test pods.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2024-07-10 13:41:38 -06:00
Travis Nielsen 0bea31ef7f pool: return error if device class update fails
Updating the device class swallowed any error if updated
for the pool. The error was not even logged, so we couldn't
troubleshoot why the new crush rule was not applied.
Log the error for troubleshooting and also fail the pool
reconcile since the desired configuration was not applied.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-07-10 12:59:43 -06:00
Travis Nielsen 28addd1301 Merge pull request #14216 from thotz/update-cosi-images
object: update cosi images
2024-07-10 12:47:07 -06:00
Travis Nielsen b92bbba671 pool: skip updating crush rules for stretch clusters
Pools in stretch clusters must all specify the same
CRUSH rule. No pools can use a different rule. When there
is a change in the device class, we do not even expect to update
the crush rules in a stretch cluster. Different device classes
are not supported in stretch clusters, and it's expected to be
a homogenous environment. Therefore, skip all crush rule updates
in stretch clusters.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-07-10 11:08:31 -06:00
Travis Nielsen b8bcd71929 Merge pull request #14421 from subhamkrai/update-go-mods
build: update controller runtime to v0.18.4
2024-07-08 10:10:10 -06:00
Travis Nielsen 1fe1a2bc23 Merge pull request #14406 from parth-gr/logrotate-csi-odf
csi: add securityContext to csi containers
2024-07-08 10:08:40 -06:00
parth-gr 2ca0d1390b csi: add securityContext to csi containers
With the recent enhancements csi containers for using
logrotate, they need to run with securityContext to privileged,
on platform like openshift.
Used the ROOK_HOSTPATH_REQUIRES_PRIVILEGED flag
wich is set with operator deployment to see
if the securityCOntext is needed or not

Closes: https://github.com/rook/rook/issues/14400

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-07-08 18:47:26 +05:30
subhamkrai d429ed8be4 build: update controller runtime to v0.18.4
this commit update cntrl runtime to v0.18.4 and other related deps/

Signed-off-by: subhamkrai <srai@redhat.com>
2024-07-05 09:05:12 +05:30
subhamkrai d116429f75 build: upgrade to go 1.22
this commit upgrade to go 1.22 in go mod file and ci related files.
Also remove go 1.21 support

Signed-off-by: subhamkrai <srai@redhat.com>
2024-07-03 17:01:33 +05:30
Travis Nielsen 8e25300ad5 Merge pull request #14199 from Nordix/update-cephfs-clusterroles
csi: add cephfs encryption support
2024-07-01 15:27:38 -06:00
Blaine Gardner e51b9e0abf Merge pull request #14391 from subhamkrai/update-go-dep
build: update go-retryablehttp from 0.7.6 to 0.7.7
2024-06-27 15:03:31 -06:00
Blaine Gardner d7eddfe98d Merge pull request #14305 from parth-gr/csi-logrotate
csi: add log rotation for csi rbd pod containers
2024-06-27 14:37:38 -06:00
subhamkrai 9fbf3d3aac build: update go-retryablehttp from 0.7.6 to 0.7.7
this commit updates go-retryablehttp from 0.7.6 to 0.7.7
and without this govulcheck ci is failing.

Signed-off-by: subhamkrai <srai@redhat.com>
2024-06-27 14:44:59 +05:30
parth-gr d3742119b9 csi: add log rotation for csi rbd pod containers
1) Make the csi rbd container logs persisted in a file
   (csi plugin, csi provisioner, csi addons sidecar)

2) Use the cephcluster api specs to configure the log rotate

3) Add log rotation to rotate the log file and
   Add a sidecar log collector container

part-of: https://github.com/rook/rook/issues/12809

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-06-27 14:20:18 +05:30
Blaine Gardner 7a53913049 Merge pull request #14294 from subhamkrai/skip-fence-when-csi-disabled
csi: skip creating networkFence when csi disabled
2024-06-25 09:36:02 -06:00
Blaine Gardner f2304bf8b8 osd: fix activate failure when block device moves
Block devices can move between reboots. In corner cases, an OSDs block
device might move to a lower-indexed device while the previous device
does not exist. For example, an OSD on /dev/sde might move to /dev/sdd
on reboot if the original /dev/sdd died. There would be no /dev/sde
after that.

Users report that NVMe drives commonly change names, even when there are
no disk failures.

For these cases, ensure the activate script properly handles cases where
the previous disk is not present on the node and where the OSD is still
available on a different disk.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2024-06-24 11:38:24 -06:00
subham rai 3a72a2a920 Merge pull request #14367 from black-dragon74/use-orig-enc-key
osd: Use old passphrase to kill the LUKS slot
2024-06-24 11:04:00 +05:30
Niraj Yadav b8b7af3495 osd: use old passphrase to kill the luks slot
While adding a new encryption key to slot 1
if there exists a key in slot 1 which is not
equal to the one we want to update it with,
we kill the slot and then add the new key to it.

While killing the slot, the existing code uses
the new key, which is not valid in such cases.

This patch modifies the code to use the key in
slot 0 (the one that we know works) to kill the slot.

Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2024-06-21 19:15:01 +05:30
subhamkrai e8aeb0d263 osd: revert limit storageClassDeviceSets to 63 chars
This reverts commit 6f5cc9b01d.

Signed-off-by: subhamkrai <srai@redhat.com>
2024-06-21 11:24:25 +05:30
subhamkrai c7d07562c4 core: revert increase storage deviceset name limit to 50
This reverts commit 1dcc99ec1d.

Signed-off-by: subhamkrai <srai@redhat.com>
2024-06-21 11:23:55 +05:30
subhamkrai 1dcc99ec1d core: increase storage deviceset name limit to 50
initially, we have the limit of 40 char set on
storageclassdeviceset.Name but seems like that is not enough and
upgrade clusters can face issue with this. So, increasing the limit to
50.

Signed-off-by: subhamkrai <srai@redhat.com>
2024-06-20 10:51:14 +05:30
Blaine Gardner 4b51c2c3bd Merge pull request #14322 from ideepika/wip-disablecrush-updates
enableCrushUpdates option: rook operator would not be able to update crush rules by default if not set
2024-06-17 12:20:26 -06:00
Travis Nielsen eed1033485 monitoring: set honor labels on the service monitor
The endpoint property honorLabels: true should be set so that
different rgw instances can show separate prometheus
values instead of being aggregated.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-06-14 15:38:34 -06:00
Travis Nielsen c5fd5cd0b5 core: update go modules for snyk security check
The snyk security check started failing with a warning
about the package being outdated:
github.com/libopenstorage/secrets/azure which depends on
github.com/Azure/azure-sdk-for-go/sdk/azidentity.
These and related packages are now updated.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-06-12 16:45:07 -06:00
Travis Nielsen 720c23abcf Merge pull request #14325 from ideepika/wip-solve-device-class-bug
pool: get the exact deviceClass name instead of crushroot+deviceClass
2024-06-12 16:13:41 -06:00
Deepika Upadhyay 41116be919 pool: get the exact deviceClass name instead of crushroot+deviceClass
this lead to creation of 2 crush rules because of difference in
deviceClass names:
eg:
creating a new crush rule for changed deviceClass
("default~hdd"-->"hdd") on crush rule "test-crush-bug-az-ab-4"

Signed-off-by: Deepika Upadhyay <deepikaupadhyay01@gmail.com>
2024-06-13 01:03:20 +05:30
Deepika Upadhyay d71f9c238f pool: add option to enableCrushUpdates to pool if required
this will be disabled by default but in scenarios where the user want to
update failureDomain, DeviceClass etc, this option can be enabled, to be
noted this can lead to lot of data rebalancing and remapping. Use with
caution

Signed-off-by: Deepika Upadhyay <deepika.upadhyay@clyso.com>
2024-06-12 22:48:25 +05:30
Travis Nielsen e90ed3b912 mon: allow overriding the mon endpoint with annotation
For an advanced network configuration, the mon endpoints
expected for the ceph public network may be different from
the public IP address that is set on the K8s node. When
host networking is enabled in rook, the IP address of the
node was only allowed previously. Now, the IP address
used for a mon when scheduled on a node can be determined
by setting an annotation on each node where mons can run:
    network.rook.io/mon-ip: <ip-address>
This annotation must be set before the mons are created.
Mon endpoints cannot be changed after they are already
running.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-06-05 10:32:55 -06:00
Jiffin Tony Thottan ba40f84123 object: update cosi images
Updating images for ceph cosi driver and side car.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2024-06-04 19:32:45 +05:30
subhamkrai 4be450ae4b csi: skip creating networkFence when csi disabled
skip creating networkFence when 'ROOK_CSI_DISABLE_DRIVER' is set to
'false'.

Signed-off-by: subhamkrai <srai@redhat.com>
2024-06-04 18:17:38 +05:30
Blaine Gardner 08a27940ed multus: add and test ipv6 support for validation tool
Add IPv6 support for multus validation tool. Also test that IPv6 support
works by specifying one of the NetAttachDefs with an IPv6 address range.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2024-06-03 17:30:36 -06:00
Travis Nielsen 2a5301a5cc Merge pull request #14269 from rook/dependabot/go_modules/github-dependencies-f460172902
build(deps): bump the github-dependencies group across 1 directory with 5 updates
2024-05-31 15:56:28 -06:00
dependabot[bot] 2ddbdebf43 build(deps): bump the github-dependencies group across 1 directory with 5 updates
Bumps the github-dependencies group with 5 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [github.com/IBM/keyprotect-go-client](https://github.com/IBM/keyprotect-go-client) | `0.13.0` | `0.14.0` |
| [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) | `1.53.0` | `1.53.10` |
| [github.com/hashicorp/vault/api](https://github.com/hashicorp/vault) | `1.13.0` | `1.14.0` |
| [github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring](https://github.com/prometheus-operator/prometheus-operator) | `0.73.2` | `0.74.0` |
| [github.com/prometheus-operator/prometheus-operator/pkg/client](https://github.com/prometheus-operator/prometheus-operator) | `0.73.2` | `0.74.0` |

Updates `github.com/IBM/keyprotect-go-client` from 0.13.0 to 0.14.0
- [Release notes](https://github.com/IBM/keyprotect-go-client/releases)
- [Changelog](https://github.com/IBM/keyprotect-go-client/blob/master/CHANGELOG.md)
- [Commits](https://github.com/IBM/keyprotect-go-client/compare/v0.13.0...v0.14.0)

Updates `github.com/aws/aws-sdk-go` from 1.53.0 to 1.53.10
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.53.0...v1.53.10)

Updates `github.com/hashicorp/vault/api` from 1.13.0 to 1.14.0
- [Release notes](https://github.com/hashicorp/vault/releases)
- [Changelog](https://github.com/hashicorp/vault/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/vault/compare/v1.13.0...v1.14.0)

Updates `github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring` from 0.73.2 to 0.74.0
- [Release notes](https://github.com/prometheus-operator/prometheus-operator/releases)
- [Changelog](https://github.com/prometheus-operator/prometheus-operator/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus-operator/prometheus-operator/compare/v0.73.2...v0.74.0)

Updates `github.com/prometheus-operator/prometheus-operator/pkg/client` from 0.73.2 to 0.74.0
- [Release notes](https://github.com/prometheus-operator/prometheus-operator/releases)
- [Changelog](https://github.com/prometheus-operator/prometheus-operator/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus-operator/prometheus-operator/compare/v0.73.2...v0.74.0)

---
updated-dependencies:
- dependency-name: github.com/IBM/keyprotect-go-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-dependencies
- dependency-name: github.com/hashicorp/vault/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
- dependency-name: github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
- dependency-name: github.com/prometheus-operator/prometheus-operator/pkg/client
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-05-31 12:28:20 -06:00
Michael Adam 823df7be98 build: add result of codegen
`make codegen`resulted in a diff, so that needed to be committed.

Signed-off-by: Michael Adam <obnox@samba.org>
2024-05-31 19:16:23 +02:00
Travis NielsenandBlaine Gardner f7f12ecf99 network: improve the IsHost function
This change is a follow-up to PR https://github.com/rook/rook/pull/13651 .

It is also meant to provide a continuation of the closed, stale
PR https://github.com/rook/rook/pull/13878 .

It is  furthermore  intended as a preparation for the introduction of
a way to enforce host networking in the next round of updates to PR
    https://github.com/rook/rook/pull/13651 .

This change improves the documentation,  implementation,  and unit testing for the   method of the
 spec.

Signed-off-by: Michael Adam <obnox@samba.org>
Co-authored-by: Blaine Gardner <b.blaine.gardner@gmail.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
2024-05-31 12:53:25 +02:00
Travis Nielsen e479b051bc osd: configure cluster full settings when osds fill up
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2024-05-30 12:10:41 -06:00
Travis Nielsen 48b657099d Merge pull request #14232 from sp98/configure-balancer-mode
mgr: setting balancer mode from cluster spec.
2024-05-29 10:44:29 -06:00
Santosh Pillai f5ca09e385 Merge pull request #14196 from sp98/skip-osd-uprade-during-migration
osd: skip upgrade if migration is pending
2024-05-29 12:33:39 +05:30
sp98 322549cbe1 osd: skip upgrade if migration is pending
Currently we allow upgrade of other OSDs while migration
of OSD, due to change in backing store, is pending. This
will not work if the updated OSD ceph version does not support
the currently applied backing store. So rook will skip any
OSD upgrade if the OSD migration is pending.

Signed-off-by: sp98 <sapillai@redhat.com>
2024-05-28 11:27:28 +05:30
Brian Dowling ef0ed93134 osd: reduce safe-to-destroy retry timeout to 15s
This reduces the safe-to-remove retry interval from 1m to 15s.

Closes #14247

Signed-off-by: Brian Dowling <bjd-dev@simplicity.net>
2024-05-27 13:54:21 -04:00
sp98 1ad37ecce9 mgr: setting balancer mode from cluster spec
This PR adds a `settings` option in the `Module` struct
which can help users to futher customize the `balancer`
module with a differet mode.

Signed-off-by: sp98 <sapillai@redhat.com>
2024-05-27 11:30:10 +05:30
subhamkrai 85c25bbc6e ci: fix codespell ci by skipping some words
fixing 'secert' to `secret` and adding workds like
`NotIn, addin` to skip work list.

Signed-off-by: subhamkrai <srai@redhat.com>
2024-05-27 10:26:34 +05:30
Blaine Gardner a138a38cdb Merge pull request #14248 from BlaineEXE/fix-mon-ipv6-networking
mon: fix bind addr when IPv6 and msgr2 are enabled
2024-05-23 07:31:12 -06:00
Blaine Gardner 573a47e736 mon: fix bind addr when IPv6 and msgr2 are enabled
When IPv6-only network mode is enabled and when mons are configured to
listen only on msgrV2 port, Rook has to add the port to the mon
--public-bind-addr in order to force mons to stop listening on the
msgrV1 port. This can cause problems for IPv6 clusters where adding
':<port>' to the end of an IP addr looks like a valid IPv6 address.

In IPv6 mode, Rook needs to set this to '[<ip>]:<port>' with the
brackets to be a non-ambiguous IPv6 address.

In IPv4 mode, Rook needs to keep its existing behavior. IPv4 mode should
also continue to be the defualt assumption if the network config isn't
specified.

If 'dualStack: true' is specified, Rook is informed that the environment
is dual stack, but Rook doesn't know which IP family to expect from the
pod IP. In this scenario, it is safest for Rook not to apply the port,
since assuming either IPv4 or IPv6 format could be incorrect. In this
scenario, mons may still listen on msgrV1 ports, but this is preferable
to an environment where mons perpetually crash.

Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
2024-05-21 14:42:06 -06:00