When attempting to use pre-existing (or cephBlockPool managed) pools and
the cephObjectStore.spec.{metadataPool,dataPool} fields are not
specified, rgw creation will fail with this error:
2024-09-25 21:00:17.744358 E | ceph-object-controller: failed to reconcile CephObjectStore "rook-ceph/test1". failed to create object store deployments: failed to create object pools: failed to create metadata pools: failed to create pool "test1.rgw.control": pool "test1.rgw.control" type is not defined as replicated or erasure coded
Signed-off-by: Joshua Hoblitt <josh@hoblitt.com>
(cherry picked from commit 9b952e0a55)
This adds an operator config setting ROOK_REVISION_HISTORY_LIMIT
defaulting to kubernetes'value for RevisionHistoryLimit.
If configured, the provided value will be used as RevisionHistoryLimit
for all Deployments rook creates.
Fixes: #12722
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit ab8fd90aa6)
default application name is updated inside the `CreatePool` method. Send
pool spec as address in order to preserve this change.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
(cherry picked from commit 1a95d6f533)
deployments on openshift where we need csi container pods
to be run on privileged, the logrotate sidecar container was missing the
privileged permission letting, side car to not run properly
Signed-off-by: parth-gr <partharora1010@gmail.com>
(cherry picked from commit 21ca333ed7)
The crash collector daemon had missed the ability to
add custom annotations via the cephcluster CR, as already
supported for all the other daemons.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit fa2941d098)
allows to set prio-limit and stats-period options to ceph exporter
Signed-off-by: Artem Torubarov <artem.torubarov@clyso.com>
(cherry picked from commit b2c39ace25)
The crush rules may have a negative step num.
Rook had assumed negative values were not possible,
but just had not been encountered previously in
a custom crush rule.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit b9188c793a)
to keep it simple we'll not delete the csi-operator resources
we'll only document this for users who want to switch to csi-drivers.
Anyways by next release we'll only support csi-operator only.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit a16ad75c92)
Only single pool will get created if there are multiple
data pool entries with same name. This PR just adds a check
to fail if duplicate pools are present.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 8fa8612201)
we have 3.12.2 as the new cephcsi release
updating the rook to use the same.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d041be4bcf)
When removing a mon from quorum, there is a race condition that
can result in mon quorum going being lost at least temporarily.
The mon pod was being deleted first, and then the mon removed
from quorum. If any other mon went down between the time the
pod of the bad mon was deleted and when the mon was removed from
quorum, there may not be sufficient quorum to complete the
action of removing the mon from quorum and the operator would
be stuck.
For example, there could be 4 mons temporarily due to timing
of upgrading K8s nodes where mons may be taken down for some
number of minutes. Say a new mon is started while the down
mon also comes back up. Now the operator sees it can remove
the 4th mon from quorum, so it starts to remove it. Now say
another mon goes down on another node that is being updated
or otherwise drained. Since the 4th mon pod was deleted
and another mon is down, there are only two mons remaining
in quorum, but 3 mons are required in quorum when there
are 4 mons. Therefore, the quorum is stuck until the
third mon comes back up.
The solution is to first remove the extra mon from quorum
before taking down the mon pod.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 8987d26a6d)
This new setting is of Boolean type and defaults to "false".
When set to "true", it changes the behavior of the
rook operator to
nable host network on all pods created by the cephcluster controller
new method to check the setting: opcontroller.EnForceHostNetwork()
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit e378588359)
When doing the clean-up of raw device OSDs, it was not taking metadata
and wal devices into account. This commit discover them via the output
of `ceph-volume raw list`, which is returning `device_db` and/or
`device_wal` if they are present.
After discovering the metadata and wal device it also clean them and
close the encrypted device if any.
Signed-off-by: Mathias Chapelain <mathias.chapelain@proton.ch>
(cherry picked from commit 52ca68d46e)
Allow RGW users to mount arbitrary volumes to RGW pods. This follows the
pattern that was established for NFS to support SSSD and LDAP, and
reuses much of the same code.
This opens the door wider for advanced users to take advantage of some
Ceph RGW features that Rook doesn't have first class support for.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit eea43b7b79)
Use os.TempDir() instead of '/tmp'. This is for the peermap tests.
Signed-off-by: chanyoung <park910113@gmail.com>
(cherry picked from commit d77f6e5e3d)
Previously, Rook would fail to create monitors for a `CephCluster` spec that
required them to be located in distinct failure domains:
```yaml
spec:
mon:
count: 3
failureDomainLabel: topology.kubernetes.io/zone
zones:
- eu-central-1a
- eu-central-1b
- eu-central-1c
```
This was due to the fact that Rook was mistakenly getting the
`failureDomainLabel` from `spec.mon.stretchCluster.failureDomainLabel`.
This commit ensures `spec.mon.failureDomainLabel` is being used instead, which
fixes the cluster deployment for the spec above.
Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
(cherry picked from commit 1c45677f5e)
Alerting on controller-runtime's workqueue_depth can be useful for
debugging controllers. Also having a prometheus target for a pod gives
another data point that the system is working as expected. It is useful
for uptime alerts.
Make the bind address configurable via the configmap while still retaining the default
behavior that it is disabled.
Resolves: #14538
Signed-off-by: Justin Cichra <jcichra@cloudflare.com>
(cherry picked from commit 74a79b24b3)
this commit contains changes that are forced by latest
controller-runtime version v0.19.0.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 2556e14780)
updating csi-operator version in go.mod and csi-operator.yaml
to use latest release v0.1. And also fixing one small bug
where csi-driver log level was set in csi-operator log level.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 89e9077e72)
Updating storageClassName for cluster-on-pvc.yaml file from gp2 to gp2-csi.
Signed-off-by: ShravaniVangur <shravanivangur@gmail.com>
(cherry picked from commit 9656d1963a)
The codegen needs updating since the swift
types were added.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 52f892af76)
The jobs to detect the ceph and csi versions now can have
custom annotationso and labels added to them. They are
very short-lived jobs, but they may need custom annotations
in some environments to run.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 5014d5bcf2)
As we have new csi-addons v0.9.0
updating the same here as well.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 123025f22c)
# Conflicts:
# go.sum
adding unit test for csi-op resources and also
fixing small pointer issue with this commit.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit f40cb392c9)
adding annotation to existing CSI driver resource so that
csi-operator can take ownership of existing CSI driver resources.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 64cce51f03)
adding changes to create CSI-operator config CR based on
every radosnamesapce and subvolumegroup.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 667e044013)
adding code changes,rbac changes required for create the new
Ceph-CSI operator CR named cephCluster in api group 'csi.ceph.io'.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 4b0b3a55d9)
For the specification see:
<https://github.com/rook/rook/blob/master/design/ceph/object/swift-and-keystone-integration.md>
* extend the API object specs for swift and keystone integration
* adapt rgw to the new go-ceph version
- The parameter lists of the API call have changes, as parameters
ignored by the RGW Admin Ops API are no longer serialized, therefore
the mock has to be adapted.
- There is now validation for the user keys that are passed to the
User get API, therefore things failed when we had empty keys in our
User proxy object.
* expand the reconcile loop for the swift and keystone integration
* fix minor mistakes in design document
* add env var to pass extra args to minikube
Minikube decides CPU cores and memory automatically based on the
available resources on the machine which may be insufficient to
run rook. This commit adds an environment variable to add arbitrary
arguments to the minikube command, so both can be specified if
desired.
* integration tests for swift and keystone
The new integration of swift or s3 and keystone support by rook
does not have any integration tests yet.
This commit introduces integration tests for swift and keystone. The
tests are done against a minimal keystone setup (keystone container
image from Yaook-project (https://yaook.cloud), sqlite as database
backend, cert-manager and trust-manager for test certificate setup).
To prevent hardcoded credentials, passwords are generated
by the tests. The integration tests use the openstack client
(keystone- and swift-functionality) (https://docs.openstack.org/
python-openstackclient/ latest/). This was a concious design decision
to use client tooling as close as possible to the end user instead of
using other go-libraries (such as gophercloud).
* add documentation on swift and keystone
Currently there is no documentation on the use of Swift to access
an object store as well as the use of OpenStack keystone for
authentication.
This commit adds documentation on the use of Swift and OpenStack
keystone, as well as CRD-related documentation and an example setup.
* add integration tests for S3 via keystone
This commit introduces integration tests for s3 and keystone. The
tests are run against the same minimal keystone setup that the tests
for swift and keystone use.
The integration tests use the aws s3 client to use client tooling as
close as possible to the end user instead of using other go-libraries.
Co-authored-by: Jan Klippel <jan.klippel@uhurutec.com>
Co-authored-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
Signed-off-by: Sebastian Riese <sebastian.riese@cloudandheat.com>
Signed-off-by: Jan Klippel <jan.klippel@uhurutec.com>
Signed-off-by: Silvio Ankermann <silvio.ankermann@cloudandheat.com>
(cherry picked from commit ee8bcad49d)
RGW can only serve a single certificate. This limitation means that the
prior behavior of using the default service for admin ops when TLS is
enabled may mean it requires additional complex certificate management
to make sure the object store uses a certificate valid for Rook internal
admin ops and user connections.
This is needlessly complex for users. Instead, change Rook's behavior
and documentation to clarify that it will use the same endpoint intended
for S3 client applications. This means that users have a more
straightforward path to enabling both Rook and consuming applications.
More info: https://github.com/rook/rook/issues/14530
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit b4a2285aa6)
issue:
external-provisioner (v5) enabled topology feature-gate by default
and the current implementation in Rook uses a conditional block to
enable the topology feature gate. This approach now does not directly
reflect the state of the `CSI_ENABLE_TOPOLOGY`.
fix:
replacing the conditional block with a direct use of the
`CSI_ENABLE_TOPOLOGY` for flag value.
Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit e30e5e881f)
Add CephObjectStore spec.hosting.advertiseEndpoint configuration. This
provides a clear documented default for which endpoint Rook "advertises"
to dependent resources like CephObjectStores, OBCs, and COSI
Buckets/Accesses and allows users to override the default behavior if
desired.
The current default is to round-robin an endpoint from
spec.hosting.dnsNames, which has proven to be troublesome for some
users' object store configurations. This change provides much-needed
disambiguation for users.
This may be a breaking change for some existing spec.hosting.dnsNames
users. This is unexpected but is documented.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit a2b0b6449c)
Joshua Hoblitt
Michael Adam
mergify[bot]
Santosh Pillai
parth-gr
subhamkrai
parth-gr
Travis Nielsen
Artem Torubarov
Madhu Rajanna
Mathias Chapelain
Blaine Gardner
chanyoung
Benoît Knecht
Justin Cichra
Praveen M
ShravaniVangur
Sebastian Riese
dependabot[bot]