default application name is updated inside the `CreatePool` method. Send
pool spec as address in order to preserve this change.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Updating the device class swallowed any error if updated
for the pool. The error was not even logged, so we couldn't
troubleshoot why the new crush rule was not applied.
Log the error for troubleshooting and also fail the pool
reconcile since the desired configuration was not applied.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Someone testing the Multus holder pod removal feature encountered an
issue where the migration process failed to lead to a system state where
PVCs could be created successfully.
The root cause was found to be a ceph csi config map wherein the primary
CephCluster entry was lacking a value for the "namespace" field.
I observed this once in my development on the holder pod removal
feature, but I was unable to reproduce and assumed it was my own error.
Since this has been seen in a user environment, it must be that the
error is a race condition, and I am unable to determine the exact source
of the bug.
I do not believe this bug would be present if the code that updates the
CSI configmap were properly idempotent, but it has many conditions based
on prior states, and I was unable to determine how to resolve this
underlying impelementation pattern issue.
Instead, I opted to separate the `clusterKey` parameter into two clear
parts:
1. `clusterID` for when `clusterKey` is used as an analogue for
`clusterID`
2. `clusterNamespace` for when `clusterKey` is used as an analogue for
`clusterNamespace`
I added unit tests to ensure that SaveClusterConfig() will be able to
detect when the namespace is currently missing, and using the new
`clusterNamespace` field, it should always know what value to use as
input when correcting the bug in already-installed systems.
I also verified that this update works when the function simultaneously
removes netNamespaceFilePath entries, and that those entries are removed
properly.
Finally, manual testing also verifies the change.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
make the kubectl output for crds more verbose.
Following changes was done for each crd:
CephBlockPool:
Added print columns:
- Type
- FailureDomain
- Replication
- EC-CodingChunks
- EC-DataChunks
- Age
Extended info field of status to include:
type and failureDomain
CephObjectStore:
Added print columns:
- Endpoint
- SecureEndpoint
- Age
Added Age to print columns of following crds:
CephObjectStoreUser, CephObjectZoneGroup,
CephObjectZone, CephBucketTopic, CephClient,
CephRBDMirror, CephFilesystemMirror,
CephFilesystemSubVolumeGroup
Signed-off-by: NymanRobin <nyman.robin@gmail.com>
Pool name either come from CephBlockPool name or
form the CephBlockPool Spec. Use the correct name
when trying to disable mirroring.
Signed-off-by: sp98 <sapillai@redhat.com>
For image mode mirroring, if cephBlockPool.Pool.Spec.Mirroring.Enable
is set to false, then remove the peer cluster and disable mirroring on
all the pool if the user has disabled mirroring on all the pool images.
If mirroring is not disabled on all the pool images, then reconcile will
fail asking the users to manually disable mirroring on those images.
Signed-off-by: sp98 <sapillai@redhat.com>
Rook has been setting the application automatically on all
pools to rbd for CephBlockPools, rook-ceph-rgw for
CephObjectStores, mgr on the built-in .mgr pool,
and nfs on the built-in .nfs pool.
The legacy pool device_health_metrics is long gone
from Pacific which is no longer supported, so we can
remove special handling for that pool in the upgrade
guide and in the code.
The application setting is now available on the pool spec
although it is not expected to commonly need to override
the default applications set by Rook.
The application for CephFilesystem pools is now being
set to cephfs, where it was previously blank.
Signed-off-by: travisn <tnielsen@redhat.com>
Corrected the NetNamespaceFilePath generation in the
subvolumegroup & radosnamespace controller by replacing
the cephCluster.Name with cephCluster.Namespace.
Signed-off-by: Praveen M <m.praveen@ibm.com>
This commit uses an csi config map API exposed by Ceph-CSI.
This will reduce the efforts of keeping the configuration in
sync with Ceph-CSI.
Signed-off-by: Praveen M <m.praveen@ibm.com>
This commit adds new CSIDriverOptions section in
cephCluster CR. This section contains settings
for read affinity and kernel+fuse Mount options
These settings will be injected directly into
rook-ceph-csi-config cm to be applicable per
ceph cluster.
Signed-off-by: Rakshith R <rar@redhat.com>
Some logging instructions use formatting syntax `%q`, but the logging method is
`Info` or `Debug` instead of `Debugf` and `Infof`. As a result, the arguments of
the method were concatenated instead of formatted properly.
Signed-off-by: Lucas Henry <polyedre@disroot.org>
When msgr2 is required, ensure the mon endpoints passed to
the csi configmap are on port 3300. The mons will be
listening on both 6789 and 3300. Existing volumes can
continue using port 6789 while new volumes will use
port 3300.
Signed-off-by: travisn <tnielsen@redhat.com>
Rook currently knows rbd pools only if they defined through CephBlockPool, so it overrides the config
mgr/prometheus/rbd_stats_pools which may have already been set for pools external to Rook. Check config
before running set command and if there some existing pools append them to enableStatsForCephBlockPools.
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
if Multus is enabled the clusterinfo should be updated with
network as multus as to run the ceph cmds in remote
executor
Signed-off-by: parth-gr <paarora@redhat.com>
`rbd pool init` cmd initializes pool for rbd images.
This commit makes modification to init only
rbd application pools, since it is not required
by other pools like ".nfs",".mgr" & "mgr_devicehealth".
Signed-off-by: Rakshith R <rar@redhat.com>
The cephcluster.ClusterName is no longer available in the k8s api
with the ObjectMeta type, so we use the cephcluster.Name
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
During deletion of a CephBlockPool CR, the underlying ceph
pool was not being deleted. During the deletion sequence
this is due to the spec being refreshed upon a call to
check for dependents in ReportDeletionNotBlockedDueToDependents().
The pool name was then coming up blank during the deletion
request and rook of course then wasn't finding the pool
to delete.
Now Rook properly sets the pool name in the named spec
every time it is converted internally from a CephBlockPool
type to a NamedPoolSpec.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The CSI package needs to load clusterInfo, today this code is in the mon
package which makes the call of LoadClusterInfo impossible without
having a circular import.
Signed-off-by: Sébastien Han <seb@redhat.com>
This removes double package imports. Example:
```
"github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
```
Only one is now being used as shown in go-staticcheck ST1019
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
If a mon is added or removed in a ceph
cluster all the CSI clusterID's belongs
to the ceph cluster should also gets updated.
Added a new cluster identifier to the csi configmap,
i.e ceph clusterNamespace which helps to identify
the clusterID's belongs to that ceph cluster.
Note:- to support the upgraded clusters,we
can ask users to update the mon IP's for the
problamatic clusterID as it wont be updated
by the operator.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Built in Ceph pools (device_health_metrics, .mgr, .nfs) must be
replicated and not erasure coded.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This introduces a new CRD to add the ability
to create rados namespace for a given
ceph block pool. Typically the name of the pool
is the name of the blockpool created by rook.
Closes: #7035
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In Quincy, the device_health_metrics pool is renamed to .mgr.
In the example cluster-test.yaml, we therefore update the
example to create the pool with the new name.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
adding observedGeneration field in the cephcluster cr
status for having better control on reconciling,
as observedGeneration field will be updated by the controller
Closes: https://github.com/rook/rook/issues/9673
Signed-off-by: parth-gr <paarora@redhat.com>
The reconcile was skipping updating most pool properties for
object stores. The implementation of pools between the file,
object, and pool controllers had some duplicate code, so
this change also factors out the common code for better
reuse in a single place. Anytime a pool is created or updated,
it will now consistently update all the pool properties
that are expected to be modifiable.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the pool application is already set, skip setting it again
to avoid a warning message being logged that it is already
set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The built-in pools device_health_metrics and .nfs created by ceph
need to be configured for replicas, failure domain, etc.
To support this, we allow the pool to be created as a CR.
Since K8s does not support underscores in the resource names
the operator must translate this special pool name into
the name expected by ceph.
This also sets the basis for allowing filesystem data
pools to specify the desired pool name instead of requiring
a generated name.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When CephBlockPool, CephFilesystem, or CephObjectStore resources are
deleted after removing their finalizer, the code path to stop monitoring
was not stopping monitoring since a non-present resource does not have a
name and namespace attached. When the object is deleted, ensure the
internal representation used to stop monitoring has a name and namespace
to fix the issue.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Clean up the code used to stop health checkers for all controllers
(pool, file, object). Health checkers should now be stopped when
removing the finalizer for a forced deletion when the CephCluster does
not exist. This prevents leaking a running health checker for a resource
that is going to be imminently removed.
Also tidy the health checker stopping code so that it is similar for all
3 controllers. Of note, the object controller now uses namespace and
name for the object health checker, which would create a problem for
users who create a CephObjectStore with the same name in different
namespaces.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The cluster info is important context for the cluster controller to
create the cluster, and all the fields must be properly set.
A test cluster name was being set temporarily, resulting in
mons incorrectly getting the wrong cluster CR name. There is no
known issue from the temporary value, it was just exposed by
https://github.com/rook/rook/pull/8678 setting the value to a label.
Now the functions are more clearly named so only unit and
integration tests should be using the test value for the cluster
name where it is not important.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit adds context parameter to k8sutil deployment functions. By
this, we can handle cancellation during API call of deployment resource.
Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
Santosh Pillai
subhamkrai
Travis Nielsen
Blaine Gardner
subham rai
NymanRobin
Praveen M
parth-gr
Rakshith R
Lucas Henry
Avan Thakkar
zhucan
Eng Zer Jun
shalev.p
Josh Soref
Sébastien Han
Alexander Trost
Madhu Rajanna
Blaine Gardner
Divyansh Kamboj
Yuichiro Ueno