When msgr2 is required, ensure the mon endpoints passed to
the csi configmap are on port 3300. The mons will be
listening on both 6789 and 3300. Existing volumes can
continue using port 6789 while new volumes will use
port 3300.
Signed-off-by: travisn <tnielsen@redhat.com>
to support mds with multus, we don't need
to add `--public-adr`, if multus is selected as
the network in cephCluster CR.
Signed-off-by: subhamkrai <srai@redhat.com>
CephFS object fails to do a initial status update.
Operator shows the following error message in logs,
'Error while updation: Operation cannot be fulfilled on
cephfilesystems.ceph.rook.io "myfs": the object has been
modified; please apply your changes to the latest version
and try again'
Function 'updateStatus' now returns the new/updated
CephFilesystem object if successfully updated or nil.
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
Introduce a new env variable ROOK_CSI_IMAGE_PULL_POLICY in rook operator configmap which should be used to
customize the imagePullPolicy for the csi driver and imagePullPolicy property in cephVersionSpec for ceph pods.
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
if Multus is enabled the clusterinfo should be updated with
network as multus as to run the ceph cmds in remote
executor
Signed-off-by: parth-gr <paarora@redhat.com>
The cephcluster.ClusterName is no longer available in the k8s api
with the ObjectMeta type, so we use the cephcluster.Name
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Block deletion of CephFilesystems when there are any raw Ceph
subvolumegroups present that have subvolumes in them. Empty
subvolumegroups will not block deletion.
One important subvolume group is "csi" which is the default location
where CSI subvolumes are kept. If this group is empty, it means that
there are no PVCs created based on the CephFilesystem in question. This
also holds true if there are external consumers of the filesystem in
external cluster mode.
Similarly, if there are any subvolumegroups (for example "_nogroup",
which includes subvolumes in the filesystem root) that contain
manually-created subvolumes, Rook will also see this and block deletion.
This comes into play currently with manually-created NFS exports.
A work-in progress aims to create a Ceph-CSI NFS export provisioner
which will likely create subvolumes in the "csi" group as well. This
implementation will catch this case also.
Rook still checks for CephFilesystemSubVolumeGroups explicitly in
addition to the check added here. This is to ensure that even empty
groups will block deletion if they are created via this CR type.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The CSI package needs to load clusterInfo, today this code is in the mon
package which makes the call of LoadClusterInfo impossible without
having a circular import.
Signed-off-by: Sébastien Han <seb@redhat.com>
This removes double package imports. Example:
```
"github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
```
Only one is now being used as shown in go-staticcheck ST1019
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
If a mon is added or removed in a ceph
cluster all the CSI clusterID's belongs
to the ceph cluster should also gets updated.
Added a new cluster identifier to the csi configmap,
i.e ceph clusterNamespace which helps to identify
the clusterID's belongs to that ceph cluster.
Note:- to support the upgraded clusters,we
can ask users to update the mon IP's for the
problamatic clusterID as it wont be updated
by the operator.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
improved the logging and also reorganized code to
handle external and non external cases for better
code readability.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
adding observedGeneration field in the cephcluster cr
status for having better control on reconciling,
as observedGeneration field will be updated by the controller
Closes: https://github.com/rook/rook/issues/9673
Signed-off-by: parth-gr <paarora@redhat.com>
If the filesystem is in a bad state such as never being
created successfully in the first place, the filesystem
removal will continuously fail and prevent the CR from
being removed. Thus, we treat the filesystem removal
as a best effort and log the issue instead of failing
the uninstall reconcile.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Added unit test for probes changes and
did re-naming of liveness probes file
as it will contain multiple types of probes
Signed-off-by: parth-gr <paarora@redhat.com>
The reconcile was skipping updating most pool properties for
object stores. The implementation of pools between the file,
object, and pool controllers had some duplicate code, so
this change also factors out the common code for better
reuse in a single place. Anytime a pool is created or updated,
it will now consistently update all the pool properties
that are expected to be modifiable.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the cluster is external, users might still want to take advantage of
the ceph-csi configmap configuration for ceph filesystem subvolume
group. This configuration is handled by the controller whenever a
cephFilesystemSubVolumeGroup CR is created or deleted.
If the present cluster CephCluster is external, the controller
(cephFilesystemSubVolumeGroup) won't handle creation nor deletion of sub volume groups.
It just assumes that the given name is correct and the controller goes ahead with the
ceph-csi configuration.
Closes: #9551
Signed-off-by: Sébastien Han <seb@redhat.com>
Prior to this, we were comparing a pointer (the memory address) with a
struct. This was obviously always failing and returned false. We must
dereference the pointer to access the data contained at that memory
location.
Closes: https://github.com/rook/rook/issues/9544
Signed-off-by: Sébastien Han <seb@redhat.com>
The rook-ceph-csi-config config map now includes CephFS subvolumegroups
when a subvolumegroup CR is created. It is useful for ceph-csi to
understand which subvolumegroup to use for a given cluster.
Signed-off-by: Sébastien Han <seb@redhat.com>
Allow specifying daemon startup probes where we also allow configuring
liveness probes. Startup probes allow Rook to tolerate when Ceph daemons
occasionally take a long time to start up while not also making
Kubernetes liveness probes slower to detect runtime failures of daemons.
Startup probes are beta in Kubernetes 1.18, so we should not enable
probes by default for earlier Kubernetes versions.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This introduces a new CRD to add the ability to create subvolumegroup
for a given ceph filesystem volume. Typically the name of the volume is
the name of the filesystem created by rook.
Closes: https://github.com/rook/rook/issues/7036
Signed-off-by: Sébastien Han <seb@redhat.com>
The built-in pools device_health_metrics and .nfs created by ceph
need to be configured for replicas, failure domain, etc.
To support this, we allow the pool to be created as a CR.
Since K8s does not support underscores in the resource names
the operator must translate this special pool name into
the name expected by ceph.
This also sets the basis for allowing filesystem data
pools to specify the desired pool name instead of requiring
a generated name.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When CephBlockPool, CephFilesystem, or CephObjectStore resources are
deleted after removing their finalizer, the code path to stop monitoring
was not stopping monitoring since a non-present resource does not have a
name and namespace attached. When the object is deleted, ensure the
internal representation used to stop monitoring has a name and namespace
to fix the issue.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Adding Recommended Labels on the resources created by rook
and using Recommended Labels in the helm chart,
for better visuals and management of k8s object
Closes: https://github.com/rook/rook/issues/8400
Signed-off-by: parth-gr <paarora@redhat.com>
Clean up the code used to stop health checkers for all controllers
(pool, file, object). Health checkers should now be stopped when
removing the finalizer for a forced deletion when the CephCluster does
not exist. This prevents leaking a running health checker for a resource
that is going to be imminently removed.
Also tidy the health checker stopping code so that it is similar for all
3 controllers. Of note, the object controller now uses namespace and
name for the object health checker, which would create a problem for
users who create a CephObjectStore with the same name in different
namespaces.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This commit adds context parameter to k8sutil job functions. By this, we
can handle cancellation during API call of job resource.
Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
This commit adds context parameter to k8sutil deployment functions. By
this, we can handle cancellation during API call of deployment resource.
Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
travisn
Blaine Gardner
zhucan
Avan Thakkar
subhamkrai
Arun Kumar Mohan
parth-gr
Joseph Lee
Eng Zer Jun
Josh Soref
FengJiankui
Sébastien Han
Alexander Trost
Madhu Rajanna
Divyansh Kamboj
Denis Egorenko
Yuichiro Ueno