An OSD on a PVC when portable=false is assigned to a node
with a node selector. The same node assignment is expected
for the lifetime of the cluster. On subsequent reconciles,
the operator was looking up the node assignment from the
nodeName on the pod spec, which is not set if the pod is down.
The operator needs to retrieve the assignment from the
deployment spec node selector.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph gid (167) provided as supplymentary group, so that all the mounts can
be accessed by the same. This is required for ssl certificate consumed by
the rgw server from ceph octopus onwards.
Closes: https://github.com/rook/rook/issues/5265
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
When an OSD pod is stuck in terminating state, there is no need
to check if the OSD is marked as out before force deleting.
As long as the osd is down we will force delete the pod.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If ROOK_CSI_ALLOW_UNSUPPORTED_VERSION=false and the csi canary image
is set to quay.io/cephcsi/cephcsi:canary, the csi driver fails
to load as expected since it is not supported. But if the
flag is set to true, it is expected to allow the canary image
which isn't versioned. The supported version check was requiring
that the version be detected in the image.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
There are many duplicated code to judge whether an OSD is on PVC
or not. Instead, let's introduce a function for this purpose.
It can also be applied to the logic of judging whether an OSD
on PVC has metadata device or not.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
provide an option to set PriorityclassName on the
CSI provisioner pods to avoid pod eviction, as the CSI provisioner
pods a critical in system for storage. the default value
for the CSI provisioner pods is set to system-cluster-critical.
provisioner pods are less priority than plugin pods.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The operator settings can come either from the configmap, an
env var, or a default value. Knowing where the setting is loaded
from is an important detail to see in the log.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
cephcsi v2.1.1 is released with fix for xfs formatting
issue. This PR updates the cephcsi image version to
v2.1.1
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the OSD pod is portable and is on a failed node, the node
will be stuck indefinitely in the terminating state. The pod
can be force deleted in this scenario in order to allow the
underlying volume to be detached and then mounted on another
node.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Update CSI spec code to fetch the resources from the
configmap or ENV and apply the specific resources
to the cephfs,rbd and kubernetes sidecar containers.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
YamlToContainerResource can be used to convert the raw
string data to the array of ContainerResource.
the ContainerResource can be applied to the
pod container resources request and limits.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This is the final conversion to controller-runtime conversion. This time the
CephCluster CRD has been converted to use the controller-runtime
library.
The controller incorporates all the previous watchers too, so the Node
and hot-plug configmap are been watched too.
Only the operator setting configmap is not being watcher since it's not
related to the CephCluster CRD.
Not only the patch converts to controller-runtime but also tries to
re-organize the tree of the repo to actually make the code more
readable and have better functions/methods/tests separations.
Closes: https://github.com/rook/rook/issues/4939
Signed-off-by: Sébastien Han <seb@redhat.com>
Remove specific mds config added to the mon configuration database when deleting a CephObjectstore
or when scaling down number of mds
Also remove mds CephX key
Signed-off-by: n.fraison <n.fraison@criteo.com>
There are some code where constants of environment variables
are defined but not used properly. Let's avoid to use string
literals as possible.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The name of a CephCluster is commonly the same as the namespace,
but not always. When looking up the ceph cluster we can look for
the first one in the namespace rather than requiring the name
to be the same as the namespace.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Use same method for the deletion of rgw CephX key and configuration in centralized mon database
when removing a CephObjectStore or scaling down number of rgw
Signed-off-by: n.fraison <n.fraison@criteo.com>
Remove specific rgw config added to the mon configuration database when deleting a CephObjectstore
or when scaling down number of rgw
Signed-off-by: n.fraison <n.fraison@criteo.com>
cephcsi v2.1.0 is released and doesnot need any deployment
template changes. This PR updates the cephcsi image
version to v2.1.0
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Currently, we need to configure the Ceph external Admin keyi
in the Rook deployment to be able to connect to an external Ceph cluster.
If we wanted to run in a multi-tenant fashion
were several K8s/Rook clusters wanted to connect to the same external ceph cluster,
each K8s deployment would have the access to the External Ceph Admin key
and could potentially access or delete the Data
from pools that belong to other k8s/Rook Clusters.
Now the admin key is optional but the helper script create-external-cluster-resources.sh
will help create the necessary keys/users to connect to that cluster.
Closes: https://github.com/rook/rook/issues/4917 and https://github.com/rook/rook/pull/5227
Signed-off-by: Sébastien Han <seb@redhat.com>
The mgr modules take some time to configure during every reconcile loop.
The OSDs were waiting until the mgr module config was completed. Since
the mgr module config is optional or at least not critical to the rest
of the reconcile we can do it in parallel.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Previously, the rbd-mirror daemon was integrated into the `CephCluster`
CRD. This wasn't really practical since we would have to wait for the
whole orchestration to be done to actually set it up. The same goes for
any CR update. Let's say you want to change the number of daemons, Rook
would go through mons, mgrs and osds until it get to rbd-mirror.
This triggers an undesired full orchestration.
With its own CRD this component just gains a lot more flexibility.
Closes: https://github.com/rook/rook/issues/5084
Signed-off-by: Sébastien Han <seb@redhat.com>
When a daemon is not safe-to-stop the operator will print the full
stack trace rather than a simple error message that the daemon
cannot be stopped.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSI CephFS provisioner fails to provision a Persistent Volume after upgrading from Rook Ceph Operator 1.1.2 to 1.1.7.
This issue can be fixed by removing givePoolAppTag function whenever there is an appName cephFs.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
In the v1.3 release the target port for the rgw daemons changed to 8080.
The upgrade was skipping the service update anytime the service already existed.
Now the service will be updated with every reconcile to ensure the target port
is correct.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The controllers should ignore events for status updates. The reconcile
only needs to happen when the spec is updated or the resource is marked
for deletion. Otherwise, the controllers may stay in an update loop as
the reconcile events are triggered again every time there is a status
update during the reconcile.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When updating the status on resources sometimes the update fails due to
the resource being an outdated version. This frequently occurs when
the same reconcile loop updates the status multiple times, or the finalizer
is added, or some other update to the resource. Upon the next reconcile
the error would generally go away since the status or finalizer didn't
need to be updated multiple times. But now the error is not expected
since the controller will retrieve the latest version of the resource
immediately before attempting to update it.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Travis Nielsen
Sébastien Han
Jiffin Tony Thottan
Satoru Takeuchi
Madhu Rajanna
Dmitry Yusupov
Dmitry Yusupov
n.fraison
RAJAT SINGH
Michael Adam