Once the cluster is connected, we need to create the CSI keys so that we
can provide persistent storage to containers, via ceph csi.
Signed-off-by: Sébastien Han <seb@redhat.com>
**Description of your changes:**
This modification adds the information extracted from 'ceph-volume inventory':
command to the device configmaps generated by the discovery daemon when
"rook discover" starts with the new boolean "--use-ceph-volume" parameter.
Resolves #
https://github.com/rook/rook/issues/2606
Now the <cephVolumeData> field contains all the information returned
from <ceph-volume inventory> command.
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
This will be helpful to list the CSI drivers
deployed in the kubernetes cluster. if we have
N number of CSI drivers deployed in the cluster,
the user can query kubectl get csidriver and get
the list of registered CSI drivers and it requires
kube 1.14+
This removes the deployment of Attacher sidecar container,
as we are setting the attacher required to false during
the CSIDriver object creation.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Failure to create a keyring prior to creating a deployment
dependent on that keyring created a race conditition resulting
in an intermittent and unnecessary pod failure. This change
creates a keyring prior to deployment creation for MDS
daemons.
Partially fixes: rook#4089
Signed-off-by: egafford <egafford@redhat.com>
Failure to create a keyring prior to creating a deployment
dependent on that keyring created a race conditition resulting
in an intermittent and unnecessary pod failure. This change
creates a keyring prior to any RGW deployment.
Partially fixes: #4089
Signed-off-by: egafford <egafford@redhat.com>
retry ObjectUser creation as long as valid input is provided
or it times out. Retry every 15sec for 5min.
integration test creates ObjectUser before ObjectStore to
ensure that ObjectUser waits for ObjectStore to be created
and available
Closes: https://github.com/rook/rook/issues/3937
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
The topology of the cluster should be based on the node labels rather
than a setting in the cluster CR. This allows a much richer and more
dynamic topology to be configured. The location will now be ignored
if specified in the cluster CR.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We removed the topologyAware CRD option since it was redundant with the
use of an OSD being backed by a PVC.
So now, if an OSD is backed by a PVC we assume the topology aware
decision and will discover zone and region labels on that host.
Signed-off-by: Sébastien Han <seb@redhat.com>
The node labels were already supported for zones and regions to add to
the CRUSH map. Now all layers of the CRUSH map will be supported
with the new labels such as topology.rook.io/rack.
The labels will be detected at the osd startup time
similar to the zone and region labels already being detected.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
- Implement exportable TolerationSet with the following differences:
- Change map key to a string to ensure it is deterministic.
The old key was a struct that included a pointer.
- Sort the outputted list by this key to ensure the order is deterministic.
- Get the tolerations from the deployment instead of the pod as pod tolerations
are sometimes modified by the system. We only want user provided tolerations
in our deployment.
- Modify tests to try duplicate keys from different structs. The old way tested duplicate
keys using the same structs.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
when running OSD on PVC, orchestration should not be started again if there is any change in the device config
map. Added a check to skip orchestration on device CM update when StorageClassDeviceSets are available,
that is, when running osd on pvc.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
If the cluster has previously been initialized and the operator
is restarted, the operator needs to perform the first orchestration
to confirm basic cluster health, then start watching for CR events.
If there was a single mon pod in a failed or pending pod state, the
orchestration was blocked even if just one mon was in this state.
The scenario is when a node is lost and a mon and the operator happen
to be running on that node. The operator will restart, but the mon
will not restart because it is tied to the node that died (unless running
on a PVC). The mon would remain in a pending state while attempting
to schedule on the node that is down indefinitely and the operator would
be blocked indefinitely. Now we will just log the error and attempt
to continue.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Some testing on OpenShift has shown that enabling Hostnetworking causes
issues with ceph-volume creating OSD.
This is a pure Red Hat OpenShift downstream issue.
This is meant to be temporary while we are waiting for the Selinux team
to get back to us.
Signed-off-by: Sébastien Han <seb@redhat.com>
Change the canaries to use the rook image instead of the busybox image to
run `sleep infinity`. This removes our dependency on an external image that
we do not control.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
Generating keyrings after deployments created a race condition
resulting in intermittent OSD pod failure on the first attempt.
This change creates keyrings prior to creation of OSD deployments
and updates the owner after the creation of a deployment.
Partially fixes: rook#4089
Signed-off-by: egafford <egafford@redhat.com>
If a volume is being attached, it should be verified that the volume
is safe to attach. The volume was assumed to be safe to attach if it was
for the same pod. But this was assuming the pod of the same name would be
on the same node. This is true for pods created from deployments, but not
for pods that are part of a stateful set. A stateful set will maintain the
pod name even as the pod is failed over to a new node. Therefore, the fencing
much check if the pod is from the same node before allowing the attach to
continue. Otherwise, we need to wait for the volume to be detached from the
other node.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The owner reference for the NFS custom resource was using the namespace
instead of the name. This causes the k8s garbage collector to remove the
nfs resources in some scenarios such as restarting etcd.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This will be helpful to list the CSI drivers deployed
in the kubernetes cluster. if we have N number of
CSI drivers deployed in the cluster, the user can
query kubectl get csidriver and get the list of
registered CSI drivers and it requires kube 1.14+
and also the functionality is disabled by default
More info here: https://github.com/ceph/ceph-csi/pull/698
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In Octopus, this commit
https://github.com/ceph/ceph/pull/30859/commits/c09b82a80a392ccd0da7677c7b424ce5cd3fa5d6
moved the quorum_status command to a socket call. Thus, the `ceph
quorum_status` command does not exist anymore, to use it we need to do
`ceph daemon mon.$ID quorum_status`. However, this command only works
locally so it won't work if we run it from the operator.
The `ceph mon_status` command is a general command that run from
anywhere and returns the mon map, let's use this one on the operator
code.
This is not a breaking change.
Signed-off-by: Sébastien Han <seb@redhat.com>
Even legacy osds were still running with this flag. In my previous patch
I was trying to minimize the impact of the change and was afraid the
config file was holding important information but that appears not to be
true.
The only option in the config could have the keyring path, but it's
passed in the CLI startup line.
Closes: https://github.com/rook/rook/issues/4063
Signed-off-by: Sébastien Han <seb@redhat.com>
Guangming Wang
RAJAT SINGH
Sébastien Han
Juan Miguel Olmo Martínez
Madhu Rajanna
Travis Nielsen
egafford
Blaine Gardner
Nizamudeen
Umanga Chapagain
d-luu
Rohan CJ
Santosh Pillai
Dmitry Yusupov
Ashley Williams
Anton Skriptsov
Song Gao