The selector labels used for mons cannot change during upgrade
or else the upgrade will hang when the cluster is checking
for mon status immediately after the upgrade begins.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit will add a new flag to the osd pods
which will be named according to the name of the
storageClassDeviceSet. This will help in differentiate
pods immediately.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
If the mon failover fails, if the mon is on a pvc, the pvc could remain
orphaned. This is likely to happen if there is not a node available
for scheduling the mon. Therefore, the operator will ensure after a failover
that any orphaned mon pvc is cleaned up.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the mons are based on pvcs, the pvc_name label was not being set
as expected. Now the label will always be set on both the canary and
the daemon pods.
The mon pvcs were also never being deleted during canary cleanup. This
code is now removed since we must be very careful when the pvcs are
removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the cluster is external there are no running pods so setting a
Selector on the Service Spec does not make sense, even worse, it makes
Kubernetes reconciling and overriding the configured endpoint.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the cluster is external the service monitor port name should match
with the name of the external metrics one.
Signed-off-by: Sébastien Han <seb@redhat.com>
If AllowUninstallWithVolumes is set in the cleanup policy then we don't
check and wait for the deletion of existing volumes from the
cephCluster.
Co-authored-by: Nitin Goyal <nigoyal@redhat.com>
Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
This allows users to specify additional labels to be added to the components
created by the Rook Ceph operator.
Additionally the rbd mirrors are now getting their annotations and
labels set.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
This change is the first part to improve the watch OSD nodes logic in
the Rook Ceph operator.
This commit changes the "is node valid check" to use the OSD placement
instead of the all placement. The OSD placement is "merged" with the
all placement already.
Second step will be to skip the first "iteration" of the "nodes added"
events and then start the actual watch logic.
Partially resolves#6111.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The operator will now allow the resource limits to be applied below the
recommended minimums. In small clusters, even the recommended minimums
may not be necessary. A warning is still printed to the operator log,
but we allow the configuration to continue.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Added more unit tests scenario for osd on pvc such as:
* block/db/wal
* block encryption
* block/db encryption
* block/db/wal encryption
Signed-off-by: Sébastien Han <seb@redhat.com>
This init container displays the status of the encrypted pvc. This can
be really useful for debugging purpose.
Signed-off-by: Sébastien Han <seb@redhat.com>
We now support the expansion of encrypted osds. We added a new init
container to run "cryptsetup resize".
Signed-off-by: Sébastien Han <seb@redhat.com>
If the deployment spec was updated or restarted, the "encryption-open"
init container would fail to open the device since it was already
opened on the node.
Because they are no reliable post stop hook on Kubernetes, the trick is
to use a small bash init container.
Signed-off-by: Sébastien Han <seb@redhat.com>
The osd prepare resources were only being set on the main container
while the init containers either did not have resources set, or were
inheriting the resource settings from the device set resources for osds.
Now the osd prepare resources will be consistently set based on the
prepareosd resources in the main section of the cluster CR.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This is simply a code simplification to remove an extra check for
whether the OSDs are encrypted. There is no change in functionality.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd prepare jobs and osd daemons may have different placement
requirements. In particular, the topology spread constraints
or pod anti-affinity should only work on either the prepare job or
osd daemon, but does not work effectively on both types at the same
time. Therefore, we split out the osd prepare placement to be separate
from the osd daemon placement. If the osd prepare placement is not set
the operator will fall back to the osd daemon placement for
backward compatibility.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a node is gone, we try to remove crash collector deployments so they
don't stay stale or pending. We must ensure the node is gone and not
just return an error even if we successfully deployment the orphan
deployment.
Signed-off-by: Sébastien Han <seb@redhat.com>
Now the metadata pvc will also be encrypted if the storageClassDeviceSet
has the "encrypted" flag turned on.
Signed-off-by: Sébastien Han <seb@redhat.com>
When a mon is failed over, the new mon is started before the old mon
is removed from quorum and purged from the cluster. If the mon has
anti-affinity, in small clusters or specific toplogies where sufficient
nodes are not available, the new mon may be prevented from starting simply
because the dead mon pod still exists. To ensure a clean failover, the
dead mon pod must first be stopped before the new mon is started.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For external cluster, it is not an issue to have a mon count set to 0
since Rook is not deploying/managing the monitors.
Signed-off-by: Sébastien Han <seb@redhat.com>
Based on the feature of metadata PVC for OSD: #4818, add the wal device.
For this, you need to create a new volumeClaimTemplates, its name must be "wal", if "wal" is
set, "metadata" device will refers specially for block.db device, and "wal" refers for block.wal
device in BlueStore.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
storageClassName: io1
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: wal
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
storageClassName: io1
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block, block.db and block.wal directly inside the container instead of
running ceph-volume activate.
Signed-off-by: Chen, Tingjie <tingjie.chen@intel.com>
The cluster CR status needs to be updated with the latest object
instead of a cached copy or else it could fail if there is some
other update. Using the Rook clientset ensures it is the latest
instead of using the cached controller runtime object.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The service selector labels cannot be updated during the upgrade or else the
mon commands will not respond until the mon deployment is also updated.
Since there is a mon command to check for the ceph versions before the
deployment is updated, the operator would remain stuck during the upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We can now encrypted OSD device that were provisioned via a storage
class using the PV interface.
The encryption works at the storageClassDeviceSets level, which means we
can have encrypted and non-encrypted sets.
Using the new key `encrypted` we can turn it on.
Signed-off-by: Sébastien Han <seb@redhat.com>
The labels cannot be updated on a deployment match selector.
In v1.4 a new label was added for ceph_daemon_type that is intended
to be on the pod labels, but cannot be applied to the match selectors.
Therefore, we suppress any new labels that are added to the daemons.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rename PodLabels function to CephDaemonAppLabels for more clarity about
what the function's purpose is.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
this commit handles all the gosec g601
error code (i.e Implicit memory aliasing
of items from a range statement).
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
subhamkrai
Varsha Rao
Travis Nielsen
fanjunwei
Sébastien Han
Raghavendra TalurandNitin Goyal
Alexander Trost
Madhu Rajanna
Chen, Tingjie
Blaine Gardner