On LUKS2 format, the resize operation requires the KEK to be passed. To
maintain a single simple init container for resize, we now open the
encrypted block with "--disable-keyring" so that further operations
won't require the keyring.
Signed-off-by: Sébastien Han <seb@redhat.com>
OSD prepare jobs only need to be run if the OSD deployments on PVCs were
not already started. For efficiency, we now query them in a batch instead
of querying them one-by-one with a label selector, which proves to be very
slow in large clusters.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The OSD PVCs were being detected one by one with a label selector.
This proves to be slow in large clusters. Instead, we now make this
more efficient by querying for all existing PVCs and looking up their
existing in memory with a map.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit handle golangci-lint linter staticcheck error.
`staticcheck` - Staticcheck is a go vet on steroids,
applying a ton of static analysis checks.
To see only `staticcheck` linter output
`golangci-lint run --disable-all -E staticcheck`
Signed-off-by: subhamkrai <srai@redhat.com>
The pvc template name has either been require to be empty, or one of the
expected values such as data and metadata. Those who upgrade from 1.1
may find that if they changed it to something else, their upgrade will
fail when not one of these values. If there is only a single template,
always treat it as the data template.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit will enable one more linter ineffassign
in golangci-lint.
This linter throws an error when variable is assigned and never used.
`golangci-lint run --disable-all -E ineffassign` is used detects ineffassign
errors only.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
The OSD deployments have some env vars duplicated in the pod spec for
the daemon container, which may cause the detection for patch changes
to fail. Now we ensure the env vars are not duplicated in the OSD pod
spec.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rook is now capable of configuring mirroring between sites. The
implementation works at different levels:
* CephBlockPool: which introduces a new `mirroring` configuration as well
as `statusCheck`. When turned on, Rook will enable mirroring on the
pool. It will also create a bootstrap peer token and store it in a
Kubernetes Secret. The name of that Secret can be found in the Status
field of the CephBlockPool CRD. This token can be fetched and used by
other clusters to configure the site as a peer. Mirroring can be
configured either at the pool or the image level.
* CephRBDMirror: which introduces a new `peers` configuration allowing
Rook to connect to peers by passing a Secret name. The administrator will
create a Kubernetes Secret with 2 keys: 'token' for the bootstrap peer
token and 'pool' for the name of pool. Once detected the rbd-mirror
controller will go ahead and import the peer configuration.
Pool mirroring status example:
```
status:
info:
rbdMirrorBootstrapPeerSecretName: pool-peer-token-test
mirroringInfo:
lastChanged: "2020-09-17T14:47:27Z"
lastChecked: "2020-09-17T14:48:27Z"
summary:
summary:
mode: image
peers:
- client_name: client.rbd-mirror-peer
direction: rx-tx
mirror_uuid: ""
site_name: rhcs
uuid: c50522a4-28a4-4bd3-ba68-e11780308882
site_name: 91eae0dd-06b1-4d2c-91f3-1311c9df382b-rook-ceph
mirroringStatus:
lastChecked: "2020-09-17T14:48:27Z"
summary:
summary:
daemon_health: OK
health: OK
image_health: OK
states:
replaying: 1
```
Signed-off-by: Sébastien Han <seb@redhat.com>
We only need to read the ceph cluster version if the cluster external
has external monitoring enabled.
This avoids any issue during upgrades from 1.3 to 1.4 when monitoring is
enabled and ceph caps have not been updated.
Signed-off-by: Sébastien Han <seb@redhat.com>
The selector labels used for mons cannot change during upgrade
or else the upgrade will hang when the cluster is checking
for mon status immediately after the upgrade begins.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit will add a new flag to the osd pods
which will be named according to the name of the
storageClassDeviceSet. This will help in differentiate
pods immediately.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
If the mon failover fails, if the mon is on a pvc, the pvc could remain
orphaned. This is likely to happen if there is not a node available
for scheduling the mon. Therefore, the operator will ensure after a failover
that any orphaned mon pvc is cleaned up.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the mons are based on pvcs, the pvc_name label was not being set
as expected. Now the label will always be set on both the canary and
the daemon pods.
The mon pvcs were also never being deleted during canary cleanup. This
code is now removed since we must be very careful when the pvcs are
removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the cluster is external there are no running pods so setting a
Selector on the Service Spec does not make sense, even worse, it makes
Kubernetes reconciling and overriding the configured endpoint.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the cluster is external the service monitor port name should match
with the name of the external metrics one.
Signed-off-by: Sébastien Han <seb@redhat.com>
If AllowUninstallWithVolumes is set in the cleanup policy then we don't
check and wait for the deletion of existing volumes from the
cephCluster.
Co-authored-by: Nitin Goyal <nigoyal@redhat.com>
Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
This allows users to specify additional labels to be added to the components
created by the Rook Ceph operator.
Additionally the rbd mirrors are now getting their annotations and
labels set.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
This change is the first part to improve the watch OSD nodes logic in
the Rook Ceph operator.
This commit changes the "is node valid check" to use the OSD placement
instead of the all placement. The OSD placement is "merged" with the
all placement already.
Second step will be to skip the first "iteration" of the "nodes added"
events and then start the actual watch logic.
Partially resolves#6111.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The operator will now allow the resource limits to be applied below the
recommended minimums. In small clusters, even the recommended minimums
may not be necessary. A warning is still printed to the operator log,
but we allow the configuration to continue.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Added more unit tests scenario for osd on pvc such as:
* block/db/wal
* block encryption
* block/db encryption
* block/db/wal encryption
Signed-off-by: Sébastien Han <seb@redhat.com>
This init container displays the status of the encrypted pvc. This can
be really useful for debugging purpose.
Signed-off-by: Sébastien Han <seb@redhat.com>
We now support the expansion of encrypted osds. We added a new init
container to run "cryptsetup resize".
Signed-off-by: Sébastien Han <seb@redhat.com>
If the deployment spec was updated or restarted, the "encryption-open"
init container would fail to open the device since it was already
opened on the node.
Because they are no reliable post stop hook on Kubernetes, the trick is
to use a small bash init container.
Signed-off-by: Sébastien Han <seb@redhat.com>
The osd prepare resources were only being set on the main container
while the init containers either did not have resources set, or were
inheriting the resource settings from the device set resources for osds.
Now the osd prepare resources will be consistently set based on the
prepareosd resources in the main section of the cluster CR.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This is simply a code simplification to remove an extra check for
whether the OSDs are encrypted. There is no change in functionality.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd prepare jobs and osd daemons may have different placement
requirements. In particular, the topology spread constraints
or pod anti-affinity should only work on either the prepare job or
osd daemon, but does not work effectively on both types at the same
time. Therefore, we split out the osd prepare placement to be separate
from the osd daemon placement. If the osd prepare placement is not set
the operator will fall back to the osd daemon placement for
backward compatibility.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a node is gone, we try to remove crash collector deployments so they
don't stay stale or pending. We must ensure the node is gone and not
just return an error even if we successfully deployment the orphan
deployment.
Signed-off-by: Sébastien Han <seb@redhat.com>
Now the metadata pvc will also be encrypted if the storageClassDeviceSet
has the "encrypted" flag turned on.
Signed-off-by: Sébastien Han <seb@redhat.com>
When a mon is failed over, the new mon is started before the old mon
is removed from quorum and purged from the cluster. If the mon has
anti-affinity, in small clusters or specific toplogies where sufficient
nodes are not available, the new mon may be prevented from starting simply
because the dead mon pod still exists. To ensure a clean failover, the
dead mon pod must first be stopped before the new mon is started.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For external cluster, it is not an issue to have a mon count set to 0
since Rook is not deploying/managing the monitors.
Signed-off-by: Sébastien Han <seb@redhat.com>
Sébastien Han
Travis Nielsen
subhamkrai
Varsha Rao
fanjunwei
Raghavendra TalurandNitin Goyal
Alexander Trost
Madhu Rajanna