The mgr crash module is always enabled so we don't need to run the
command again. It saves some orchestration time.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the PVC template increases in size for the storageClassDeviceSet
in the cluster CR, we need to increase the size of the PVC and restart
the OSD. The OSD is restarted by changing a variable on the pod spec
that indicates the desired size of the OSD.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When integrating rook with istio, service port names need to be prefixed by their protocol.
Trying to fix this by removing static port names in favor of configurable names.
Closes: #5587
add protocol prefix to service port names, hardcoded as discussed. renamed container ports
Signed-off-by: Alex Flom <alexander.flom@gmail.com>
When the cluster is loaded and we restart an OSD, it will need some time
to respond to socket calls, basically more to be ready.
Increasing the initialDelaySeconds of the liveness probe fixes that
issue. For OSD, it waits for 45 sec, where other daemons 10 sec.
Closes: https://github.com/rook/rook/issues/5492
Signed-off-by: Sébastien Han <seb@redhat.com>
If the cluster has 3 OSDs, during the upgrade when the OSD restarts it
will temporarily disappear from the "ceph versions" list so we need a
more robust way to list the number of OSDs and always returns the total
number of OSDs.
Closes: https://github.com/rook/rook/issues/5521
Signed-off-by: Sébastien Han <seb@redhat.com>
Even if we cannot determine wheter all the OSDs are running on the host,
we still need to perform the ok-to-stop check. It is far more common to
have more than 3 OSDs anyways.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the cluster is using HostNetworking, Rook was either picking up the
external or internal IP of the node. This resulted in mon endpoints have
public IP addresses. Those IP are not reachable from within the cluster
so OSD/CSI couldn't access the monitors from the configmap endpoint.
Also, exposing the cluster on a public network does not seem realistic,
so sticky with private/internal IP addresses is better.
Closes: https://github.com/rook/rook/issues/5495
Signed-off-by: Sébastien Han <seb@redhat.com>
If host networking is not enabled add preferred pod anti-affinity on rgw pods to
improve spread of them
Signed-off-by: n.fraison <n.fraison@criteo.com>
Some external consumers of Rook-Ceph are using the
lib-bucket-provisioner operator which creates different CRDs for both
objectbuckets.objectbucket.io and objectbucketclaims.objectbucket.io.
They have the `additionalConfig` declared in their spec. Rook-Ceph does
not and thus ignores it.
For the OB creation to work, the `additionalConfig` must be acknowledged
by the operator and populate or initialize it if empty.
This is preventing the following error:
controller.go:190] error syncing 'default/ceph-retain-bucket-converged': error creating OB "": ObjectBucket.objectbucket.io "obc-default-ceph-retain-bucket-converged" is invalid: spec.endpoint.additionalConfig: Invalid value: "null": spec.endpoint.additionalConfig in body must be of type object: "null", requeuing
Signed-off-by: Sébastien Han <seb@redhat.com>
The csi driver does not need to be started before the cluster
is created. When the csi driver fails to load, neither should
it prevent the cluster from being created. Therefore, we
initialize the basic constructs in the csi driver that are required
for cluster creation, then start the csi driver in a goroutine
so the cluster creation can continue. If the csi driver fails
or takes a long time to load, it will no longer affect cluster
creation.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Updated initializeBlockPVC method to add
crushDeviceClass along with
immediateExecuteArgs instead of
metadataArg. Also added unit tests for this method.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
If a mon is stuck in terminating state on a failed node,
force delete the pod so it can attempt to restart. If
the mon is backed by a PV the mon can start on another
node as well.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator should only print helpful info messages
when an OSD is going to be updated. If the OSD hasn't
changed it is just a debug message.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd health check every 5 min is not frequent enough to respond
quickly enough to osds going down. Reducing the time interval
to one minute for faster response time.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The pod deletion is needed by other daemons besides the osds,
so we move the helper method into the k8sutil package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When obc is deleted, the attached ceph user will be also removed
This will cause obc creation failure for the same bucket since user
was missing. With this patch the user will be retained revoking his permissions
Also removed 'GetBucketPolicy' permission from Allowed action. The same set of
permission is used for denying the owner/old user of OBC. So that in the next
obc request for the bucket, owner can set polciy for the new user
Closes: https://github.com/rook/rook/issues/5510
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisks which holds more details:
* confirmation: the confirmation message to sanitize disks,
use "yes-really-sanitize-disks" to confirm
This will **only** wipe the metadata, so it's a fast cleanup allowing
you to re-install later but won't remove all the data from the drive.
Signed-off-by: Sébastien Han <seb@redhat.com>
The crash collector was missing the security context to properly run on
OCP, thus crashing when trying to post a crash dump.
Signed-off-by: Sébastien Han <seb@redhat.com>
x ClusterSpec
x RackSpec
The lack of the json tag required manifests to use 'Annotations' rather than 'annotations' to propogate annotations to the stateful set pods. This change will allow the software to function as documented.
Closes: https://github.com/rook/rook/issues/5097
Signed-off-by: Ryan Thompson <ryan@zengyant.com>
As of Nautilus Ceph auto-tunes its osd_memory_target on the fly so we don't need to force it.
ROok does not support Mimic anymore so this can be removed.
Signed-off-by: Sébastien Han <seb@redhat.com>
In 1.3 we added support for the metadata pvcs, thus requiring the name
to be set on the templates to differentiate between data and metadata.
For backward compatibility we need to assume the template is for data
if it is not set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During cluster delection, the finalizer will not be removed until
all the PVCs are removed. Since this could wait for a long time,
as long as the deletion event is re-queued, we need to cancel
the goroutine that is watching for cluster deletion so it can
avoid running multiple goroutines as the event runs again
and again.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When a cluster is requested for deletion there is no longer
a reason to monitor the cluster health such as whether mons
are in quorum, osds are up and in, or monitor the ceph status.
The cluster will wait potentially for a long time if the
PVCs are not deleted by the admin, therefore we stop monitoring
the cluster even before the finalizer is removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the admin key is available, set the external creds struct
to use that admin username and key instead of requiring the
callers to check every time if they should use the admin
key or the external key.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The Ceph version information is not needed to be passed
to the osd health checker since mimic support was dropped.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph username is all that needs to be passed to the goroutines
that are monitoring the health of Ceph. This simplifies the
initialization of the goroutines.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The monitoring goroutines that check for mon and osd health,
the ceph cluster status, and the bucket provisioner should
only be started once for each cluster. Previous to this change
and since the controller runtime conversion the goroutines were
being started again every time a reconcile was run. This means
you could potentially have many goroutines monitoring the same
thing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This PR is for rebasing #4683 with master. With new controller runtime changes most the changes with #4683 got redundant.
Only valid change is waiting for all the object buckets to be cleaned up before removing the finalizer.
Co-authored-by: Elise Gafford <egafford@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The operator continues to run even though the controller-runtime
failed to start. Since the controller-runtime is essential, the operator
is non-functional after that. It makes troubleshooting more difficult.
Closes: https://github.com/rook/rook/issues/5434
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
pg_num_min needs to be set on all new and existing rgw
metadata pools so the autoscaler won't immediately
increase the pg count. The pg_num_min was only being set
on existing pools and not on new pools.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During cluster deletion, we currently only retry for a couple minutes
to wait for the pvcs to be deleted. After the timeout, we proceed
with the cluster deletion. To properly protect the pvcs for proper
cleanup, the finalizer should not be removed until the pvcs
are all confirmed to be deleted. In order to not block other cluster
events, we re-queue the deletion event to run again every 10s
until the pvcs are deleted or the finalizer is manually removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The controller setting for requeuing an event moved to the
opcontroller package and was no longer needed in the main
controller package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Sébastien Han
Travis Nielsen
Alex Flom
Madhu Rajanna
n.fraison
subhamkrai
Jiffin Tony Thottan
Ryan Thompson
Santosh PillaiandElise Gafford
Satoru Takeuchi
Michael Fritch