If the mon is backed by a pvc, ensure the pvc is deleted when
the mon is failed over. The pvc is being left in a bound state
even though it will no longer be used in the cluster.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During mon failover if the mon fails to be removed, the new mon
config was not being saved. The mon removal anyway is not retried
if it fails, so it should make a best effort to remove all the
resources related to the mon, and then always update the mon
config after the failover.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
adds deploy.sh script to deploy validatingwebhookconfiguration and create secrets.
adds new command ceph admission-controller to start webhook servers.
adds validation for various rook custom resources
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
This commit fixes an argument of the CreateCrashCollectorSecret function as
it is expected to be the namespace of the cluster.
Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
When the operator loads the mons and checks for a valid maxMonId
the check of the id against the currently running mons was outdated
from legacy mon ids. The check now has to parse the new id which is
just the letter instead of having a number from legacy mon ids.
Without this fix, the maxMonId would stay incorrect if it was somehow
corrupted.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
radosgw-admin as of Octopus uses a different time format, it uses
"2006-01-02T15:04:05.999999999Z". It's close from RFC3339 but not quite
the same. This change is needed in order for the operator image (with a
Ceph Octopus based image) to perform radosgw-admin call correctly.
Signed-off-by: Sébastien Han <seb@redhat.com>
We can now explicitly set any property on a given pool by using the new
Property field in the CephBlockPool Spec.
Also, this fixes the case where both `CephBlockPool` and `CephCluster`
are created at the same time. When Rook creates the pool, the cluster is
still being bootstrapped and the global option
`osd_pool_default_pg_autoscale_mode` has not bee set yet. So the pool
gets created but its `pg_autoscale_mode` property is set to `warn`
instead of `on`.
Closes: https://github.com/rook/rook/issues/5608V
Signed-off-by: Sébastien Han <seb@redhat.com>
The volume sources for an OSD on PVC were relying on the ordering
of the volume sources slice. While this was working, it was causing
headaches and potential future developer error when being
interpreted incorrectly. The addition of the wal pvc device
would make the issue even more pronounced. Now each PVC used
for the OSD will be identified in a map by name such as data
and metadata.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When obc creation fails in Grant() or Provision() the resources such as rgw user or bucket
created while provisioning need to remove.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
This commit contains CR implementations for:
CephObjectRealm
CephObjectZoneGroup
CephObjectZone
Also there are changes made to the objectstore
to add rgws in the object-store to zones and
zone groups in a multisite configuration and
the removal of the --default parameter for any
realms/zonegroups/zones that are created.
Signed-off-by: Ali Maredia <amaredia@redhat.com>
When running Octopus, we still need to turn on the balancer module. The
distinction is tricky, the module is "enabled" but it has to be "turned
on" and configured.
This was only working when the `pg_autoscaler` module was explicitly
listed in the CephCluster spec mgr section module section but not
automatically configured when running Octopus (the `pg_autoscaler`
module is already enabled).
Signed-off-by: Sébastien Han <seb@redhat.com>
The mgr crash module is always enabled so we don't need to run the
command again. It saves some orchestration time.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the PVC template increases in size for the storageClassDeviceSet
in the cluster CR, we need to increase the size of the PVC and restart
the OSD. The OSD is restarted by changing a variable on the pod spec
that indicates the desired size of the OSD.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When integrating rook with istio, service port names need to be prefixed by their protocol.
Trying to fix this by removing static port names in favor of configurable names.
Closes: #5587
add protocol prefix to service port names, hardcoded as discussed. renamed container ports
Signed-off-by: Alex Flom <alexander.flom@gmail.com>
When the cluster is loaded and we restart an OSD, it will need some time
to respond to socket calls, basically more to be ready.
Increasing the initialDelaySeconds of the liveness probe fixes that
issue. For OSD, it waits for 45 sec, where other daemons 10 sec.
Closes: https://github.com/rook/rook/issues/5492
Signed-off-by: Sébastien Han <seb@redhat.com>
When the cluster is using HostNetworking, Rook was either picking up the
external or internal IP of the node. This resulted in mon endpoints have
public IP addresses. Those IP are not reachable from within the cluster
so OSD/CSI couldn't access the monitors from the configmap endpoint.
Also, exposing the cluster on a public network does not seem realistic,
so sticky with private/internal IP addresses is better.
Closes: https://github.com/rook/rook/issues/5495
Signed-off-by: Sébastien Han <seb@redhat.com>
If host networking is not enabled add preferred pod anti-affinity on rgw pods to
improve spread of them
Signed-off-by: n.fraison <n.fraison@criteo.com>
Some external consumers of Rook-Ceph are using the
lib-bucket-provisioner operator which creates different CRDs for both
objectbuckets.objectbucket.io and objectbucketclaims.objectbucket.io.
They have the `additionalConfig` declared in their spec. Rook-Ceph does
not and thus ignores it.
For the OB creation to work, the `additionalConfig` must be acknowledged
by the operator and populate or initialize it if empty.
This is preventing the following error:
controller.go:190] error syncing 'default/ceph-retain-bucket-converged': error creating OB "": ObjectBucket.objectbucket.io "obc-default-ceph-retain-bucket-converged" is invalid: spec.endpoint.additionalConfig: Invalid value: "null": spec.endpoint.additionalConfig in body must be of type object: "null", requeuing
Signed-off-by: Sébastien Han <seb@redhat.com>
The csi driver does not need to be started before the cluster
is created. When the csi driver fails to load, neither should
it prevent the cluster from being created. Therefore, we
initialize the basic constructs in the csi driver that are required
for cluster creation, then start the csi driver in a goroutine
so the cluster creation can continue. If the csi driver fails
or takes a long time to load, it will no longer affect cluster
creation.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a mon is stuck in terminating state on a failed node,
force delete the pod so it can attempt to restart. If
the mon is backed by a PV the mon can start on another
node as well.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator should only print helpful info messages
when an OSD is going to be updated. If the OSD hasn't
changed it is just a debug message.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd health check every 5 min is not frequent enough to respond
quickly enough to osds going down. Reducing the time interval
to one minute for faster response time.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The pod deletion is needed by other daemons besides the osds,
so we move the helper method into the k8sutil package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When obc is deleted, the attached ceph user will be also removed
This will cause obc creation failure for the same bucket since user
was missing. With this patch the user will be retained revoking his permissions
Also removed 'GetBucketPolicy' permission from Allowed action. The same set of
permission is used for denying the owner/old user of OBC. So that in the next
obc request for the bucket, owner can set polciy for the new user
Closes: https://github.com/rook/rook/issues/5510
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisks which holds more details:
* confirmation: the confirmation message to sanitize disks,
use "yes-really-sanitize-disks" to confirm
This will **only** wipe the metadata, so it's a fast cleanup allowing
you to re-install later but won't remove all the data from the drive.
Signed-off-by: Sébastien Han <seb@redhat.com>
The crash collector was missing the security context to properly run on
OCP, thus crashing when trying to post a crash dump.
Signed-off-by: Sébastien Han <seb@redhat.com>
As of Nautilus Ceph auto-tunes its osd_memory_target on the fly so we don't need to force it.
ROok does not support Mimic anymore so this can be removed.
Signed-off-by: Sébastien Han <seb@redhat.com>
In 1.3 we added support for the metadata pvcs, thus requiring the name
to be set on the templates to differentiate between data and metadata.
For backward compatibility we need to assume the template is for data
if it is not set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Sébastien Han
Travis Nielsen
Vineet Badrinath
binoue
UMEZAWA Takeshi
Jared Watts
Andreas Feldmann
Jiffin Tony Thottan
Ali Maredia
Alex Flom
Madhu Rajanna
n.fraison