The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for cassandra for full fidelity of all the
settings.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph provider had been refactored to read the crds
and operator manifests from a file instead of copying the
manifests into the test code. Now the helpers are refactored
to allow the other providers to also reduce the manifest
duplication in tests.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Operator can run on a namespace not named rook-ceph
The ClusterRoleBinding muste rely on .Release.Namespace helm value
Also move the ClusterRole and ClusterRoleBinding in appropriate yaml file
Signed-off-by: n.fraison <n.fraison@criteo.com>
When there are multiple mgr daemons, the mgr sidecar owns reconciling
the services for the active mgr. However, for efficiency the sidecar
only reconciles the services if the active mgr changes. Now the operator
will also reconcile the mgr services to ensure that services are re-created
after being deleted, or otherwise in an incorrect state.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The stretch clusters pass a new parameter to the mon daemons which is
only available in v16.2.5 and newer. Older versions of Ceph will fail
to run with stretch clusters in rook.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We now pass the networking spec to the clusterInfo so that the executor
can make the right decision on how to execute a command.
This is a small change that allows us to remove the CephBlockPool CR
since it's checking for rbd images. On a Multus deployment, the operator
does not have the network annotations, thus has no access to the OSD
network then rbd commands are hanging forever.
Signed-off-by: Sébastien Han <seb@redhat.com>
The mon daemon in a stretch cluster now can have its location set
as a CLI param instead of setting it with a separate command.
This enables mon failover to set the location of a mon immediately
when it is joining quorum instead of having a delayed command
to set the location.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
when osd pods removed, they are not able
to add back due to missing `ceph auth`
or different `ceph auth`.
this commit mounts the secret `rook-ceph-admin-keyring`
inside the osd activate initcontainer which has
admin keyring.
Signed-off-by: subhamkrai <srai@redhat.com>
The lib-bucket-provisioner was missing the cephCluster spec so the check
in RunAdminCommandNoMultisite would fail.
This is similar to ad54ed8aac just a
different place.
Signed-off-by: Sébastien Han <seb@redhat.com>
The CephClusterSpec was missing from the object context, so the check
for the network provider in RunAdminCommandNoMultisite() was not
discovering the network property correctly.
Signed-off-by: Sébastien Han <seb@redhat.com>
The Rados Gateway Admin OPS API has changed its behavior from Nautilus
to Pacific. Calling user create on an existing user wil generate
additional keys to the user on Nautilus. Where in Pacific it will report
an error with UserAlreadyExists.
So to handle both scenarios, let's first get the user, and if the user
does not exist (NoSuchUser) we then create it.
Signed-off-by: Sébastien Han <seb@redhat.com>
By default, keys stored in Vault KV Secret Engine are versioned and keys
are soft-deleted, see Vault documentation:
When deleting data the standard vault kv delete command will perform a soft
delete.
It will mark the version as deleted and populate a deletion_time timestamp.
Soft deletes do not remove the underlying version data from storage,
which allows the version to be undeleted.
The vault kv undelete command handles undeleting versions. A version's
data is permanently deleted only when the key has more versions than are
allowed by the max-versions setting, or when using vault kv destroy.
When the destroy command is used the underlying version data will be
removed and the key metadata will be marked as destroyed.
If a version is cleaned up by going over max-versions the version
metadata will also be removed from the key.
Signed-off-by: Sébastien Han <seb@redhat.com>
Prior to that the access and secret keys were left empty if the user
already existed, which led to updating the secret with empty values when
the operator restarts.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the CephCluster is configured with Multus and multiple networks are
used to deploy Ceph some commands are failing to be executed from the
Operator. These commands, in particular, `radosgw-admin` ones need access
to the "ceph public network" to talk to OSDs. Unfortunately, the
Rook-Ceph Operator does not have the network annotations and thus
doesn't have the networks available and cannot reach OSDs. So the commands end
up hanging and eventually time out.
Applying the annotations to the Operator pod is possible but will result
in restarting the operator too and this should be avoided at all costs.
Also, applying the annotations beforehand is not possible since the
Multus declaration is in the CephCluster specification. So we would have
no idea what to do.
So the current approach runs a new sidecar container in the mgr pod to
act as a proxy for "some" ceph commands, only the `radosgw-admin` ones
for multi-site setup. This is a small container with admin access
running idle waiting for commands to be executed. In a sense, it is
similar to the toolbox but we didn't want to clearly expose it, so
running as a sidecar is quite nice.
Proxying command is obviously not always recommended since we add an
extra hop in the network path. Now each request has to go from the
operator pod to the API server to the remote pod to Ceph. Previously,
the command only goes from the operator to Ceph.
It's worth noting that external mode is not impacted since no rgw pod
is configured. This scenario is flexible and allows us to scale
pretty well since any CephCluster with Multus will see its mgr sidecar
deployed and can then talk to Ceph. We are not limited.
Signed-off-by: Sébastien Han <seb@redhat.com>
It's better to create PDB of RGW if ObjectStore's `instances` field is 2.
In the current implementation, at least three desired RGW instances are needed
to create this. If `instances` is 1, we can still safely skip the creation
of PDB because it's useless.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
ci is complaining due to empty field in
values.yaml with error
```
coalesce.go:199: warning: destination for config is a table. Ignoring non-table value <nil>
```
this commit make those field empty map/list.
Closes: https://github.com/rook/rook/issues/8239
Signed-off-by: subhamkrai <srai@redhat.com>
The adminops client is already created in newBucketChecker(), use that
instance in checkObjectStoreHealth() than creating a new one.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Travis Nielsen
Sébastien Han
n.fraison
subhamkrai
Satoru Takeuchi
Jiffin Tony Thottan