Just like we do for the monitors, we set an ownerref to the OSD PVC so
when the cluster is deleted we remove the PVC too.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
on Azure, all SSDs are reporting as a rotational disk.
This commit will add `tuneFastDeviceClass` flag which will
tune for an SSD even if the cloud provider is reporting
rotational. We have a flag `tuneDeviceClass`
which works vice-versa.
Signed-off-by: subhamkrai <srai@redhat.com>
OSD prepare jobs only need to be run if the OSD deployments on PVCs were
not already started. For efficiency, we now query them in a batch instead
of querying them one-by-one with a label selector, which proves to be very
slow in large clusters.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The OSD PVCs were being detected one by one with a label selector.
This proves to be slow in large clusters. Instead, we now make this
more efficient by querying for all existing PVCs and looking up their
existing in memory with a map.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The pvc template name has either been require to be empty, or one of the
expected values such as data and metadata. Those who upgrade from 1.1
may find that if they changed it to something else, their upgrade will
fail when not one of these values. If there is only a single template,
always treat it as the data template.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator will now allow the resource limits to be applied below the
recommended minimums. In small clusters, even the recommended minimums
may not be necessary. A warning is still printed to the operator log,
but we allow the configuration to continue.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd prepare jobs and osd daemons may have different placement
requirements. In particular, the topology spread constraints
or pod anti-affinity should only work on either the prepare job or
osd daemon, but does not work effectively on both types at the same
time. Therefore, we split out the osd prepare placement to be separate
from the osd daemon placement. If the osd prepare placement is not set
the operator will fall back to the osd daemon placement for
backward compatibility.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We can now encrypted OSD device that were provisioned via a storage
class using the PV interface.
The encryption works at the storageClassDeviceSets level, which means we
can have encrypted and non-encrypted sets.
Using the new key `encrypted` we can turn it on.
Signed-off-by: Sébastien Han <seb@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The volume sources for an OSD on PVC were relying on the ordering
of the volume sources slice. While this was working, it was causing
headaches and potential future developer error when being
interpreted incorrectly. The addition of the wal pvc device
would make the issue even more pronounced. Now each PVC used
for the OSD will be identified in a map by name such as data
and metadata.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the PVC template increases in size for the storageClassDeviceSet
in the cluster CR, we need to increase the size of the PVC and restart
the OSD. The OSD is restarted by changing a variable on the pod spec
that indicates the desired size of the OSD.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In 1.3 we added support for the metadata pvcs, thus requiring the name
to be set on the templates to differentiate between data and metadata.
For backward compatibility we need to assume the template is for data
if it is not set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Now, the CephBlockPool CRD is managed with the controller-runtime.
So the watcher is outside of the main controller reconciliation loop of
CephCluster which brings numerous benefit such as:
* having its own reconciliation loop
* won't block anything from the main CephCluster controller loop
* fast than waiting for CephCluster loop to completion
Partially close: https://github.com/rook/rook/issues/1981
Signed-off-by: Sébastien Han <seb@redhat.com>
The rook types used across the storage providers moved from the v1alpha2
package to the v1 package. This commit points the packages at their new
location. Implementation is expected to remain unchanged.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the OSD on PVC is backed by a metadata block PVC, the Ceph CRUSH
device class should be set to something else rather than the rotational
property of the drive.
Closes: https://github.com/rook/rook/issues/4881
Signed-off-by: Sébastien Han <seb@redhat.com>
We now support the addition of the PVC that acts as a metadata device
for a given OSD.
For this, you need to create a new `volumeClaimTemplates`, its name must
be "metadata" otherwise, Rook won't pick it up.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block and block.db directly inside the container instead of
running ceph-volume activate. This is much cleaner.
Closes: https://github.com/rook/rook/issues/3852
Signed-off-by: Sébastien Han <seb@redhat.com>
TuneSlowDeviceClass wasn't part of the StorageClassDeviceSet types but
only in VolumeSource which is not exposed in the CR spec so setting the
option would never work.
The only issue is that this apply to all volumeClaimTemplates
where each can point to a particular storage class...
Signed-off-by: Sébastien Han <seb@redhat.com>
When using the "errors" package, using `%+v` (extended format),
each Frame of the error's StackTrace will be printed in detail.
Let's only print `%v` to print the error.
If the error has a Cause it will be printed recursively.
Basically `%+v` has been replaced with `%v` for all `error` type
interfaces, whether the logger is Info, Warning or Error.
Signed-off-by: Sébastien Han <seb@redhat.com>
OSD on PVC does not upgrade when the user upgrades the ceph version on cluster-on-pvc yaml.
The solution incudes:
- upgrade osd prepare and daemon pods on upgrade
- skip c-v prepare if filesystem is already present on the pvc device.
- skip lvm release in case of upgrade.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.
Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Arun Kumar Mohan
Sébastien Han
subhamkrai
Travis Nielsen
UMEZAWA Takeshi
Santosh Pillai
rohan47
rohan47andAshish Ranjan