14.2.15 lvm batch command prepare report changes
output format. This commit skips md check if ceph version
is greater than 14.2.13.
Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
The deviceClass property was being ignored when creating the
non-pvc OSDs. Now the deviceClass will be specified as a property
for individual devices, all devices on a node, or all OSDs in the
cluster, depending on the level where the config is applied in the
cluster CR.
Co-authored-by: shenjiatong <yshxxsjt715@gmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The code was assuming that devices were passed by the user as
"/dev/sda", this is bad! We all know people should be using paths like
/dev/disk/by-id so we must support them.
Closes: https://github.com/rook/rook/issues/6685
Signed-off-by: Sébastien Han <seb@redhat.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
When deploying on non-PVC, we still need to gather c-v logs on failures.
Now if the bootstrap fails, the c-v log will be printed.
Signed-off-by: Sébastien Han <seb@redhat.com>
callCephVolume() was using our exec interface with CombinedOutput which
means return both stdout and stderr. This is fine when preparing OSDs
since we read back the stdout and also want any errors if any (stderr).
However when listing OSD we only want the output since ceph-volume tends
to mix stdout and stderr in the output and does not always silence the
warning types/harmless errors. If the listing has a problem we report it
by printing the ceph-volume log file.
Signed-off-by: Sébastien Han <seb@redhat.com>
Now the metadata pvc will also be encrypted if the storageClassDeviceSet
has the "encrypted" flag turned on.
Signed-off-by: Sébastien Han <seb@redhat.com>
Based on the feature of metadata PVC for OSD: #4818, add the wal device.
For this, you need to create a new volumeClaimTemplates, its name must be "wal", if "wal" is
set, "metadata" device will refers specially for block.db device, and "wal" refers for block.wal
device in BlueStore.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
storageClassName: io1
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: wal
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
storageClassName: io1
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block, block.db and block.wal directly inside the container instead of
running ceph-volume activate.
Signed-off-by: Chen, Tingjie <tingjie.chen@intel.com>
We can now encrypted OSD device that were provisioned via a storage
class using the PV interface.
The encryption works at the storageClassDeviceSets level, which means we
can have encrypted and non-encrypted sets.
Using the new key `encrypted` we can turn it on.
Signed-off-by: Sébastien Han <seb@redhat.com>
File permissions need to be reduced to the minimum to avoid config
files from being accessed. While a connection to the pod would be
required to access the files, min privileges are set just as
another layer of security. This fixes gosec error G306.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit fixes where there are areas in which files and directories are
written and created with statically defined permissions. Many of these
permissions are rather open, potentially allowing other system tenants to
view and interact with their contents, which may be sensitive.
Fixed G303: Poor file permissions used for directory by using the most narrow permission
Resoves: https://github.com/rook/rook/issues/4580
Signed-off-by: Nizamudeen <nia@redhat.com>
this commit suppress the gosec errors for
g204: Audit use of command execution.
g304: File path provided as taint input.
g101: Look for hard coded credentials.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
To ensure a file handle is closed, we defer the close command
so it is guaranteed to run when the method returns. The closing
of the file handle is not going to fail in our usage since we aren't
using the SetDeadline on the files that would cancel a request
and return an error.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisk which holds more details:
* method: indicates if the entire disk should be sanitized or simply ceph's metadata.
Possible choices are 'complete' or 'quick' (default)
* dataSource: indicate where to get random bytes from to write on the disk.
Possible choices are 'zero' (default) or 'random'
Using random sources will consume entropy from the system and will take much more time then the zero source
* iteration: overwrite N times instead of the default (1). Takes an integer value
See the documentation for more details.
Signed-off-by: Sébastien Han <seb@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add the ability to provision Ceph OSDs with Drive Groups.
This adds Drive Groups to the CephCluster CRD, and it sets code
in place for propagating this config to the OSD provisioning pod.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Updated initializeBlockPVC method to add
crushDeviceClass along with
immediateExecuteArgs instead of
metadataArg. Also added unit tests for this method.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisks which holds more details:
* confirmation: the confirmation message to sanitize disks,
use "yes-really-sanitize-disks" to confirm
This will **only** wipe the metadata, so it's a fast cleanup allowing
you to re-install later but won't remove all the data from the drive.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the metadataDevice is specified, it was a regression that it was
being interpreted also as a data device. ceph-volume was failing
when the metadataDevice was added as both a data and metadata device.
So we ensure it is not added to the list of data devices.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit adds support for LVs to the device availability check
in the OSD prepare pod.
The availability of an LV is checked by "ceph-volume lvm list".
If it returns non-empty result, the LV is in use and not available.
Closes: https://github.com/rook/rook/issues/5075
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
The helpers for executing a process have long required an actionName
param which is not being used. Now we remove the old param
while also cleaning up various other usages of the exec
package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the OSD on PVC is backed by a metadata block PVC, the Ceph CRUSH
device class should be set to something else rather than the rotational
property of the drive.
Closes: https://github.com/rook/rook/issues/4881
Signed-off-by: Sébastien Han <seb@redhat.com>
We now support the addition of the PVC that acts as a metadata device
for a given OSD.
For this, you need to create a new `volumeClaimTemplates`, its name must
be "metadata" otherwise, Rook won't pick it up.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block and block.db directly inside the container instead of
running ceph-volume activate. This is much cleaner.
Closes: https://github.com/rook/rook/issues/3852
Signed-off-by: Sébastien Han <seb@redhat.com>
If we detect a partition we should not use it if the ceph version is not
at least 14.2.8 since it has the necessary changes to support partitions
with ceph-volume.
Signed-off-by: Sébastien Han <seb@redhat.com>
14.2.7 will only be a CVE fix release so 14.2.8 is the release that will
contain the fixes for ceph-volume partition.
Signed-off-by: Sébastien Han <seb@redhat.com>
Multiple things:
1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.
2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.
3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.
Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
We now use a defined directory for each prepare job so that logs won't
be entangled with each other.
Also, if an error occurs, we print the log file too so we don't have to
ssh onto a host to find the c-v debug log.
Closes: https://github.com/rook/rook/issues/3888
Signed-off-by: Sébastien Han <seb@redhat.com>
When using the "errors" package, using `%+v` (extended format),
each Frame of the error's StackTrace will be printed in detail.
Let's only print `%v` to print the error.
If the error has a Cause it will be printed recursively.
Basically `%+v` has been replaced with `%v` for all `error` type
interfaces, whether the logger is Info, Warning or Error.
Signed-off-by: Sébastien Han <seb@redhat.com>
"OSD on PVC" doesn't work for PV backed by LV. Fixing this problem
by the following changes.
- Rook accepts LVM disk type.
- If a LV-backed device is passed, Rook/Ceph invokes
"ceph-volume lvm prepare" with "--data vg/lv"
instead of "--data /path/to/device".
- If a LV-backed device is passed, Rook/Ceph suppresses
activation/deactivation of VG that owns this LV.
Fixes: https://github.com/rook/rook/issues/4185
Signed-off-by: dulltz <isrgnoe@gmail.com>
Co-authored-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>