Commit Graph
33 Commits
Author SHA1 Message Date
shenjiatong 8d3d460e70 ceph: refactor and remove duplicate codes
Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
2020-12-18 18:01:30 +08:00
shenjiatong 04aabb3896 ceph: analyze result for new typed return from lvm batch
Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
2020-12-18 11:11:03 +08:00
Sébastien Han 5d9612c2ae ceph: fix metadata device passed by-id
The code was assuming that devices were passed by the user as
"/dev/sda", this is bad! We all know people should be using paths like
/dev/disk/by-id so we must support them.

Closes: https://github.com/rook/rook/issues/6685
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-25 15:00:47 +01:00
Sébastien Han 6b211eacba ceph: add unit test for osd batch mode
We now have much better coverage on the c-v on non-pvc scenario.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-05 18:06:11 +01:00
Daichi Sakaue b9f29349ed ceph: use raw mode instead of lvm mode even if lv-backed PVC
It can simplify OSD management in many ways and can avoid many problems
that come from LVM.

Closes: https://github.com/rook/rook/issues/5939
Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-14 01:46:25 +00:00
subhamkrai 829778f251 ceph: handle golangci-lint linter ineffassign
this commit will enable one more linter ineffassign
in golangci-lint.

This linter throws an error when variable is assigned and never used.

`golangci-lint run --disable-all -E ineffassign` is used detects ineffassign
errors only.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-09-21 14:31:18 +05:30
Sébastien Han adab0e07e5 Merge pull request #6059 from leseb/lvm-list
ceph: improve callCephVolume() for list and prepare
2020-08-18 13:30:49 +02:00
Sébastien Han dbac1c11d0 ceph: improve callCephVolume() for list and prepare
callCephVolume() was using our exec interface with CombinedOutput which
means return both stdout and stderr. This is fine when preparing OSDs
since we read back the stdout and also want any errors if any (stderr).
However when listing OSD we only want the output since ceph-volume tends
to mix stdout and stderr in the output and does not always silence the
warning types/harmless errors. If the listing has a problem we report it
by printing the ceph-volume log file.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-17 12:43:43 +02:00
Sébastien Han 4f9435d226 ceph: add support for wal encrypted device on pvc
Similar to https://github.com/rook/rook/pull/5977 but for wal devices.
So if a cluster is deployed on PVC and wal devices, those will be
encrypted as well.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-13 15:42:40 +02:00
Sébastien Han 9738c2ba21 ceph: use callCephVolume() to list osds
Small refactor to use the common function helper callCephVolume() to
build ceph-volume arguments.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-12 22:24:14 +02:00
Sébastien Han a91c96cb47 ceph: add support for encrypted metadata pvc
Now the metadata pvc will also be encrypted if the storageClassDeviceSet
has the "encrypted" flag turned on.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-12 22:24:11 +02:00
Chen, Tingjie 0e87348093 ceph: add support for wal device for OSD on PVC
Based on the feature of metadata PVC for OSD: #4818, add the wal device.

For this, you need to create a new volumeClaimTemplates, its name must be "wal", if "wal" is
set, "metadata" device will refers specially for block.db device, and "wal" refers for block.wal
device in BlueStore.

A template will look like this:

```
volumeClaimTemplates:
- metadata:
    name: data
  spec:
    resources:
      requests:
        storage: 10Gi
    # IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
    storageClassName: gp2
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
- metadata:
    name: metadata
  spec:
    resources:
      requests:
        storage: 6Gi
    # IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
    storageClassName: io1
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
- metadata:
    name: wal
  spec:
    resources:
      requests:
        storage: 6Gi
    # IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
    storageClassName: io1
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
```

We now map block, block.db and block.wal directly inside the container instead of
running ceph-volume activate.

Signed-off-by: Chen, Tingjie <tingjie.chen@intel.com>
2020-08-10 08:48:20 +08:00
Sébastien Han 89b3225441 ceph: add encryption support for osd pvc
We can now encrypted OSD device that were provisioned via a storage
class using the PV interface.
The encryption works at the storageClassDeviceSets level, which means we
can have encrypted and non-encrypted sets.
Using the new key `encrypted` we can turn it on.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-05 10:06:11 +02:00
Travis Nielsen e74c7eaef8 ceph: refactor context and clusterInfo passed to the ceph commands
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.

Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-16 15:54:54 -06:00
Travis Nielsen 4681f9e73d ceph: consolidate ceph config and client packages
The ceph config and client packages are conceptually the same.
To avoid circular dependencies in some cases, we simplify by combining
the packages into the client package.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-16 15:54:43 -06:00
subhamkrai 4378a30c64 ceph: updated initializeBlockPVC method and added unit tests
Updated initializeBlockPVC method to add
crushDeviceClass along with
immediateExecuteArgs instead of
metadataArg. Also added unit tests for this method.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-05-29 12:30:19 +05:30
Sébastien Han c799e11032 ceph: enhancement cleanup with disks
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisks which holds more details:

* confirmation: the confirmation message to sanitize disks,
use "yes-really-sanitize-disks" to confirm

This will **only** wipe the metadata, so it's a fast cleanup allowing
you to re-install later but won't remove all the data from the drive.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-05-27 18:14:02 +02:00
Travis Nielsen e8f9cfcb71 exec: always write commands to debug log
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-19 07:49:54 -06:00
Travis Nielsen f2ecaa2bda exec: remove the unused actionName param
The helpers for executing a process have long required an actionName
param which is not being used. Now we remove the old param
while also cleaning up various other usages of the exec
package.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-19 07:49:53 -06:00
Sébastien Han c08c3ced05 ceph: add support for metadata PVC for OSD on PVC
We now support the addition of the PVC that acts as a metadata device
for a given OSD.
For this, you need to create a new `volumeClaimTemplates`, its name must
be "metadata" otherwise, Rook won't pick it up.

A template will look like this:

```
volumeClaimTemplates:
- metadata:
    name: data
  spec:
    resources:
      requests:
        storage: 10Gi
    # IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
    storageClassName: gp2
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
- metadata:
    name: metadata
  spec:
    resources:
      requests:
        storage: 6Gi
    # IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
    storageClassName: gp2
    volumeMode: Block
    accessModes:
      - ReadWriteOnce
```

We now map block and block.db directly inside the container instead of
running ceph-volume activate. This is much cleaner.

Closes: https://github.com/rook/rook/issues/3852
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-02-11 17:30:02 +01:00
Sébastien Han 227d2d527a ceph: osd store refactor
Multiple things:

1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.

2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.

3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.

Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-01-23 19:13:09 +01:00
Sébastien Han 35ee1af42a ceph: expose c-v log per pvc
We now use a defined directory for each prepare job so that logs won't
be entangled with each other.
Also, if an error occurs, we print the log file too so we don't have to
ssh onto a host to find the c-v debug log.

Closes: https://github.com/rook/rook/issues/3888
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-01-16 10:16:11 +01:00
Sébastien Han 5ce2ed220e ceph: use "github.com/pkg/errors"
We now use the error package.
Kubernetes errors have been renamed kerrors since they are lower than
'errors'.

Closes: https://github.com/rook/rook/issues/4054
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-12-09 16:58:32 +01:00
dulltzandSatoru Takeuchi dfe45ac6b0 ceph: support OSD on PVC backed by LV
"OSD on PVC" doesn't work for PV backed by LV. Fixing this problem
by the following changes.

- Rook accepts LVM disk type.
- If a LV-backed device is passed, Rook/Ceph invokes
  "ceph-volume lvm prepare" with "--data vg/lv"
  instead of "--data /path/to/device".
- If a LV-backed device is passed, Rook/Ceph suppresses
  activation/deactivation of VG that owns this LV.

Fixes: https://github.com/rook/rook/issues/4185
Signed-off-by: dulltz <isrgnoe@gmail.com>
Co-authored-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2019-12-05 17:26:27 +00:00
Sébastien Han 76c33e0f85 Revert "ceph: osd hide 'restorecon' when necessary"
This reverts commit d224fbdef1.
Since https://github.com/ceph/ceph/pull/31421, the monkey patching of
'restorecon' is not needed anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-11-12 15:34:12 +01:00
Sébastien Han d224fbdef1 ceph: osd hide 'restorecon' when necessary
Some testing on OpenShift has shown that enabling Hostnetworking causes
issues with ceph-volume creating OSD.
This is a pure Red Hat OpenShift downstream issue.

This is meant to be temporary while we are waiting for the Selinux team
to get back to us.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-30 11:32:15 +01:00
Santosh Pillai 8ea693a740 ceph: fix operator reconcile to restart osds daemons
OSD on PVC does not upgrade when the user upgrades the ceph version on cluster-on-pvc yaml.
The solution incudes:
   - upgrade osd prepare and daemon pods on upgrade
   - skip c-v prepare if filesystem is already present on the pvc device.
   - skip lvm release in case of upgrade.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2019-10-23 16:56:15 +05:30
Blaine Gardner 704786f273 Ceph: use combined output from ceph-volume
ceph-volume now sends status and log output to stderr, which breaks
`ceph-volume lvm prepare` if not other things as well. Use the combined
stdout and stderr output from ceph-volume to fix this.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-09-09 10:29:51 -06:00
Michael Vollman 496c638306 ceph: add db-devices flag and fix DatabaseSizeMB
Adding support for the db-devices flag for ceph versions 14.2.1 and
newer.  This flag will allow for explictly setting the db device for
bluestore configurations to the device specified as metadataDevice.

Fixing the databaseSizeMB config parameter to ensure it is used to set
the size of the db-device partition when specified at either the global
or individual device level. (#3652)

Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
2019-08-29 20:39:41 -04:00
rohan47andAshish Ranjan d2f52aebe5 Adds support for storageClassDeviceSet in rook-ceph operator
- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.

Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2019-08-12 09:24:13 -06:00
Santosh Pillai ec2e6ea888 use ceph fsid to filter ceph volume OSDs
- Updated getCephVolumeOSDs method to use fsid filter while retriving devices.
- This solves "failed to fetch mon config (--no-mon-config to skip)" error when multiple ceph clusters are running on same Node

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2019-06-05 12:05:56 +05:30
travisn bdc3cf8146 osd: fix the device filter and improve device provisioning reliability
All devices detected by the discovery pod were being passed to the OSD provisioning pod
thus not always honoring the desired device list that should be provisioned.
Now the provisioning pod will be given the desired state from the crd,
then apply that state depending on the actual devices detected.
Also added a helper to ensure OSDsPerDevice is always valid.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-02-26 16:11:25 -07:00
travisn 78b8011226 osds: prototype provisioning with ceph-volume
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-06 21:28:58 -07:00