Commit Graph
23 Commits
Author SHA1 Message Date
Sébastien Han e6e9c8c901 ceph: do not print pointer memory address
When printing pointers of struct (regardless if it is also a slice or
not) we must implement a stringer interface so they can be displayed
properly, this is especially useful when looking at the prepare job
logs.

Before this patch:

```
2021-02-10 06:39:06.783508 D | inventory: discovered disks are [0xc00029c240 0xc00024a000 0xc00086c480 0xc00024a480]
```

After this patch:

```
2021-02-11 13:05:15.730500 D | inventory: discovered disks are:
2021-02-11 13:05:15.730636 D | inventory: &{Name:dm-0 Parent: HasChildren:false DevLinks:/dev/disk/by-id/dm-uuid-LVM-aIFRq5P3hwE2YhukbOmsEZoLY61HxvX62A1xxb6ajTymVE0N8S4Ga0KWlHSgpvNw /dev/disk/by-id/dm-name-ceph--e29bf82e--86f1--4c8e--8412--c468def5a844-osd--block--2044a1ea--6e5e--4423--88cf--8947bf6957d1 Size:32208060416 UUID:91a852c2-8727-4277-9d92-879ee1996503 Serial: Type:lvm Rotational:true Readonly:false Partitions:[] Filesystem:ceph_bluestore Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/mapper/ceph--e29bf82e--86f1--4c8e--8412--c468def5a844-osd--block--2044a1ea--6e5e--4423--88cf--8947bf6957d1 KernelName:dm-0 Encrypted:false}
2021-02-11 13:05:15.730657 D | inventory: &{Name:dm-1 Parent: HasChildren:false DevLinks:/dev/disk/by-id/dm-uuid-LVM-GyLVW5UxDRcws8PwsbQuY2CjrQwdn1VxgPQjsKZ2pGVaS1r2Ct0OGZ4P80yEMtqW /dev/disk/by-id/dm-name-ceph--334c640f--7a7f--4fdb--9db4--2f9459e19f9a-osd--block--66ce11c8--20f0--4600--ae50--e8849c9a0af6 Size:32208060416 UUID:cdbcb218-1b03-4d8c-b7d2-25cb88e33b6a Serial: Type:lvm Rotational:true Readonly:false Partitions:[] Filesystem:ceph_bluestore Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/mapper/ceph--334c640f--7a7f--4fdb--9db4--2f9459e19f9a-osd--block--66ce11c8--20f0--4600--ae50--e8849c9a0af6 KernelName:dm-1 Encrypted:false}
2021-02-11 13:05:15.730686 D | inventory: &{Name:dm-2 Parent: HasChildren:false DevLinks:/dev/disk/by-id/dm-name-ceph--a011b5c2--cc3f--497a--b19a--6d284461cbc4-osd--block--4de5491c--ecf8--455d--9858--547001fd0321 /dev/disk/by-id/dm-uuid-LVM-QGzgO4E228fvT3NsighNABGdvAKxQEhFOh9rKiDLnSOTk7jLZWu2r1RIMXe5GNqW Size:32208060416 UUID:2238ce24-79e4-41a0-b8ce-e381ac41d5b0 Serial: Type:lvm Rotational:true Readonly:false Partitions:[] Filesystem:ceph_bluestore Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/mapper/ceph--a011b5c2--cc3f--497a--b19a--6d284461cbc4-osd--block--4de5491c--ecf8--455d--9858--547001fd0321 KernelName:dm-2 Encrypted:false}
2021-02-11 13:05:15.730713 D | inventory: &{Name:vda1 Parent:vda HasChildren:false DevLinks:/dev/disk/by-path/pci-0000:00:05.0-part1 /dev/disk/by-path/virtio-pci-0000:00:05.0-part1 /dev/disk/by-uuid/0fdf5e49-6b3c-42f2-a1c5-bf77e490dfa4 /dev/disk/by-partuuid/f9dd32c9-a4a5-654f-aee4-651276fdc754 /dev/disk/by-label/boot2docker-data Size:19998934528 UUID: Serial: Type:part Rotational:true Readonly:false Partitions:[] Filesystem:ext4 Vendor: Model: WWN: WWNVendorExtension: Empty:false CephVolumeData: RealPath:/dev/vda1 KernelName:vda1 Encrypted:false}
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-11 14:06:42 +01:00
Satoru Takeuchi 88f4b9c148 ceph: support crypt type device in OSD on PVC
Ceph supported encrypted OSD in the following two ways.

- Encrypt OSD by Ceph itself
- Encrypt OSD by user

However, Rook supports only the first way. Let's support this way too.

With supporting this way, Rook can use variety of encryption methods.
For example, TPM can be used to manage encrypt key. In fact, I use TPM
as follows.

https://github.com/cybozu-go/sabakan/blob/57ff3ab560acb99d23aa8901c1018bf865f9ff4d/docs/disk_encryption.md#disk-encryption

Signed-off-by: UMEZAWA Takeshi <takeshi-umezawa@cybozu.co.jp>
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-08-03 21:17:06 +00:00
Travis Nielsen 32730123ea ceph: add support for multipath devices
Add mpath to the list of supported device types,
although it will only work for OSDs on PVCs.
OSDs on a raw device without PVCs have not
yet been tested.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-05-06 17:12:25 -06:00
Dmitry Yusupov e88704120b edgefs: raw disk empty check regression fix
Signed-off-by: Dmitry Yusupov <dmitry.yusupov@nexenta.com>
2020-04-29 11:48:09 -07:00
Satoru Takeuchi afb6dbe1ed ceph: fix a wrong comment and a warning message about populating device info
PopulateDeviceInfo may fail for reasons other than lsblk failure.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-24 13:00:15 +00:00
morimoto-cybozu ab83738aac ceph: fix device path passed to "ceph-volume inventory"
This commit fixes the argument for "ceph-volume inventory".
When a device "/dev/mapper/foo" is being checked for its availability,
the argument should not be "/dev/foo" nor "/dev/dm-1".

Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2020-03-27 11:24:47 +00:00
Sébastien Han 227d2d527a ceph: osd store refactor
Multiple things:

1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.

2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.

3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.

Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-01-23 19:13:09 +01:00
Sébastien Han dd659de46f ceph: add partition support
We now support partitions via 2 ways:

* if `useAllDevice: true`: partitions will be taken into account and
presented as OSD candidate
* if specified in the cluster CR: it'll picked up as well

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-01-16 09:34:46 +01:00
dulltzandSatoru Takeuchi dfe45ac6b0 ceph: support OSD on PVC backed by LV
"OSD on PVC" doesn't work for PV backed by LV. Fixing this problem
by the following changes.

- Rook accepts LVM disk type.
- If a LV-backed device is passed, Rook/Ceph invokes
  "ceph-volume lvm prepare" with "--data vg/lv"
  instead of "--data /path/to/device".
- If a LV-backed device is passed, Rook/Ceph suppresses
  activation/deactivation of VG that owns this LV.

Fixes: https://github.com/rook/rook/issues/4185
Signed-off-by: dulltz <isrgnoe@gmail.com>
Co-authored-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2019-12-05 17:26:27 +00:00
Juan Miguel Olmo Martínez 7c942604f6 ceph: Get <ceph-volume inventory> data in dev. configmaps
**Description of your changes:**
This modification adds the information extracted from 'ceph-volume inventory':
command to the device configmaps generated by the discovery daemon when
"rook discover" starts with the new boolean "--use-ceph-volume" parameter.

Resolves #
https://github.com/rook/rook/issues/2606

Now the <cephVolumeData> field contains all the information returned
from <ceph-volume inventory> command.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2019-11-06 10:22:56 +01:00
morimoto-cybozu 815927d943 clusterd: return explicit errors
PopulateDeviceInfo() in pkg/clusterd/disk.go returns nil as *sys.LocalDisk
if an error has occurred.  This causes a nil pointer exception at
getAvailableDevices() in pkg/daemon/ceph/osd/daemon.go.  At least the returned
value should be checked at the caller.
I added an explicit error to the return value of PopulateDeviceInfo().  This
naturally revokes an error check at the caller.
I modified PopulateDeviceUdevInfo() in the same file too.

Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2019-10-15 08:30:45 +00:00
rohan47andAshish Ranjan d2f52aebe5 Adds support for storageClassDeviceSet in rook-ceph operator
- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.

Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2019-08-12 09:24:13 -06:00
Sébastien Han 928c4a01da Merge pull request #3418 from InfuseAI/feature/add-linear-type-to-supported-disks
Add linear to support diskType
2019-07-11 17:35:46 +02:00
Ash Wu e6d43d9d0b Add linear to support diskType
Linear raid can be created using `mdadm --create --level=linear`,

By creating a linear raid disk on top of a logical volume,
we can pass the linear device to `ceph-volume lvm batch --prepare`
to provision a new OSD on top of the logical volume since
`ceph-volume` does not take lv as the data device.

Signed-off-by: Ash Wu <hSATAC@gmail.com>
2019-07-10 11:14:26 +08:00
Ash Wu b0eb246919 Remove unused clusterd.GetAvailableDevices
clusterd.GetAvailableDevices is not used anywhere.

Signed-off-by: Ash Wu <hSATAC@gmail.com>
2019-07-09 17:16:37 +08:00
travisn f7c266f7ec skip sgdisk detection if not found in nautilus image
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-06 21:28:57 -07:00
Huamin Chen 82425aafcc prepare osd in a job per node. Once all osds are prepared, store osd info in orchestration configmap.
Operator watches the configmap, starts one osd replica set per osd.

Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-07-05 16:02:27 -06:00
travisn d5d3540497 osd: detect disk uuid with sgdisk
Signed-off-by: travisn <tnielsen@redhat.com>
2018-06-19 13:05:51 -07:00
travisn 4078b1b7f9 osd: device discovery to load part uuid after restart
Signed-off-by: travisn <tnielsen@redhat.com>
2018-06-01 21:03:03 -07:00
Huamin Chen 6c11ff52d4 add device discovery daemon to operator:
run "rook discover" on storage nodes and discover devices on each node. The discovered disks are saved in a per node configmap, local-device-nodename.
Device information consits of name and persistent names, uuid, partition, filesystem, rotational, readonly, size, etc.

Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-05-07 17:49:02 +00:00
David González Ruiz eb59858124 Recognize crypt type as avaiable type and test the feature
Signed-off-by: David González Ruiz <godboole@gmail.com>
2018-02-01 15:13:31 +01:00
David González Ruiz 093d881b9f [OSD] Add crypt type to the array of supported block devices
[skip ci]

Signed-off-by: David González Ruiz <godboole@gmail.com>
2018-01-31 17:41:22 +01:00
Travis Nielsen 4e6774c063 remove legacy standalone code 2017-10-17 12:23:57 -07:00