This implements the "Ceph Config via Ceph Cluster CRD" design document
as a `cephConfig:` structure on the CRD.
This also fixes the `yq` commands used to manipulate the
`cluster-test.yaml` that caused CI issues for this PR and potentially
unknowingly others.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
github ci runner is allocating both 14 and 64 Gb disks to the setup.
Rook CI currently hard codes the lsblk with a 14G filter.
This PR updates the filter to use either 14G or 64G disks.
Signed-off-by: sp98 <sapillai@redhat.com>
The issues are fixed from the clean disk action
repo and we shouldn't require any workaround. Also,
I'm not using git revert to revert the commit is earlier
clean disk action was mentioned in every CI which was
duplicate and now we have placed the action to composite yaml
Signed-off-by: subhamkrai <srai@redhat.com>
Since `google-cloud-sdk` is renamed with `google-cloud-cli`,
the github action is failing to remove the older name and this
is causing ci issues. Applying changes suggested by community to
manually remove some packages to resolve the issue.
Signed-off-by: subhamkrai <srai@redhat.com>
Update the multus canary test to reflect modern knowledge about how it
should be configured.
No longer test for the network device in OSD pods. Pods will utterly
fail to start if Multus is unable to attach interfaces.
Instead, look to the OSD map to test the connections more wholistically.
OSDs must have map IPs that include both public and cluster network.
This implicitly tests that the interfaces exist in the Pod, and it
additionally verifies other details, like Ceph `*_network` configs are
set propertly.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
using `rook/ceph:master` tags take longer time
to pull and also, we should be using `rook/ceph:local-build`
tag for our ci. This will help canary raw test to be more stable.
Signed-off-by: subhamkrai <srai@redhat.com>
add a new toolbox yaml manifest which will use the
rook image instead of ceph image
for running s5 cmd container needs to run with rook image
closes: https://github.com/rook/rook/issues/12227
Signed-off-by: parth-gr <paarora@redhat.com>
Let's use same ceph version(latest Reef) in both
cluster-test and toolbox.yaml so that we don't need
to pull image twice. Alos, github action helper script
was calling `deploy_manifest_with_local_build` which
is required for operator.yaml and not for toolbox.yaml.
Signed-off-by: subhamkrai <srai@redhat.com>
Adding CephCOSIDriver CRD and controller. The controller will bring up
the ceph cosi driver when first object store is created in the rook
operator namespace. Then admin can defined COSI CRDs like BucketClass
and BucketAccessClass for different object stores deployed via Rook.
Using the BucketClass and BucketAccessClass, user can define
BucketAccess for backend bucket in the RGW. The CephCOSIDriver CRD
defines configuration options for ceph cosi driver. In the first version
its usability is minimal. Even if it is not defined Rook will bring up
the ceph cosi driver with default values.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
currently we just print the error is anything fails in
script for rgw, So by that there is no panic or
failiure of script if something wrong happened,
Added Explicittly forcing to catch the error from
the error message that is thrown
Closes: https://github.com/rook/rook/issues/12244
Signed-off-by: parth-gr <paarora@redhat.com>
The canary test is waiting for the prometheus module,
which is now disabled by default. For the canary test,
we need to enable the prometheus module for the external
cluster test.
Signed-off-by: travisn <tnielsen@redhat.com>
this commit bump minmum k8s version to 1.21.14 and
max k8s version to latest 1.26.0. Keeping support for
most recent 6 versions.
Signed-off-by: subhamkrai <srai@redhat.com>
The PSPs have long since been deprected. In K8s 1.21 the PSPs
were first deprecated, and support was completely removed
for them in 1.25. With Rook v1.11, the min supported version of
K8s is now 1.21. To reduce confusion in the documentation,
mention of the PSPs is now removed from the 1.11 docs.
For the corner case that users still require the PSPs,
the helm chart still contains the option for creating PSPs
or other users can still create the psp.yaml.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
A new variable is added to rook-ceph-operator-config
ConfigMap to allow using loop devices for osd.
This feature is intended to be used for testing purposes only.
Signed-off-by: Shinya Hayashi <shinya-hayashi@cybozu.co.jp>
Run the ganesha-rados-grace command in a remote pod when multus
networking is enabled.
Signed-off-by: parth-gr <paarora@redhat.com>
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The build process sometimes fails with intermittent problems. Some of them
are known problems and then we retry build process. However, there still
are unknown problems. In this case, it's hard to find the reason because
the build process exits immediately.
ref.
https://github.com/rook/rook/runs/8225144002?check_suite_focus=true#step:3:926
```
+ case "$o" in
+ exit 1
Error: Process completed with exit code 1.
```
To make debugging easier, let's print the output of `make`. This log won't be
too long since `make` prints most messages to stderr.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The whereabouts manifests in the master branch
have moved around, so until the new approach
is investigated we pin to the latest release
version v0.5.3
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rook supports raw mode OSD in host-based cluster. So we can also
support OSD on logical volume in this kind of cluster.
Logical volumes aren't picked by filters (i.e. `useAllDevices: true`
and `device{Path,}Filter` to avoid unwanted LV consumption on upgrade.
Closes: https://github.com/rook/rook/issues/2047
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
It's better to test the creation of encrypted devices in host-based clusters.
This test will reduce the potential risks of regression when modifying
the osd-creation code.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
If csi is configured not the use the
hostnetworking, deploy the holder
pod for executing the commands with nsenter.
The implementation is same as multus.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
OSD on PVC is tested in many patterns but OSD on device is not.
It's preferable to add the following patterns for OSD on device.
- An OSD on a raw disk
- Two OSDs on a device
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The following steps in CI scripts create corrupted GPT headers.
```console
$ sudo sgdisk --zap-all --clear --mbrtogpt -g -- "$DISK"
$ sudo dd if=/dev/zero of="$DISK" bs=1M count=10
```
It results in the failure of the succeeding `sgdisk --print "$DISK".
Here is an example.
```console
$ sudo parted /dev/sdb mklabel msdos
...
$ sudo sgdisk --zap-all --clear --mbrtogpt -g -- /dev/sdb
...
$ sudo dd if=/dev/zero of=/dev/sdb bs=1M count=10
...
$ sudo sgdisk --print /dev/sdb
Caution: invalid main GPT header, but valid backup; regenerating main header
from backup!
Warning: Invalid CRC on main header data; loaded backup partition table.
Warning! One or more CRCs don't match. You should repair the disk!
Main header: ERROR
Backup header: OK
Main partition table: OK
Backup partition table: OK
Invalid partition data!
$ echo $?
2
```
We can safely use the simple `sgdisk --zap-all "$DEVICE"` here.
It's not necessary to convert "mbr" to "gpt" because we'll make new GPT
labels just after this command.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Once we are done cleaning up the content of the encrypted osd, let's
close the main LUKS device.
This avoids having dm devices on the system after the cleanup.
Closes: https://github.com/rook/rook/issues/10181
Signed-off-by: Sébastien Han <seb@redhat.com>
This new integration test will deploy a cluster with multus enabled. It
will be comprised of two network interfaces for ceph public and cluster
communications.
For now, it only deploys a Ceph cluster up to the OSDs.
Closes: https://github.com/rook/rook/issues/9784
Signed-off-by: Sébastien Han <seb@redhat.com>
Data inconsistency might happen if the disk is accessed just after disk
zapping because direct I/O is not synchronous by itself.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This introduces a new CRD to add the ability to create subvolumegroup
for a given ceph filesystem volume. Typically the name of the volume is
the name of the filesystem created by rook.
Closes: https://github.com/rook/rook/issues/7036
Signed-off-by: Sébastien Han <seb@redhat.com>
Alexander Trost
sp98
subhamkrai
Blaine Gardner
Travis Nielsen
parth-gr
Jiffin Tony Thottan
Rakshith R
Joshua Hoblitt
Shinya Hayashi
Blaine Gardner
Satoru Takeuchi
Josh Soref
Madhu Rajanna
Sébastien Han
Yuval Manor