Commit Graph
13292 Commits
Author SHA1 Message Date
Sébastien Han c8ee5674dd Merge pull request #8103 from leseb/rm-jenkins
ci: remove jenkins from master branch
2021-07-23 18:20:10 +02:00
Sébastien Han d55fb8d13b Merge pull request #8351 from leseb/refact-exec-helpers
ceph: remove unnecessary exec helpers
2021-07-23 18:12:57 +02:00
Sébastien Han 0811359d28 Merge pull request #8358 from leseb/move-to-quay
ceph: move all of our docker.io reference to quay.io
2021-07-23 18:03:53 +02:00
Sébastien Han 3d75920bd7 Merge pull request #8353 from leseb/mock-admin-ops-api
ceph: mock object store user create
2021-07-23 17:57:41 +02:00
Sébastien Han daf53ea585 ceph: add unit test for object store user dependents
Now that we have the mock client for the admin ops api we can add unit
tests.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-23 10:01:53 +02:00
Sébastien Han 733c0f72f6 ceph: mock object store user create
Since the mock client was added to the rgw/admin package from the
go-ceph project in https://github.com/ceph/go-ceph/pull/532 we can now
mock the HTTP client and validate the entire reconcile.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-23 10:01:53 +02:00
subhamkrai 492a8176e0 ci: action for release build
this commit adds github action for
release builds and also modify the
helper script `build-release.sh`.

Closes: https://github.com/rook/rook/issues/8041
Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-23 13:09:30 +05:30
Sébastien Han 6d77a9976c ceph: remove unnecessary exec helpers
Both `ExecuteCommandWithOutputFileTimeout()` and
`ExecuteCommandWithOutputFile()` generate unnecessary system calls by
creating/reading/removing files where the stream output of the command
can simply be used. So sticking with `ExecuteCommandWithOutput()` and
`ExecuteCommandWithCombinedOutput()` for reading outputs is sufficient.

Closes: https://github.com/rook/rook/issues/8343
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-23 09:16:33 +02:00
Travis Nielsen 9c06d236b6 Merge pull request #8356 from travisn/cronjob-beta
ceph: Update test to watch for v1 cronjob instead of v1beta1
2021-07-22 12:05:46 -06:00
Travis Nielsen 013e3a22a3 Merge pull request #8359 from leseb/rbd-mirror-code
ceph: remove `removeExtraMirrors()`
2021-07-22 11:34:21 -06:00
Travis Nielsen 16403ac901 Merge pull request #8350 from leseb/fix-7183
ceph: remove operator config `ROOK_ALLOW_MULTIPLE_FILESYSTEMS`
2021-07-22 11:33:40 -06:00
Travis Nielsen 07ddcda0ce ceph: update test to watch for v1 cronjob
The v1beta1 cronjob is deprecated and needs to use v1 for K8s 1.16 and newer

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-22 11:25:55 -06:00
Travis Nielsen fd141af92c Merge pull request #8319 from BlaineEXE/disable-raw-mode-for-disks
ceph: disable raw mode for disks
2021-07-22 11:17:54 -06:00
Blaine Gardner c8a2db368a ceph: disable raw mode for disks
Even if we can use raw mode, do NOT use raw mode on disks. Ceph bluestore disks can
sometimes appear as though they have "phantom" Atari (AHDI) partitions created on them
when they don't in reality. This is due to a series of bugs in the Linux kernel when it
is built with Atari support enabled. This behavior does not appear for raw mode OSDs on
partitions, and we need the raw mode to create partition-based OSDs. We cannot merely
skip creating OSDs on "phantom" partitions due to a bug in `ceph-volume raw inventory`
which reports only the phantom partitions (and malformed OSD info) when they exist and
ignores the original (correct) OSDs created on the raw disk.

Resolves https://github.com/rook/rook/issues/7940

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-07-22 09:59:56 -06:00
Travis Nielsen ab728e0183 Merge pull request #7977 from parth-gr/update-PodDisruptionBudget
ceph: update PodDisruptionBudget from v1beta1 to v1
2021-07-22 09:14:32 -06:00
Travis Nielsen 85f9b57ad6 Merge pull request #8340 from ashangit/master_purge_job_multi_ns
ceph: define specific ServceAccount/RBAC for ceph osd purge job
2021-07-22 07:15:47 -06:00
Sébastien Han 7c5a86cb85 ceph: remove operator config ROOK_ALLOW_MULTIPLE_FILESYSTEMS
Multiple filesystems are supported as of the Ceph Pacific release. So we
remove the flag from the operator configuration and just have a ceph
version check instead.

The Operator configuration option `ROOK_ALLOW_MULTIPLE_FILESYSTEMS`
has been removed in favor of simply verifying the Ceph version is
at least Pacific.
Multiple filesystems are stable since Ceph Pacific.
So users who had `ROOK_ALLOW_MULTIPLE_FILESYSTEMS` enabled will
need to update their Ceph version to Pacific.

Closes: https://github.com/rook/rook/issues/7183
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-22 11:42:33 +02:00
Sébastien Han 6bce1ff3e9 ceph: move all of our docker.io reference to quay.io
Recently, the builds of `ceph/ceph` image moved to quay.io, see
https://github.com/ceph/ceph-build/pull/1883 for more details.
Current images will remain but new builds will happen on quay.io only.

This means that tags such as `v14.2`, `v15.2`,`v16.2` will need to
switch to quay.io to get updates.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-22 11:17:05 +02:00
parth-gr 77813f7093 ceph: update PodDisruptionBudget from v1beta1 to v1
This commit update the PodDisruptionBudget policy to use version v1
Updated to policy/v1 as policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+

Closes: https://github.com/rook/rook/issues/7917
Signed-off-by: parth-gr <paarora@redhat.com>
2021-07-22 14:09:33 +05:30
n.fraison f7bc76abad ceph: define specific ServceAccount/RBAC for ceph osd purge job
Currently ceph osd purge job rely on rook-ceph-system service account which is only created
on the rook operator namespace
When multiple clusters are deployed on distinct namespace the ceph osd purge failed to run
as the service account is not available
Creating a specific service account rook-ceph-purge-osd with appropriate rights on each cluster
namespace and operator namespace to still manage case where cluster is installed
along side the operator

Signed-off-by: n.fraison <n.fraison@criteo.com>
2021-07-22 00:25:32 +02:00
Travis Nielsen 5f30d538a4 Merge pull request #8348 from travisn/disable-v1beta1-ci
build: Min version for cassandra and nfs tests is k8s 1.16
2021-07-21 10:05:16 -06:00
Sébastien Han 0eaa1e71d1 ceph: remove removeExtraMirrors()
Since https://github.com/rook/rook/pull/7180 rook does not deployment
multiple deployments of the rbd-mirror daemon but uses replicas. Users
have upgraded from subsequent 1.6.x release and have that supported so
we can remove this code that was scale up/down the number of
deployments.

Closes: https://github.com/rook/rook/issues/7183
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-21 15:11:40 +02:00
Travis Nielsen 7b1df8bc80 Merge pull request #8352 from leseb/fix-yq-path
ci: change `yq` path to /usr/local/bin/yq
2021-07-21 07:09:44 -06:00
Sébastien Han db40a31bb2 ci: change yq path to /usr/local/bin/yq
So it will override any existing installation.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-21 14:15:07 +02:00
Travis Nielsen e7f44a8f3f build: min version for cassandra and nfs tests is k8s 1.16
The minimum K8s version for v1 CRDs is 1.16. Since support for
v1beta1 cassandra and nfs was removed and only v1 is supported,
the CI will now only run back to 1.16 for those providers.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 15:40:15 -06:00
Travis Nielsen e8f2edb3d4 Merge pull request #8252 from travisn/upgrade-from-1.6
ceph: Test upgrades from v1.6 to master
2021-07-20 13:15:58 -06:00
Travis Nielsen e660811ae0 Merge pull request #8344 from ashangit/rook-ceph-system
ceph: create rook-ceph-system ClusterRoleBinding in operator namespace
2021-07-20 11:45:07 -06:00
Travis Nielsen d1f02f22f3 ceph: test upgrades from v1.6 to master
With v1.7 approaching, the upgrade integration test will now test from
v1.6.x to the latest master, which will effectively become the v1.7
release soon.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 11:03:40 -06:00
Travis Nielsen c846f7a9fe Merge pull request #8326 from travisn/all-provider-crd-gen
Convert NFS and Cassandra CRDs to v1 and auto-generate the CRDs
2021-07-20 10:57:03 -06:00
Travis Nielsen 552a67bc3e Merge pull request #8330 from travisn/reconcile-mgr-services
ceph: Reconcile mgr services with every reconcile
2021-07-20 08:19:17 -06:00
Travis NielsenandNicolaj Græsholt f114db952b nfs: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for nfs for full fidelity of all settings.

Co-authored-by: Nicolaj Græsholt <figaw@hotmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
Travis Nielsen 093cf6dbbd cassandra: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for cassandra for full fidelity of all the
settings.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
Travis Nielsen 6ad9190f55 ceph: factor out test manifest helpers for providers
The ceph provider had been refactored to read the crds
and operator manifests from a file instead of copying the
manifests into the test code. Now the helpers are refactored
to allow the other providers to also reduce the manifest
duplication in tests.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:17:29 -06:00
Travis Nielsen 14227600cc Merge pull request #7535 from travisn/mon-stretch-location
ceph: Set the location on the mon daemon for stretch clusters
2021-07-20 08:17:01 -06:00
Sébastien Han 5e3ce936bf Merge pull request #8339 from leseb/multus-rbd-commands
ceph: proxy rbd commands when multus is enabled
2021-07-20 16:14:55 +02:00
n.fraison db2c8792b0 ceph: create rook-ceph-system ClusterRoleBinding in operator namespace
Operator can run on a namespace not named rook-ceph
The ClusterRoleBinding muste rely on .Release.Namespace helm value
Also move the ClusterRole and ClusterRoleBinding in appropriate yaml file

Signed-off-by: n.fraison <n.fraison@criteo.com>
2021-07-20 16:02:23 +02:00
Travis Nielsen 107f89260d ceph: reconcile mgr services with every reconcile
When there are multiple mgr daemons, the mgr sidecar owns reconciling
the services for the active mgr. However, for efficiency the sidecar
only reconciles the services if the active mgr changes. Now the operator
will also reconcile the mgr services to ensure that services are re-created
after being deleted, or otherwise in an incorrect state.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 07:23:58 -06:00
Travis Nielsen 3d629ea2a4 ceph: set minimum version for stretch clusters to v16.2.5
The stretch clusters pass a new parameter to the mon daemons which is
only available in v16.2.5 and newer. Older versions of Ceph will fail
to run with stretch clusters in rook.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 07:22:01 -06:00
Sébastien Han 1900160939 ceph: proxy rbd commands when multus is enabled
We now pass the networking spec to the clusterInfo so that the executor
can make the right decision on how to execute a command.
This is a small change that allows us to remove the CephBlockPool CR
since it's checking for rbd images. On a Multus deployment, the operator
does not have the network annotations, thus has no access to the OSD
network then rbd commands are hanging forever.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-20 12:28:00 +02:00
Travis Nielsen a0b4b30101 ceph: example manifest for stretch cluster in aws
Show an example of configuring a stretch cluster in AWS.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-19 14:30:45 -06:00
Travis Nielsen fca761c950 ceph: set the location on the mon daemon
The mon daemon in a stretch cluster now can have its location set
as a CLI param instead of setting it with a separate command.
This enables mon failover to set the location of a mon immediately
when it is joining quorum instead of having a delayed command
to set the location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-19 14:30:43 -06:00
Sébastien Han ce5777e528 Merge pull request #8155 from subhamkrai/osd-keyring
ceph: create new keyring for osd
2021-07-19 17:18:36 +02:00
subhamkrai d254c935e8 ceph: create new keyring for osd
when osd pods removed, they are not able
to add back due to missing `ceph auth`
or different `ceph auth`.

this commit mounts the secret `rook-ceph-admin-keyring`
inside the osd activate initcontainer which has
admin keyring.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-19 19:26:06 +05:30
Travis Nielsen b65e088ec3 Merge pull request #8325 from travisn/ceph-16.2.5
ceph: Update default ceph image to latest v16.2.5
2021-07-16 14:25:51 -06:00
Travis Nielsen 64df4cdbb6 ceph: update default ceph image to latest v16.2.5
The latest pacific release is 16.2.5 so we set this as the default
for Rook to deploy.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-16 10:21:48 -06:00
Travis Nielsen 9092c58405 Merge pull request #7969 from cybozu-go/ceph-add-the-description-to-create-osd-on-lv
docs: add the example of pvc-based ceph cluster on top of bare metal
2021-07-15 16:54:03 -06:00
Satoru Takeuchi fe8efeb028 docs: add the example of pvc-based ceph cluster on top of bare metal
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-07-14 21:11:39 +00:00
Satoru Takeuchi fa153421db Merge pull request #8304 from cybozu-go/ceph-mount-dev-directory-in-lvm-mode-osd-on-lv-backed-pvc
ceph: mount dev directory in lvm mode osd on  osd on lv-backed pvc
2021-07-14 05:25:59 +09:00
Sébastien Han 6dad71a7f1 Merge pull request #8310 from leseb/bz-1981849
ceph: add missing ceph cluster spec
2021-07-13 19:23:14 +02:00
Sébastien Han 1fd76a1168 ceph: add missing ceph cluster spec
The lib-bucket-provisioner was missing the cephCluster spec so the check
in RunAdminCommandNoMultisite would fail.
This is similar to ad54ed8aac just a
different place.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-13 18:45:39 +02:00