Commit Graph
7961 Commits
Author SHA1 Message Date
Travis Nielsen 213b1b4339 ceph: allow any device class for the osds in a pool
The device class can be any arbitrary value for a pool, depending
on the device class either automatically detected for the OSDs
or an arbitrary device class applied to OSDs in the cluster.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-21 12:14:43 -06:00
Travis Nielsen d7690d52e0 Merge pull request #7717 from leseb/ci-master-alive-runner
ci: let master/release runners alive on failure
2021-04-21 11:10:04 -06:00
Sébastien Han 834e5b2685 ci: let master/release runners alive on failure
If some jobs from master/release branches fail then we keep the runner
alive so we can debug it.
By default, runners will naturally timeout after 6 hours. See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-21 18:36:12 +02:00
Sébastien Han c26c5633c7 Merge pull request #7713 from leseb/use-ceph-v16.2.1
ceph: use v16.2.1 for operator image
2021-04-21 17:21:55 +02:00
Sébastien Han 4a425eddca ceph: use v16.2.1 for operator image
This is the first bugfix release in the Pacific stable series. It
addresses a security vulnerability in the Ceph authentication framework.
 We recommend users to update to this release. For a detailed release
notes with links & changelog please refer to the official blog entry at
https://ceph.io/releases/v16-2-1-pacific-released.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-21 16:01:20 +02:00
Sébastien Han 3eb33c9988 Merge pull request #7708 from cybozu-go/ci-ceph-fix-disk-flapping-problem-in-master
ceph: prevent disk flapping problem in master
2021-04-21 11:56:07 +02:00
Satoru Takeuchi 957465af88 ceph: prevent disk flapping problem in master
We resolved disk flapping problem (#7405) with #7467. However, this fix
was not applied to integration test in master.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 04:10:09 +00:00
Satoru Takeuchi 8a48dbb00d ceph: stop vendoring
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 02:01:56 +00:00
Travis Nielsen af6ad815ce Merge pull request #7703 from BlaineEXE/ceph-vol-report-v2-in-pacific
ceph: use ceph-volume v2 report format for Pacific
2021-04-20 15:16:11 -06:00
Lars Lehtonen 50a57985bf nfs: fix multiple imports
This removes multiple imports.

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
2021-04-20 12:57:12 -07:00
Blaine Gardner b4946aba95 ceph: use ceph-volume v2 report format for Pacific
This fixes cases where raw provisioning mode still cannot be used for
Rook v1.6 and Ceph Pacific clusters where json.Unmarshal fails to read
the ceph-volume output after OSD provisioning.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-20 12:45:05 -06:00
Travis Nielsen 6fc357006f Merge pull request #7472 from synarete/crush-initial-weight
ceph: allow passing 'osd-crush-initial-weight'
2021-04-20 12:34:01 -06:00
Shachar Sharon 9766e9b8fb ceph: allow passing 'osd-crush-initial-weight'
Ceph support the option '--osd-crush-initial-weight' upon OSD start,
which sets an explicit weight (in TiB units) to specific OSD. Allow
passing this option all the way from the user (similar to
'DeviceClass'), for the special case where end users wants it cluster
to have non-even balance over specific OSDs (e.g., one of the OSDs is
placed over a partition alongside OS-partition).

ROOK issue: https://github.com/rook/rook/issues/7448

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2021-04-20 20:50:54 +03:00
Travis Nielsen 8c1ac86325 Merge pull request #7691 from leseb/macos-build
ci: build and run various make commands on MacOS
2021-04-20 10:36:42 -06:00
Travis Nielsen f6984255fc Merge pull request #7680 from subhamkrai/fix-build
ci: make build not failing
2021-04-20 09:46:46 -06:00
Sébastien Han 16a9b1c381 Merge pull request #7640 from leseb/remove-rbd-mirror-ceph-conf
ceph: remove rbd-mirror ceph config/key
2021-04-20 17:29:14 +02:00
Sébastien Han 1d4488659e Merge pull request #7682 from leseb/fix-7669
ceph: use v14 image again for integration tests
2021-04-20 17:21:56 +02:00
Sébastien Han 57131e1a90 ci: build and run various make commands on MacOS
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00
Sébastien Han 2b87efef89 ci: add missing condition for action
Let's skip multi-cluster-mirroring if skip-ci label is set.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00
Blaine Gardner 02e2a3d2eb build: fix new sed line to work on mac
The POSIX spec for sed requires the flag -i to have a string specified.
`sed -i'' -e '<expression>'` is portable

The POSIX spec for sed also requires the insert "i" option to have
newlines. The following is portable:
```
sed -i'' -e '<line-num>i\
text'
```

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-20 17:20:05 +02:00
Sébastien Han c9dfd43f35 ceph: remove rbd-mirror ceph config/key
After the peer is imported via a token, ceph internally stores the
information of the remote cluster. So the rbd-mirror daemon does not
need to have files (key and config) for the remote site.
This means lighter code for us, as well no more restarts for the
rbd-mirror daemon once a peer is added to the configuration.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 16:55:09 +02:00
Sébastien Han 1028cd41f9 Merge pull request #7623 from leseb/add-rbd-mirroring-ci
ci: add rbd-mirror integration test
2021-04-20 16:53:59 +02:00
Sébastien Han d35ddcbecc ceph: use v14 again
Since Rook 1.6 is using raw mode for simple OSD scenarios we can use v14
again without having issue with ceph-volume.

We keep the upgraade test with v14.2.12 since Rook 1.5 does not have the
raw code to deploy OSDs.

Closes: https://github.com/rook/rook/issues/7669
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 16:51:46 +02:00
Travis Nielsen 56e7af3ee0 Merge pull request #7331 from thotz/healthcheckwhensslenabled
ceph: fix healthcheck in case ssl enabled for rgw
2021-04-20 08:32:17 -06:00
Sébastien Han 9a28e130b6 ci: add rbd-mirror integration test
Long due but finally there, the CI now test the block mirroring between
2 clusters.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 16:27:53 +02:00
Sébastien Han 77ddb71ed3 Merge pull request #7694 from leseb/mergify-auto-merge-1.6
bot: set the correct k8s version for mergify auto-merge on 1.6
2021-04-20 11:20:23 +02:00
Sébastien Han 05848dd34a bot: set the correct k8s version for mergify auto-merge on 1.6
In the release-1.6 branch, the k8s versions are different, we run
v1.20.2 where the master branch runs v1.21.0.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 11:07:11 +02:00
Jiffin Tony Thottan 70f94a2d3c ceph: removing serviceIP from bucketChecker struct
serviceIP is not used by the health bucket checker, hence removing it.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-20 14:24:11 +05:30
Jiffin Tony Thottan bcbe707bcb ceph: fix healthcheck incase ssl enabled for rgw
In case ssl enabled for RGW, bucket healthcheck won't work since access is denied from the server.
In that case configure S3 agent using "InsecureSkipVerify: true" option.

Fixes: 7288
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-20 14:23:48 +05:30
Sébastien Han b14009c3fc Merge pull request #7356 from Rakshith-R/remove-host-net
ceph: option to enable/disable host networking in CSI pods
2021-04-20 10:39:49 +02:00
subhamkrai 9305f12ecf ci: make build not failing
currently, the build action is not failing if there
is/are any modified files after the build. this commit
will validate build with `git status` after `make build`
command will fail if any changed file found.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-20 13:10:33 +05:30
Rakshith R f5a3f6fa71 ceph: add option to toggle host networking in csi plugin pods
As host networking is no longer necessary for CSI driver,
it will be disabled by default.
An option CSI_ENABLE_HOST_NETWORK is added to rook-ceph-operator-config
configmap and enableCSIHostNetwork in helm chart values.yaml to
enable/disable host network in CSI plugin pods.

Closes: #7203

Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-20 12:57:09 +05:30
Sébastien Han 48978f9919 Merge pull request #7692 from leseb/reset-rnote
docs: reset the pending release notes
2021-04-20 09:20:38 +02:00
Sébastien Han 4c21e5b7c4 docs: reset the pending release notes
Since the v1.6 release is completed, reset the pending release notes
for v1.7.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 09:15:35 +02:00
Sébastien Han 96945a539e Merge pull request #7681 from leseb/fix-mds-flag-on-upgrade
ceph: always apply config flags for mds and rgw
2021-04-20 08:31:12 +02:00
Travis Nielsen ec9f1b57b4 Merge pull request #7683 from leseb/fix-crd-unknow-field
ceph: preserve volume claim template metadata in schema
2021-04-19 12:13:59 -06:00
Travis Nielsen 12c9fb1a80 Merge pull request #7595 from leseb/always-run-actios
ci: always run all the actions
2021-04-19 11:40:54 -06:00
Sébastien Han 21d6001ed2 ceph: preserve volume claim template metadata in schema
The schema generation was not generating the metadata properties
under the VolumeClaimTemplate type, resulting in the properties being
ignored by the api server. The preserve unknown fields tag was
not working recursively on the volumeClaimTemplates since the subtypes
were defined. Now we post-process the schema so we can preserve
the unknown fields for the volume claim templates metadata.

We now use 'yq' to hot-fix the CRDs, the only 'downside' is that 'yq'
will not preserve the existing formatting, this is a known problem:

https://github.com/mikefarah/yq/issues/465

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 19:03:13 +02:00
Sébastien Han 341aefc95d ceph: revert "ceph: preserve volume claim template metadata in schema"
This reverts commit a9fb9d20bb. During the
CSV generation we strip the description lines from the CRD spec, however
the 'sed' command relies on the 'descrition' section to operate. Next
commit will fix this.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 19:01:13 +02:00
Sébastien Han ca1cf6672e ceph: always apply config flags for mds and rgw
We now always set the config flags on reconcile. Previously we were
looking for non-existing rgw or mds which means that on upgrade we would
never set those flags. However, we want to set those flags on "old"
cluster that got upgraded.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 17:39:43 +02:00
Sébastien Han 6a33395c4a ci: separate PR and release/master branch push into different files
We now have a single file with all the integration test that will run
when code is pushed to a release branch or merged in master.

Also, we have separate files with integration tests that run on pull
requests against master or a release branch.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:43:31 +02:00
Sébastien Han 5a7f405010 ci: remove the build action
This action is not needed anymore since other actions build already.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:41:47 +02:00
Sébastien Han aecec6f870 ci: only run cassandra job on cassandra code
Only run the cassandra integration if the PR involves cassandra, has the
"cassandra" label on it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:41:47 +02:00
Sébastien Han 8c95ac2880 bot: auto-label cassandra pull requests
Just like ceph if a PR title starts with "cassandra" let's add the
"cassandra" label to it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 12:41:47 +02:00
Sébastien Han f4cfc7a03d Merge pull request #7678 from alrs/ceph-osd-imports
ceph: fix multiple imports
2021-04-19 10:25:58 +02:00
Lars Lehtonen bb323de403 ceph: fix multiple imports
This mops up the last of the multiple-imports within ceph-related
packages, in pkg/daemon/ceph/agent/flexvolume/manager/ceph and
pkg/daemon/ceph/osd.

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
2021-04-19 00:06:10 -07:00
Blaine Gardner d3b094710a Merge pull request #7663 from BlaineEXE/radosgw-admin-omap-message-info-to-debug
ceph: change radosgw-admin workaround log to debug
2021-04-16 14:19:11 -06:00
Travis Nielsen 157d342330 Merge pull request #7662 from travisn/pool-size-min
ceph: Allow pool replica min size 0 for EC pools
2021-04-16 13:40:34 -06:00
Travis Nielsen 9da066eb3f ceph: allow pool replica min size 0 for EC pools
In v1.6 EC pools are being disallowed by the schema since the minimum
is set to 1 on the replicated.size. We expect the replicated.size to be
0 if an EC pool is being created.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-16 12:53:07 -06:00
Blaine Gardner 8aef8b756d ceph: change radosgw-admin workaround log to debug
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-16 12:51:54 -06:00