Commit Graph
13292 Commits
Author SHA1 Message Date
Jiffin Tony Thottan af86fbed72 docs: add documentation for service serving cert
Add necessary changes in object-openshift.yaml, ceph-object-store-crd.md for service serving cert support for RGW

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-26 18:23:33 +05:30
Jiffin Tony Thottan 809a8d174c test: adding unit test for service serving cert
include unit tests for the new apis and functionality introduced by the feature
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-26 18:23:29 +05:30
Jiffin Tony Thottan aea21d9c84 ceph: service server cert support for rgw
Service serving certificates are intended to applications that require
encryption in openshift. These certificates are issued as TLS web server
certificates. Currently RGW supports TLS authentication with help of
certs passed as secrets, in this case we add following details as
`service.annotations` in the Objectstore Gateway Spec :
```
service:
  annotations:
    service.beta.openshift.io/serving-cert-secret-name: <name for
autogenerated secret>
```

More details about service serving cert can be found at :
https://docs.openshift.com/container-platform/4.6/security/certificates/service-serving-certificate.html

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-26 18:15:12 +05:30
Sébastien Han ccb4221cb3 Merge pull request #7742 from BlaineEXE/object-bucket-infinite-pending-regression
ceph: add regression test for obc infinite pending
2021-04-26 10:27:41 +02:00
Morteza Ghasempour 0b638c1c52 docs: remove "this is long video b"
On Node Port service section, output of the `kubectl -n rook-ceph get service` command had a unwanted string "this is long video" that I've removed it.

Signed-off-by: m.ghasempour <mso.ghasempour@digikala.com>
2021-04-24 09:35:23 +04:30
Travis Nielsen f57db467f6 Merge pull request #7747 from cybozu-go/docs-ceph-fix-pacific-related-descriptions
docs: fix ceph pacific related descriptions
2021-04-23 08:40:48 -06:00
Satoru Takeuchi c8700bdae3 docs: fix ceph pacific related descriptions
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-23 14:08:20 +00:00
Travis Nielsen dbb34a3c79 Merge pull request #7715 from iamniting/1781-1944580
ceph: Expand Mon Pvc's
2021-04-23 08:01:30 -06:00
Nitin Goyal 99bc72d79f ceph: stop using updatePVCIfChanged and start using ExpandPVCIfRequired
delete updatePVCIfChanged method on "Cluster" from osd package and use
the common one ExpandPVCIfRequired from k8sutil pkg

Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
2021-04-23 08:47:49 +05:30
Nitin Goyal 3575c3bf78 ceph: expand Mon Pvc's
Expand Mon Pvc's if storage request has increased for the Mons in the
cephcluster crd.

Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
2021-04-23 08:47:49 +05:30
Nitin Goyal 2e8f513197 core: add func ExpandPVCIfRequired in k8sutil pkg
ExpandPVCIfRequired will expand PVC if and only if current PVC size is
less than the desired PVC size.

Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
2021-04-23 08:47:49 +05:30
Blaine Gardner 72b66c39c2 ceph: add regression test for obc infinite pending
Add an integration regression test for Ceph Object Storage Object Bucket
Claim (OBC) to verify that the OBC stays in "Bound" state after being
created. The claim should not fall back into "Pending" (or any other)
state soon after being created.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-22 15:23:34 -06:00
Travis Nielsen 8c9ddcd415 Merge pull request #6917 from alexcpn/patch-3
Update ceph-teardown.md
2021-04-22 12:24:55 -06:00
Travis Nielsen 116bf7dcf1 Merge pull request #7733 from travisn/ceph-15.2.11
ceph: Example manifests to deploy with v15.2.11
2021-04-22 12:12:41 -06:00
Travis Nielsen e9a5d9f087 Merge pull request #7568 from parth-gr/OSDs-for-added-node
ceph: update OSDs reconcile
2021-04-22 12:11:29 -06:00
Alex Punnen ffc0a252b3 docs: document command for detecting resources still in use
have found many times that the namespaces is in Terminating state even after trying to clean properly. Added the command to check this and remove all dangling objects

Signed-off-by: Alex Punnen <alexcpn@gmail.com>
2021-04-22 12:07:41 -06:00
parth-gr 80a9b26bfd ceph: operator loops forever over existing nodes
When the node get added to the cluster the operator always watches for node events and starts a reconcile when a node is ready.
Checking the Node if they already have OSDs present and stopping them from reconcile.

Closes: https://github.com/rook/rook/issues/7534
Signed-off-by: parth-gr <paarora@redhat.com>
2021-04-22 22:38:08 +05:30
Sébastien Han 6a97721576 ci: use latest pacific for mirroring test
So we can get https://github.com/ceph/ceph/pull/40985 and have a more
stable CI. Once v16.2.2 is released we can revert this.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 18:30:05 +02:00
parth-gr f6bd25ae01 ceph: update OSDs reconcile
OSDs not automatically started when adding nodes to existing cluster.
Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush map before querying for the node in the crush map, and reconcile when new nodes get added.

Closes: https://github.com/rook/rook/issues/7462
Signed-off-by: parth-gr <partharora1010@gmail.com>
2021-04-22 20:40:14 +05:30
Travis Nielsen 43594adbbb ceph: example manifests deploy with v15.2.11
The latest octopus release is v15.2.11 so we should use
it in the default examples to pick up the latest fixes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-22 08:54:08 -06:00
Sébastien Han a9641a9355 Merge pull request #7716 from leseb/debug-cii
ci: mitigate block locking issues and flapping mirroring test
2021-04-22 15:38:59 +02:00
Travis Nielsen 57c27a4b9e Merge pull request #7724 from Madhu-1/csi-v3.3.1
ceph: update cephcsi to v3.3.1
2021-04-22 07:15:07 -06:00
Sébastien Hanandsubhamkrai srai@redhat.com a59df3c4dd ci: add missing variable to the release/master action
The UDEV rule was missing the device to apply the rule out.

Co-authored-by: subhamkrai srai@redhat.com
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 14:07:46 +02:00
Sébastien Han 1930eb3b4c ceph: print provision error entirely
If the ceph-volume prepare call fails, let's print the output so we can
see it in the operator's logs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:37 +02:00
Sébastien Han d17fc6ac12 ci: increase the number of lock retries
We have seen new cases where retrying to lock the device 3 times is not
enough. It's the same race we had experienced where Ceph tries to
acquire a lock on the device but systemd-udevd does the same too.
Retrying 20 times every 0.1sec seems to mitigate that issue in the CI.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:37 +02:00
Sébastien Han 77be184f13 ci: wait a bit longer for the mirroring to happen
30sec was not enough see:

```
  # let's wait a bit for the image to be present
  timeout 30 sh -c 'until [ "$(kubectl exec -n rook-ceph-secondary deploy/rook-ceph-tools -t -- rbd -p replicapool ls|grep -c test)" -eq 1 ]; do echo "waiting for image to be mirrored" && sleep 1; done'
  shell: /bin/bash -e {0}
  env:
    GOROOT: /opt/hostedtoolcache/go/1.16.3/x64
    MINIKUBE_HOME: /home/runner/work/_temp
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:47.053+0000 7f824b13e2c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:47.053+0000 7f824b13e2c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:48.317+0000 7f31f6d372c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:48.317+0000 7f31f6d372c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:49.597+0000 7fa3a2ac52c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:49.597+0000 7fa3a2ac52c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:50.873+0000 7fab17fb12c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:50.873+0000 7fab17fb12c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:52.169+0000 7f944f52f2c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:52.169+0000 7f944f52f2c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
Error: Process completed with exit code 124.
```

However, when logging into the runner, the image was replicated.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:16 +02:00
Satoru Takeuchi a4277d2913 Merge pull request #7723 from travisn/build-linux-only
build: Only build linux platform
2021-04-22 17:35:05 +09:00
Madhu Rajanna 6bc3c1ac3a ceph: update cephcsi to v3.3.1
As we have cephcsi v3.3.1 release
with couple of bug fixes and a CVE fix
updating the cephcsi to latest release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-22 13:57:01 +05:30
Sébastien Han 34f96ad676 Merge pull request #7718 from travisn/unrestricted-device-class
ceph: Allow any device class for the osds in a pool
2021-04-22 10:09:55 +02:00
Travis Nielsen b17f1b4d0c Merge pull request #7690 from cybozu-go/stop-vendoring
ceph: stop vendoring
2021-04-21 15:06:19 -06:00
Travis Nielsen 2398b1636b build: only build linux platform
No need to include darwin and windows in the list of platforms
since rook is only containerized and expected to run in a linux
environment.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-21 14:57:31 -06:00
Travis Nielsen 213b1b4339 ceph: allow any device class for the osds in a pool
The device class can be any arbitrary value for a pool, depending
on the device class either automatically detected for the OSDs
or an arbitrary device class applied to OSDs in the cluster.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-21 12:14:43 -06:00
Travis Nielsen d7690d52e0 Merge pull request #7717 from leseb/ci-master-alive-runner
ci: let master/release runners alive on failure
2021-04-21 11:10:04 -06:00
Sébastien Han 834e5b2685 ci: let master/release runners alive on failure
If some jobs from master/release branches fail then we keep the runner
alive so we can debug it.
By default, runners will naturally timeout after 6 hours. See: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-21 18:36:12 +02:00
Sébastien Han c26c5633c7 Merge pull request #7713 from leseb/use-ceph-v16.2.1
ceph: use v16.2.1 for operator image
2021-04-21 17:21:55 +02:00
Sébastien Han 4a425eddca ceph: use v16.2.1 for operator image
This is the first bugfix release in the Pacific stable series. It
addresses a security vulnerability in the Ceph authentication framework.
 We recommend users to update to this release. For a detailed release
notes with links & changelog please refer to the official blog entry at
https://ceph.io/releases/v16-2-1-pacific-released.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-21 16:01:20 +02:00
Sébastien Han 3eb33c9988 Merge pull request #7708 from cybozu-go/ci-ceph-fix-disk-flapping-problem-in-master
ceph: prevent disk flapping problem in master
2021-04-21 11:56:07 +02:00
Satoru Takeuchi 957465af88 ceph: prevent disk flapping problem in master
We resolved disk flapping problem (#7405) with #7467. However, this fix
was not applied to integration test in master.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 04:10:09 +00:00
Satoru Takeuchi 8a48dbb00d ceph: stop vendoring
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 02:01:56 +00:00
Travis Nielsen af6ad815ce Merge pull request #7703 from BlaineEXE/ceph-vol-report-v2-in-pacific
ceph: use ceph-volume v2 report format for Pacific
2021-04-20 15:16:11 -06:00
Lars Lehtonen 50a57985bf nfs: fix multiple imports
This removes multiple imports.

Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
2021-04-20 12:57:12 -07:00
Blaine Gardner b4946aba95 ceph: use ceph-volume v2 report format for Pacific
This fixes cases where raw provisioning mode still cannot be used for
Rook v1.6 and Ceph Pacific clusters where json.Unmarshal fails to read
the ceph-volume output after OSD provisioning.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-20 12:45:05 -06:00
Travis Nielsen 6fc357006f Merge pull request #7472 from synarete/crush-initial-weight
ceph: allow passing 'osd-crush-initial-weight'
2021-04-20 12:34:01 -06:00
Shachar Sharon 9766e9b8fb ceph: allow passing 'osd-crush-initial-weight'
Ceph support the option '--osd-crush-initial-weight' upon OSD start,
which sets an explicit weight (in TiB units) to specific OSD. Allow
passing this option all the way from the user (similar to
'DeviceClass'), for the special case where end users wants it cluster
to have non-even balance over specific OSDs (e.g., one of the OSDs is
placed over a partition alongside OS-partition).

ROOK issue: https://github.com/rook/rook/issues/7448

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2021-04-20 20:50:54 +03:00
Travis Nielsen 8c1ac86325 Merge pull request #7691 from leseb/macos-build
ci: build and run various make commands on MacOS
2021-04-20 10:36:42 -06:00
Travis Nielsen f6984255fc Merge pull request #7680 from subhamkrai/fix-build
ci: make build not failing
2021-04-20 09:46:46 -06:00
Sébastien Han 16a9b1c381 Merge pull request #7640 from leseb/remove-rbd-mirror-ceph-conf
ceph: remove rbd-mirror ceph config/key
2021-04-20 17:29:14 +02:00
Sébastien Han 1d4488659e Merge pull request #7682 from leseb/fix-7669
ceph: use v14 image again for integration tests
2021-04-20 17:21:56 +02:00
Sébastien Han 57131e1a90 ci: build and run various make commands on MacOS
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00
Sébastien Han 2b87efef89 ci: add missing condition for action
Let's skip multi-cluster-mirroring if skip-ci label is set.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00