Commit Graph
7961 Commits
Author SHA1 Message Date
shenjiatong e1fd23933f ceph: add phase to cephfilesystems crd
add a Phase to columns that should be printed

Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
2021-04-28 15:18:11 +08:00
Sébastien Han 8fd932b18b Merge pull request #7685 from moritazy/master
docs: remove unrelated text "this is long video b"
2021-04-28 09:13:28 +02:00
Sébastien Han 302b90014e Merge pull request #7631 from BlaineEXE/allow-volumeclaimtemplate-metadata
api: persist VolumeClaimTemplate metadtata
2021-04-28 09:10:33 +02:00
Jiffin Tony Thottan f9c7634d88 ceph: support for OBC operations on enabled TLS RGW endpoint
OBC operations will fail if RGW only enabled HTTPS. With this change, OBC operations
will use needed TLS certs to succeed the operation.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-28 12:37:38 +05:30
Jiffin Tony Thottan 7a5928d2c5 ceph: add api to fetch cacert for RGW server
Rook operator sends S3 to RGW server for bucket operations performed by healthchecker,
bucketprovisioner etc. Add api to fetch cacert if the TLS enabled for RGW.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-28 12:25:36 +05:30
Blaine Gardner b60c495561 Merge pull request #7704 from alrs/nfs-multiple-imports
nfs: fix multiple imports
2021-04-27 15:47:34 -06:00
Blaine Gardner d260b81e97 core: persist VolumeClaimTemplate metadtata
Allow metadata on VolumeClaimTemplates in StorageClassDeviceSets to be
persisted. The latest changes to controller-gen allows us to embed
objectmeta for embedded types.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-27 13:37:48 -06:00
Travis Nielsen 6f61d95dc3 ceph: clarify log message to continue with upgrade if needed
The ceph upgrade blocks if the ceph heatlh is HEALTH_ERR. If the user
wants to force the upgrade despite the health error, they will need
to set skipUpgradeChecks: true in the cephcluster CR. This message
will hopefully make that setting more discoverable.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-27 11:31:28 -06:00
Travis Nielsen 3cb6628865 Merge pull request #7759 from travisn/topology-affinity-upgrade
ceph: Detect the osd topology affinity during upgrade
2021-04-27 10:18:09 -06:00
Sébastien Han 80713db94d Merge pull request #7677 from subhamkrai/improve-action-writing
ci: improve github action writing
2021-04-27 17:33:14 +02:00
Travis Nielsen 71bbab9bf6 Merge pull request #7606 from rohan47/multus_experimental
ceph: moved multus support out of experimental
2021-04-27 09:17:46 -06:00
Travis Nielsen 08f896e865 ceph: detect the osd topology affinity during upgrade
The nodeAffinity for portable OSDs is needed to ensure the OSDs are
tied to the same zone or other failure domain where the OSD was
prepared and where it is expected to run. During upgrade the topology
affinity was not being detected since the osd prepare job is skipped
in that case. Now the topology will be detected from the running
OSD daemon pod's node.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-27 08:51:51 -06:00
subhamkrai 7974d46b20 ci: improve github action writing
we have lots of duplicate codes for
github action. this commit refactor
them to a bash script which will minimize
duplicate codes.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-04-27 20:15:52 +05:30
Sébastien Han afb7fb483a Merge pull request #7757 from BlaineEXE/upgrade-doc-crd-failures
docs: workaround preserveUnknownFields upgrade
2021-04-27 10:23:27 +02:00
Satoru Takeuchi 2beaa5db79 Merge pull request #7719 from cybozu-go/ceph-simplify-cmd-timeout
ceph: unity timeout values of commands
2021-04-27 07:12:36 +09:00
Blaine Gardner c90f3c3721 docs: workaround preserveUnknownFields upgrade
When upgrading Rook clusters created before v1.5,
`kubectl apply -f crds.yaml` fails due to the existence of
the usupported `spec.preserveUnknownFields`. Add text documenting how to
work around this issue in the Ceph upgrade docs.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-26 15:14:45 -06:00
rohan47 a654bc777b ceph: moved multus support out of experimental
moved multus support out of experimental.

Signed-off-by: rohan47 <rohgupta@redhat.com>
2021-04-27 00:31:09 +05:30
Sébastien Han bb5f233233 Merge pull request #7598 from thotz/rgwserviceservercert
ceph: service server cert support for rgw
2021-04-26 15:24:20 +02:00
Satoru Takeuchi 709e8d5ba8 ceph: unity timeout values of commands
There is no strong reason to have different timeout values.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-26 12:57:40 +00:00
Jiffin Tony Thottan af86fbed72 docs: add documentation for service serving cert
Add necessary changes in object-openshift.yaml, ceph-object-store-crd.md for service serving cert support for RGW

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-26 18:23:33 +05:30
Jiffin Tony Thottan 809a8d174c test: adding unit test for service serving cert
include unit tests for the new apis and functionality introduced by the feature
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-26 18:23:29 +05:30
Jiffin Tony Thottan aea21d9c84 ceph: service server cert support for rgw
Service serving certificates are intended to applications that require
encryption in openshift. These certificates are issued as TLS web server
certificates. Currently RGW supports TLS authentication with help of
certs passed as secrets, in this case we add following details as
`service.annotations` in the Objectstore Gateway Spec :
```
service:
  annotations:
    service.beta.openshift.io/serving-cert-secret-name: <name for
autogenerated secret>
```

More details about service serving cert can be found at :
https://docs.openshift.com/container-platform/4.6/security/certificates/service-serving-certificate.html

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-26 18:15:12 +05:30
Sébastien Han ccb4221cb3 Merge pull request #7742 from BlaineEXE/object-bucket-infinite-pending-regression
ceph: add regression test for obc infinite pending
2021-04-26 10:27:41 +02:00
Morteza Ghasempour 0b638c1c52 docs: remove "this is long video b"
On Node Port service section, output of the `kubectl -n rook-ceph get service` command had a unwanted string "this is long video" that I've removed it.

Signed-off-by: m.ghasempour <mso.ghasempour@digikala.com>
2021-04-24 09:35:23 +04:30
Travis Nielsen f57db467f6 Merge pull request #7747 from cybozu-go/docs-ceph-fix-pacific-related-descriptions
docs: fix ceph pacific related descriptions
2021-04-23 08:40:48 -06:00
Satoru Takeuchi c8700bdae3 docs: fix ceph pacific related descriptions
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-23 14:08:20 +00:00
Travis Nielsen dbb34a3c79 Merge pull request #7715 from iamniting/1781-1944580
ceph: Expand Mon Pvc's
2021-04-23 08:01:30 -06:00
Nitin Goyal 99bc72d79f ceph: stop using updatePVCIfChanged and start using ExpandPVCIfRequired
delete updatePVCIfChanged method on "Cluster" from osd package and use
the common one ExpandPVCIfRequired from k8sutil pkg

Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
2021-04-23 08:47:49 +05:30
Nitin Goyal 3575c3bf78 ceph: expand Mon Pvc's
Expand Mon Pvc's if storage request has increased for the Mons in the
cephcluster crd.

Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
2021-04-23 08:47:49 +05:30
Nitin Goyal 2e8f513197 core: add func ExpandPVCIfRequired in k8sutil pkg
ExpandPVCIfRequired will expand PVC if and only if current PVC size is
less than the desired PVC size.

Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
2021-04-23 08:47:49 +05:30
Blaine Gardner 72b66c39c2 ceph: add regression test for obc infinite pending
Add an integration regression test for Ceph Object Storage Object Bucket
Claim (OBC) to verify that the OBC stays in "Bound" state after being
created. The claim should not fall back into "Pending" (or any other)
state soon after being created.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-22 15:23:34 -06:00
Travis Nielsen 8c9ddcd415 Merge pull request #6917 from alexcpn/patch-3
Update ceph-teardown.md
2021-04-22 12:24:55 -06:00
Travis Nielsen 116bf7dcf1 Merge pull request #7733 from travisn/ceph-15.2.11
ceph: Example manifests to deploy with v15.2.11
2021-04-22 12:12:41 -06:00
Travis Nielsen e9a5d9f087 Merge pull request #7568 from parth-gr/OSDs-for-added-node
ceph: update OSDs reconcile
2021-04-22 12:11:29 -06:00
Alex Punnen ffc0a252b3 docs: document command for detecting resources still in use
have found many times that the namespaces is in Terminating state even after trying to clean properly. Added the command to check this and remove all dangling objects

Signed-off-by: Alex Punnen <alexcpn@gmail.com>
2021-04-22 12:07:41 -06:00
parth-gr 80a9b26bfd ceph: operator loops forever over existing nodes
When the node get added to the cluster the operator always watches for node events and starts a reconcile when a node is ready.
Checking the Node if they already have OSDs present and stopping them from reconcile.

Closes: https://github.com/rook/rook/issues/7534
Signed-off-by: parth-gr <paarora@redhat.com>
2021-04-22 22:38:08 +05:30
Sébastien Han 6a97721576 ci: use latest pacific for mirroring test
So we can get https://github.com/ceph/ceph/pull/40985 and have a more
stable CI. Once v16.2.2 is released we can revert this.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 18:30:05 +02:00
parth-gr f6bd25ae01 ceph: update OSDs reconcile
OSDs not automatically started when adding nodes to existing cluster.
Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush map before querying for the node in the crush map, and reconcile when new nodes get added.

Closes: https://github.com/rook/rook/issues/7462
Signed-off-by: parth-gr <partharora1010@gmail.com>
2021-04-22 20:40:14 +05:30
Travis Nielsen 43594adbbb ceph: example manifests deploy with v15.2.11
The latest octopus release is v15.2.11 so we should use
it in the default examples to pick up the latest fixes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-22 08:54:08 -06:00
Sébastien Han a9641a9355 Merge pull request #7716 from leseb/debug-cii
ci: mitigate block locking issues and flapping mirroring test
2021-04-22 15:38:59 +02:00
Travis Nielsen 57c27a4b9e Merge pull request #7724 from Madhu-1/csi-v3.3.1
ceph: update cephcsi to v3.3.1
2021-04-22 07:15:07 -06:00
Sébastien Hanandsubhamkrai srai@redhat.com a59df3c4dd ci: add missing variable to the release/master action
The UDEV rule was missing the device to apply the rule out.

Co-authored-by: subhamkrai srai@redhat.com
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 14:07:46 +02:00
Sébastien Han 1930eb3b4c ceph: print provision error entirely
If the ceph-volume prepare call fails, let's print the output so we can
see it in the operator's logs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:37 +02:00
Sébastien Han d17fc6ac12 ci: increase the number of lock retries
We have seen new cases where retrying to lock the device 3 times is not
enough. It's the same race we had experienced where Ceph tries to
acquire a lock on the device but systemd-udevd does the same too.
Retrying 20 times every 0.1sec seems to mitigate that issue in the CI.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:37 +02:00
Sébastien Han 77be184f13 ci: wait a bit longer for the mirroring to happen
30sec was not enough see:

```
  # let's wait a bit for the image to be present
  timeout 30 sh -c 'until [ "$(kubectl exec -n rook-ceph-secondary deploy/rook-ceph-tools -t -- rbd -p replicapool ls|grep -c test)" -eq 1 ]; do echo "waiting for image to be mirrored" && sleep 1; done'
  shell: /bin/bash -e {0}
  env:
    GOROOT: /opt/hostedtoolcache/go/1.16.3/x64
    MINIKUBE_HOME: /home/runner/work/_temp
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:47.053+0000 7f824b13e2c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:47.053+0000 7f824b13e2c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:48.317+0000 7f31f6d372c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:48.317+0000 7f31f6d372c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:49.597+0000 7fa3a2ac52c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:49.597+0000 7fa3a2ac52c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:50.873+0000 7fab17fb12c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:50.873+0000 7fab17fb12c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
unable to get monitor info from DNS SRV with service name: ceph-mon
rbd: couldn't connect to the cluster!
rbd: listing images failed: (2) No such file or directory
2021-04-22T08:17:52.169+0000 7f944f52f2c0 -1 failed for service _ceph-mon._tcp
2021-04-22T08:17:52.169+0000 7f944f52f2c0 -1 monclient: get_monmap_and_config cannot identify monitors to contact
command terminated with exit code 2
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
waiting for image to be mirrored
Error: Process completed with exit code 124.
```

However, when logging into the runner, the image was replicated.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-22 11:30:16 +02:00
Satoru Takeuchi a4277d2913 Merge pull request #7723 from travisn/build-linux-only
build: Only build linux platform
2021-04-22 17:35:05 +09:00
Madhu Rajanna 6bc3c1ac3a ceph: update cephcsi to v3.3.1
As we have cephcsi v3.3.1 release
with couple of bug fixes and a CVE fix
updating the cephcsi to latest release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-22 13:57:01 +05:30
Sébastien Han 34f96ad676 Merge pull request #7718 from travisn/unrestricted-device-class
ceph: Allow any device class for the osds in a pool
2021-04-22 10:09:55 +02:00
Travis Nielsen b17f1b4d0c Merge pull request #7690 from cybozu-go/stop-vendoring
ceph: stop vendoring
2021-04-21 15:06:19 -06:00
Travis Nielsen 2398b1636b build: only build linux platform
No need to include darwin and windows in the list of platforms
since rook is only containerized and expected to run in a linux
environment.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-21 14:57:31 -06:00