Commit Graph
8752 Commits
Author SHA1 Message Date
Sébastien Han 005b469955 Merge pull request #10445 from leseb/do-not-fail-ondeactivate-failure
osd: add lvm flag for lvm osd activate
2022-06-16 15:55:20 +02:00
Alexander Trost 9aaf363eac Merge pull request #10234 from parth-gr/external-document 2022-06-16 07:59:25 +00:00
Sébastien Han d768efab26 Merge pull request #10448 from travisn/remove-multus-issue
docs: Remove obsolete limitation from multus docs
2022-06-16 09:37:57 +02:00
parth-gr 65a0d81d00 docs: update ceph external cluster flags
Some optional Flags are already added
for external script but not documented,
So to expose the functionality to users
adding them to the doc list

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-16 11:42:54 +05:30
Travis Nielsen b5f57f7124 docs: remove obsolete limitation from multus docs
With the new holder pod that was implemented for the csi
driver to maintain a consistent network endpoint across
csi plugin restarts, there is no longer a limitation
that needs to be documented.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-06-15 14:25:28 -06:00
Sébastien Han 1bcb884571 osd: add lvm flag for lvm osd activate
Ancient OSDs based on LVM and activated by the Rook binary are still in
production. They fail on the deactivation of the VolumeGroup due to
incorrect filter. Setting the OSD as LVM adjusts the filter and make the
deactivate call working.

Repro without and with the correct filter:

```
[root@ca862389706c /]# vgchange -an -vv ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  global/use_lvmpolld not found in config: defaulting to 1
  devices/sysfs_scan not found in config: defaulting to 1
  devices/scan_lvs not found in config: defaulting to 0
  devices/multipath_component_detection not found in config: defaulting to 1
  devices/md_component_detection not found in config: defaulting to 1
  devices/fw_raid_component_detection not found in config: defaulting to 0
  devices/ignore_suspended_devices not found in config: defaulting to 0
  devices/ignore_lvm_mirrors not found in config: defaulting to 1
  devices/scan_lvs not found in config: defaulting to 0
  devices/allow_mixed_block_sizes not found in config: defaulting to 0
  devices/hints not found in config: defaulting to "all"
  activation/activation_mode not found in config: defaulting to "degraded"
  metadata/record_lvs_history not found in config: defaulting to 0
  devices/search_for_devnames not found in config: defaulting to "auto"
  activation/monitoring not found in config: defaulting to 1
  global/locking_type not found in config: defaulting to 1
  global/wait_for_locks not found in config: defaulting to 1
  global/prioritise_write_locks not found in config: defaulting to 1
  global/locking_dir not found in config: defaulting to "/run/lock/lvm"
  devices/md_component_detection not found in config: defaulting to 1
  devices/md_component_checks not found in config: defaulting to "auto"
  devices/multipath_wwids_file not found in config: defaulting to "/etc/multipath/wwids"
  global/use_lvmlockd not found in config: defaulting to 0
  report/output_format not found in config: defaulting to "basic"
  log/report_command_log not found in config: defaulting to 0
  devices/use_devicesfile not found in config: defaulting to 0
  /dev/log: stat failed: No such file or directory
  devices/global_filter not found in config: defaulting to global_filter = [ "a|.*|" ]
  Setting devices/filter to filter = [ "a|^/mnt/.*|", "r|.*|" ]
  devices/global_filter not found in config: defaulting to global_filter = [ "a|.*|" ]
  Setting devices/filter to filter = [ "a|^/mnt/.*|", "r|.*|" ]
  devices/devicesfile not found in config: defaulting to "system.devices"
  Obtaining the complete list of VGs to process
  VG name on command line not found in list of VGs: ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  Processing VG ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  Locking /run/lock/lvm/V_ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d WB
  Unlocking /run/lock/lvm/V_ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  Volume group "ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d" not found
  Cannot process volume group ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  global/notify_dbus not found in config: defaulting to 1
[root@ca862389706c /]# echo $?
5
[root@ca862389706c /]# vi /etc/lvm/lvm.conf
[root@ca862389706c /]# vgchange -an -vv ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  global/use_lvmpolld not found in config: defaulting to 1
  devices/sysfs_scan not found in config: defaulting to 1
  devices/scan_lvs not found in config: defaulting to 0
  devices/multipath_component_detection not found in config: defaulting to 1
  devices/md_component_detection not found in config: defaulting to 1
  devices/fw_raid_component_detection not found in config: defaulting to 0
  devices/ignore_suspended_devices not found in config: defaulting to 0
  devices/ignore_lvm_mirrors not found in config: defaulting to 1
  devices/scan_lvs not found in config: defaulting to 0
  devices/allow_mixed_block_sizes not found in config: defaulting to 0
  devices/hints not found in config: defaulting to "all"
  activation/activation_mode not found in config: defaulting to "degraded"
  metadata/record_lvs_history not found in config: defaulting to 0
  devices/search_for_devnames not found in config: defaulting to "auto"
  activation/monitoring not found in config: defaulting to 1
  global/locking_type not found in config: defaulting to 1
  global/wait_for_locks not found in config: defaulting to 1
  global/prioritise_write_locks not found in config: defaulting to 1
  global/locking_dir not found in config: defaulting to "/run/lock/lvm"
  devices/md_component_detection not found in config: defaulting to 1
  devices/md_component_checks not found in config: defaulting to "auto"
  devices/multipath_wwids_file not found in config: defaulting to "/etc/multipath/wwids"
  global/use_lvmlockd not found in config: defaulting to 0
  report/output_format not found in config: defaulting to "basic"
  log/report_command_log not found in config: defaulting to 0
  devices/use_devicesfile not found in config: defaulting to 0
  /dev/log: stat failed: No such file or directory
  /dev/block/7:0: size is 2097152 sectors
  /dev/nbd0: size is 0 sectors
  /dev/nbd8: size is 0 sectors
  /dev/vda: size is 83886080 sectors
  /dev/block/7:1: size is 0 sectors
  /dev/vda1: size is 83883999 sectors
  /dev/block/7:2: size is 0 sectors
  /dev/block/7:3: size is 0 sectors
  /dev/block/7:4: size is 0 sectors
  /dev/block/7:5: size is 0 sectors
  /dev/block/7:6: size is 0 sectors
  /dev/block/7:7: size is 0 sectors
  /dev/vdb: size is 83886080 sectors
  /dev/nbd1: size is 0 sectors
  /dev/nbd9: size is 0 sectors
  /dev/vdc: size is 83886080 sectors
  /dev/vdd: size is 83886080 sectors
  /dev/nbd2: size is 0 sectors
  /dev/nbd10: size is 0 sectors
  /dev/nbd3: size is 0 sectors
  /dev/nbd11: size is 0 sectors
  /dev/nbd4: size is 0 sectors
  /dev/nbd12: size is 0 sectors
  /dev/nbd5: size is 0 sectors
  /dev/nbd13: size is 0 sectors
  /dev/nbd6: size is 0 sectors
  /dev/nbd14: size is 0 sectors
  /dev/nbd7: size is 0 sectors
  /dev/nbd15: size is 0 sectors
  devices/global_filter not found in config: defaulting to global_filter = [ "a|.*|" ]
  Setting devices/filter to filter = [ "a|^/mnt/.*|", "r|^/dev/loop.*|", "a|^/dev/.*|", "r|.*|" ]
  /dev/block/7:0: using cached size 2097152 sectors
  /dev/block/7:0: using cached size 2097152 sectors
  /dev/block/7:0: No lvm label detected
  /dev/vda: using cached size 83886080 sectors
  /dev/vda1: using cached size 83883999 sectors
  /dev/vda1: using cached size 83883999 sectors
  /dev/vda1: No lvm label detected
  /dev/vdb: using cached size 83886080 sectors
  /dev/vdb: using cached size 83886080 sectors
  /dev/vdc: using cached size 83886080 sectors
  /dev/vdc: using cached size 83886080 sectors
  /dev/vdd: using cached size 83886080 sectors
  /dev/vdd: using cached size 83886080 sectors
  devices/global_filter not found in config: defaulting to global_filter = [ "a|.*|" ]
  Setting devices/filter to filter = [ "a|^/mnt/.*|", "r|^/dev/loop.*|", "a|^/dev/.*|", "r|.*|" ]
  devices/devicesfile not found in config: defaulting to "system.devices"
  /dev/block/7:0: using cached size 2097152 sectors
  /dev/nbd0: using cached size 0 sectors
  /dev/nbd8: using cached size 0 sectors
  /dev/vda: using cached size 83886080 sectors
  /dev/block/7:1: using cached size 0 sectors
  /dev/vda1: using cached size 83883999 sectors
  /dev/block/7:2: using cached size 0 sectors
  /dev/block/7:3: using cached size 0 sectors
  /dev/block/7:4: using cached size 0 sectors
  /dev/block/7:5: using cached size 0 sectors
  /dev/block/7:6: using cached size 0 sectors
  /dev/block/7:7: using cached size 0 sectors
  /dev/vdb: using cached size 83886080 sectors
  /dev/nbd1: using cached size 0 sectors
  /dev/nbd9: using cached size 0 sectors
  /dev/vdc: using cached size 83886080 sectors
  /dev/vdd: using cached size 83886080 sectors
  /dev/nbd2: using cached size 0 sectors
  /dev/nbd10: using cached size 0 sectors
  /dev/nbd3: using cached size 0 sectors
  /dev/nbd11: using cached size 0 sectors
  /dev/nbd4: using cached size 0 sectors
  /dev/nbd12: using cached size 0 sectors
  /dev/nbd5: using cached size 0 sectors
  /dev/nbd13: using cached size 0 sectors
  /dev/nbd6: using cached size 0 sectors
  /dev/nbd14: using cached size 0 sectors
  /dev/nbd7: using cached size 0 sectors
  /dev/nbd15: using cached size 0 sectors
  Obtaining the complete list of VGs to process
  Processing VG ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d VxbTWs-fMrC-KErD-Pc2l-NBs6-xffp-7n1qab
  Locking /run/lock/lvm/V_ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d WB
  metadata/lvs_history_retention_time not found in config: defaulting to 0
  /dev/vdd: using cached size 83886080 sectors
  Running command for VG ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d VxbTWs-fMrC-KErD-Pc2l-NBs6-xffp-7n1qab
  Deactivating logical volume ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d/osd-block-058f4926-6ee7-4165-9a55-db5a1fd22d2f.
  activation/verify_udev_operations not found in config: defaulting to 0
  Getting driver version
  Removing ceph--a8ae9f01--a440--4a0e--8e4d--592d3bce3a9d-osd--block--058f4926--6ee7--4165--9a55--db5a1fd22d2f (252:2)
  Deactivated 1 logical volumes in volume group ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d.
  0 logical volume(s) in volume group "ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d" now active
  Unlocking /run/lock/lvm/V_ceph-a8ae9f01-a440-4a0e-8e4d-592d3bce3a9d
  global/notify_dbus not found in config: defaulting to 1
[root@ca862389706c /]# echo $?
0
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-06-15 16:10:16 +02:00
Sébastien Han bfe4f7c111 osd: do not return if deactivate fail
Let it fail further once we run c-v activate. The error reported by LVM
could potentially ignored. If there is a failure, the c-v command will
fail anyways.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-06-15 15:42:48 +02:00
Blaine Gardner 1d34443f65 Merge pull request #10440 from parth-gr/ci-update
ci: remove the seprate quincy test
2022-06-13 13:15:55 -06:00
parth-gr dc0447a882 ci: remove the seprate quincy test
A seprate version test was introduce for rgw-endpoint
but from now as we are using quincy in master so
we can remove it and update the current canary test.

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-13 19:12:42 +05:30
Travis Nielsen a27a2aa4ca Merge pull request #10405 from humblec/master-label
mon: update node labels as per new key value
2022-06-10 08:03:43 -06:00
Humble Chirammal 750faa5747 mon: update node labels as per new key value
From kube v1.24 onwards, Kubeadm applies a "node-role" label to its
control plane Nodes.Currently this label key is node-role.kubernetes.io/master
and it will be renamed to node-role.kubernetes.io/control-plane. Kubeadm
also uses the same "node-role" as key for a taint it applies on control
plane Nodes. This taint key should also be renamed to
"node-role.kubernetes.io/control-plane".

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-06-10 08:12:04 +05:30
Travis Nielsen ac9734d532 Merge pull request #10424 from Madhu-1/cephcsi-v3.6.2
csi: update cephcsi from 3.6.1 to 3.6.2
2022-06-09 07:49:01 -06:00
Madhu Rajanna 3e70d278cb csi: update cephcsi from 3.6.1 to 3.6.2
As we have new cephcsi patch release i.e v3.6.2
updating Rook dependency to the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-09 14:52:51 +05:30
Satoru Takeuchi 831cdf3b2b Merge pull request #10333 from cybozu-go/test-add-canary-test-for-encrypted-osd
test: Fix encrypted OSDs on latest Ceph releases and add canary test for encrypted osd in host-based cluster
2022-06-09 08:17:02 +09:00
Satoru Takeuchi 6b0f539f61 test: add canary test for encrypted osd in host-based clusters
It's better to test the creation of encrypted devices in host-based clusters.
This test will reduce the potential risks of regression when modifying
the osd-creation code.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-06-08 22:34:01 +00:00
Satoru Takeuchi 66f745f11f osd: bypass ceph-conf validation failure in osd activation
"ceph.conf" must have the "fsid" global configuration to activate encrypted OSDs
after the following Ceph's PR is merged.

https://github.com/ceph/ceph/commit/25655e5a8829e001adf467511a6bde8142b0a575

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-06-08 20:41:00 +00:00
Travis Nielsen 786b326654 Merge pull request #10341 from travisn/cluster-telemetry
core: Report basic cluster and version telemetry
2022-06-08 10:49:01 -06:00
Travis Nielsen e9c8226c6e Merge pull request #10278 from parth-gr/external-sc
csi: make storage class updated with csi-users in external cluster
2022-06-08 07:48:08 -06:00
parth-gr b7c165b411 csi: make storage class updated with csi-users in external cluster
For using restricted auth users we need to update the
restricted users in the Storage class, so updating
it in the json output for external cluster

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-08 14:49:46 +05:30
parth-gr dcba689129 csi: update restricted auth flag
remove the restriction of per rados namespace as
there can be multiple namespaces and the csi-users
can only have 1 type in a storage class

Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-08 14:21:49 +05:30
Travis Nielsen fe989ad773 core: report basic cluster and version telemetry
For the ceph telemetry report the csi version, k8s version,
and metrics from the desired state in the cephcluster CR such
as mon count, stretch cluster, number of device sets, and
so on.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-06-07 12:13:51 -06:00
Travis Nielsen 1acc0621af Merge pull request #10342 from Madhu-1/fix-csi-pod-network
ceph: Add holder pod if csi host networking is disabled
2022-06-07 10:16:51 -06:00
Travis Nielsen 6200195275 Merge pull request #10376 from subhamkrai/webhokk-e2e
test: add e2e test for webhook validation
2022-06-07 09:40:59 -06:00
Travis Nielsen 5d17a4089e Merge pull request #10399 from subhamkrai/format-python-files
external: format python files using tool black
2022-06-07 09:35:35 -06:00
Travis Nielsen 10efd34fd5 Merge pull request #10306 from Madhu-1/update-snap-v6
csi: update sidecar to latest release
2022-06-07 08:41:13 -06:00
Madhu Rajanna 767cfcbf09 csi: update sidecar to latest release
updating node-driver-registrar to v2.5.1 release
, nfsplugin to 4.0.0 and csi-snapshotter to v6.0.1

Note:- with snapshotter v6.0.1 v1beta1 snapshot CRD
is not supported. If you want betav1 snapshot support
please switch to v5.x.x snapshotter.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-07 19:27:13 +05:30
Travis Nielsen d3e7a17db9 Merge pull request #10149 from parth-gr/ceph-quincy
build: Update examples and base image to use Ceph v17.2.0 Quincy version
2022-06-07 07:45:08 -06:00
Alexander Trost 73a77ef480 Merge pull request #10404 from subhamkrai/python-black-format 2022-06-07 12:20:45 +02:00
subhamkrai 4d92c2770a test: add e2e test for webhook validation
adding e2e test for webhook validation to test invalid
cluster, pool,objectStore, radosNamespace, subVolumeGroup CR
that will each be rejected by their admission controller.

Closes: https://github.com/rook/rook/issues/10024
Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-07 14:01:02 +05:30
subhamkrai d5bac59a74 ci: use python formatting tool black
formatting other python files using black

Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-07 13:49:14 +05:30
Madhu Rajanna 276005cfd5 csi: add holder pod if csi hostnetworking is disabled
If csi is configured not the use the
hostnetworking, deploy the holder
pod for executing the commands with nsenter.
The implementation is same as multus.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-07 13:09:12 +05:30
parth-gr 38dcf09843 build: update examples and base image to use ceph v17.2.0 quincy version
The operator base image has been changed to
ceph v17.2.0 quincy release version `quay.io/ceph/ceph:v17.2.0`

Closes: https://github.com/rook/rook/issues/10140
Signed-off-by: parth-gr <paarora@redhat.com>
2022-06-07 12:55:47 +05:30
Travis Nielsen 7957de80cf Merge pull request #10116 from Madhu-1/update-sidecar-repo
csi: Change sidecar images repo to registry.k8s.io
2022-06-06 12:47:13 -06:00
Travis Nielsen 9f39dec7a9 Merge pull request #10385 from subhamkrai/better-file-name
ci: more specific ci config folder names
2022-06-06 09:57:43 -06:00
subhamkrai 54482015ec ci: more specific ci config folder names
Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-06 20:23:34 +05:30
subhamkrai 00f21c0866 external: format python files using tool black
`black` is python formatter tool which
automatically formats python files without
manual changing. just run `black <python-file>`
and done.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-06 20:01:27 +05:30
Sébastien Han bb80aba93f Merge pull request #10373 from leseb/surface-errors-better
osd: do not hide the error
2022-06-06 09:30:44 +02:00
Sébastien Han 4e10d05558 Merge pull request #10380 from leseb/csi-soft-reconcile
csi: do not block the reconcile if all clusters are not ready
2022-06-06 09:24:32 +02:00
Madhu Rajanna 15da42cf0c csi: change sidecar images repo to new link
Change sidecar images repo to registry.k8s.io
which is a redirect to k8s.gcr.io.

Ref: https://github.com/kubernetes/k8s.io/wiki/
New-Registry-url-for-Kubernetes-(registry.k8s.io)

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-06 11:02:38 +05:30
Satoru Takeuchi f744087342 Merge pull request #10389 from travisn/obsolete-probe
core: Remove obsolete probe helper
2022-06-06 11:46:22 +09:00
Travis Nielsen 2f88745673 core: remove obsolete probe helper
The liveness probe is applied by a more general helper
rather than the specific helpers for each daemon.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-06-03 13:27:39 -06:00
Travis Nielsen b5ebb2ed97 Merge pull request #10383 from xu-cheng/topology
osd: ignore the region label when it is the same as zone label in topology
2022-06-03 11:41:53 -06:00
Cheng XU 9b03d0759b osd: ignore the region label when it is the same as zone label in topology
Ceph requires all topology label being unique. And it is quite common to
have topology.kubernetes.io/region and topology.kubernetes.io/zone being set
to the same value. In this case, let's just ignore the region label.

See https://github.com/rook/rook/issues/9419#issuecomment-1144311818

Signed-off-by: Cheng XU <git@xuc.me>
2022-06-03 10:13:22 -07:00
Sébastien Han 219e017564 osd: do not hide the error
Return the error instead of hiding it, err could be non-nil for a lot of
reasons, eg: a canceled context or a non-existing object.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-06-03 11:00:10 +02:00
Sébastien Han 95b2e60679 csi: do not block the reconcile if all clusters are not ready
If some clusters don't have their monitors up and running, we don't have
a clusterInfo (internal detail struct). We don't want to block the csi
controller reconcile if all the clusters are not ready. So let's proceed
with the elements that are ready and reconcile again to see if others
have made progress.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-06-03 10:59:31 +02:00
Sébastien Han dc525d2121 Merge pull request #10386 from subhamkrai/update-cni
ci: add new cni flag in minikube start
2022-06-03 08:50:19 +02:00
Sébastien Han eb08db40e9 Merge pull request #10350 from subhamkrai/support-py2
external-script: remove signature import
2022-06-03 08:49:58 +02:00
subhamkrai 6c3f3438a2 ci: add new cni flag in minikube start
adding cni flag in minikube start command
since it is required for multus ci.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-06-03 10:48:19 +05:30
Satoru Takeuchi 2ba0fe2c4d Merge pull request #10382 from travisn/canary-tests
build: Flannel flag to minikube is causing test failures
2022-06-03 08:28:57 +09:00
Travis Nielsen 7a938090a1 build: flannel flag to minikube is causing test failures
The minikube node is staying in NotReady state and preventing
the operator from even starting in the canary and other suites
that use the setup-cluster-resources action.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-06-02 15:33:57 -06:00