Commit Graph
8232 Commits
Author SHA1 Message Date
Sébastien Han 23832c8cc7 osd: add namespace in vault client
If the KMS connection details for Vault have a VAULT_NAMESPACE we must
use it when building the vault client to auto-detect the KV internal
version.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-10 12:13:41 +01:00
Travis Nielsen 014eef7bb9 Merge pull request #9624 from yati1998/addons-doc
doc: Adds doc for implementation of csi-addons
2022-02-09 09:21:48 -07:00
yati1998 045cc87e03 docs: adds doc for implementation of csi-addons
This commit adds the documentation for implementation
of csi-addons.

Signed-off-by: yati1998 <ypadia@redhat.com>
2022-02-09 15:10:57 +05:30
Travis Nielsen 4c63785867 Merge pull request #9696 from travisn/update-pool-properties
object: Update object pool properties during reconcile
2022-02-07 07:34:08 -07:00
Sébastien Han 40dc6cf947 Merge pull request #9703 from parth-gr/get-and-create
csi: make create and get process separate for csi users
2022-02-07 11:30:42 +01:00
Travis Nielsen 221b3dfb72 object: update pool properties during reconcile
The reconcile was skipping updating most pool properties for
object stores. The implementation of pools between the file,
object, and pool controllers had some duplicate code, so
this change also factors out the common code for better
reuse in a single place. Anytime a pool is created or updated,
it will now consistently update all the pool properties
that are expected to be modifiable.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-02-04 13:41:12 -07:00
Blaine Gardner ba7318c388 Merge pull request #9643 from BlaineEXE/multus-csi-design
docs: add CSI section to Multus design doc
2022-02-04 11:19:23 -07:00
Satoru Takeuchi 5e038806a8 Merge pull request #9682 from cybozu-go/core-eliminate-nautilus-code
core: eliminate unnecessary code
2022-02-04 23:39:45 +09:00
parth-gr 4f43f2d93e csi: make create and get process separate for csi users
Currently if we run external-script and user already exit,
it fails to create/run same user again if the caps are upgraded,
so we need to check first if user is already present.
And for upgrading user we can do upgrade after this
PR: https://github.com/rook/rook/pull/9609 will get merged.

Closes: https://bugzilla.redhat.com/show_bug.cgi?id=2048458

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-04 20:08:31 +05:30
Satoru Takeuchi 2d8f9b67c6 core: eliminate unnecessary code
We can eliminate the OSD creation code for Ceph < v15.2.0 because Nautilus(v14.2.z)
is no longer supported. In other words, we can now expect that all newly-created
OSDs on PVC are raw mode.

Please note that the existing lvm mode OSDs on PVCs still be supported after this change.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-02-04 13:59:01 +00:00
Blaine Gardner 2bf1e1f07b docs: add CSI section to Multus design doc
Add a section detailing a design to handle the volume-operations-blocked
on CSI restart issue when running with Multus.

See: https://github.com/rook/rook/issues/8085

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-02-03 11:57:07 -07:00
Sébastien Han 2209e14ded Merge pull request #9694 from leseb/subvolumegroup-crd-external
subvolumegroup: handle external mode
2022-02-03 17:59:53 +01:00
Sébastien Han 49f75c8ec3 subvolumegroup: handle external mode
If the cluster is external, users might still want to take advantage of
the ceph-csi configmap configuration for ceph filesystem subvolume
group. This configuration is handled by the controller whenever a
cephFilesystemSubVolumeGroup CR is created or deleted.
If the present cluster CephCluster is external, the controller
(cephFilesystemSubVolumeGroup) won't handle creation nor deletion of sub volume groups.
It just assumes that the given name is correct and the controller goes ahead with the
ceph-csi configuration.

Closes: #9551
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-03 16:28:07 +01:00
Travis Nielsen d24eab2229 Merge pull request #9684 from travisn/roadmap-1.9
docs: Update the roadmap for v1.9
2022-02-01 13:04:27 -07:00
Travis Nielsen 6a37b752d6 docs: update the roadmap for v1.9
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-31 15:36:58 -07:00
Travis Nielsen 9eef0c257d Merge pull request #9680 from Madhu-1/fix-snapshot-test
csi: fix snapshot tests
2022-01-31 08:01:45 -07:00
Madhu Rajanna 44353167d7 csi: fix snapshot CRD and controller cleanup oder
cleanup the CRD and controller in defer
to make sure the CRD and controller are
removed as filesystem/block test also
install CRD and controller separately.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-31 13:17:55 +05:30
Madhu Rajanna 97e5b78c11 csi: cover error cases in snapshot ready check
Currently, we sleep only if the snapshot is not ready
and try again, this covers only the happy path.
the checks might fail to get the snapshot object
or the `.status.readyToUse` might not be set yet.
First sleep and then try to check snapshot is
ready or not, if we do this we cover all the cases
when checking the snapshot ready status.
Changed sleep internal to do incremental sleep to
give more time for tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-31 13:07:37 +05:30
Satoru Takeuchi 2f867c03b8 Merge pull request #9675 from subhamkrai/clean
core: remove obsolete ceph-volume executor code
2022-01-31 13:12:23 +09:00
Travis Nielsen 096dabfa6c Merge pull request #9578 from micalgenus/fix/pacific-rgw-rolling
rgw: gateway deployment strategy RollingUpdate on Pacific
2022-01-28 08:12:06 -07:00
subhamkrai 6bdd5b2062 core: remove obsolete ceph-volume executor code
In volume.go, remove if condition in `callCephVolume`
method where we always pass false.

In exec.go, removing `ExecuteCommandWithFile*` methods
which was not used anywhere.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-01-28 20:17:55 +05:30
Travis Nielsen c985336e76 Merge pull request #9672 from Madhu-1/fix-9670
csi: skip validation when installing snapshotter
2022-01-28 07:47:23 -07:00
Sébastien Han 278ca803f9 Merge pull request #9663 from leseb/fix-9544
core: dereference pointer before trying to compare with deepequal
2022-01-28 11:20:29 +01:00
Madhu Rajanna c40ff24410 csi: skip validation when installing snapshotter
in kubernetes 1.18 we are seeing CRD validation
errors when installing with kubectl, adding
validate=false to skip the validation and
to make tests works with kubernetes 1.18

closes: #9670

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-28 08:25:07 +05:30
Travis Nielsen 563e1865ce Merge pull request #9667 from Madhu-1/fix-snap-test
csi: check deployment for snapshot controller
2022-01-27 12:07:25 -07:00
Travis Nielsen 7bde20d5eb Merge pull request #9661 from Madhu-1/fix-9655
helm: add annotations and labels for VSC
2022-01-27 11:51:55 -07:00
Travis Nielsen 3e4edc0f1d csi: add patch permission to volumesnapshotcontents
The update to the snaphots 5.0 controller requires
rbac for patching the volumesnapshotcontents.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-27 11:35:06 -07:00
Madhu Rajanna 53e12d687f csi: check deployment for snapshot controller
The external-snapshotter was deployed as statefulset
in 4.x and now its deployed as a deployment. updated
the check in CI to make sure deployment is created.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 23:06:16 +05:30
Sébastien Han 5404ec13a2 core: dereference pointer before trying to compare with deepequal
Prior to this, we were comparing a pointer (the memory address) with a
struct. This was obviously always failing and returned false. We must
dereference the pointer to access the data contained at that memory
location.

Closes: https://github.com/rook/rook/issues/9544
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-27 17:41:42 +01:00
Travis Nielsen 888f809f17 Merge pull request #9665 from Madhu-1/fix-snapshotter
csi: bump csi snapshotter image to v5
2022-01-27 09:25:29 -07:00
Madhu RajannaandMathieu Parent cf46615688 csi: bump csi snapshotter image to v5
updating the csi-snapshotter and dependencies
to v5.0.1 released version.

Co-authored-by: Mathieu Parent <mathieu.parent@insee.fr>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 21:17:51 +05:30
Travis Nielsen d0ac3b484b Merge pull request #9651 from sathieu/patch-2
Bump csi resizer and volume replication images
2022-01-27 08:30:09 -07:00
Sébastien Han 43dd0a692c Merge pull request #9658 from travisn/merge-all-osd-resources
osd: Merge all osd resources including custom
2022-01-27 10:09:18 +01:00
Madhu Rajanna 08dadaa486 helm: add annotations and labels for VSC
add support to specify annotations and
labels for the volumesnapshotclass.

closes: #9655

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 13:26:10 +05:30
Travis Nielsen f9ac966048 osd: merge all osd resources including custom
The merging of OSD resource limits and requests assumed
that only the cpu and memory resources needed to be merged.
It is also possible to set custom resource properties such
as intel.com/sriov_net_in: '1' for use with multus, so
the merging needs to be more general.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-26 10:46:54 -07:00
Mathieu Parent 88a449d09c csi: bump csi resizer and volume replication images
Upgraded images:

- k8s.gcr.io/sig-storage/csi-resizer: v1.3.0 -> v1.4.0
- quay.io/csiaddons/volumereplication-operator: v0.1.0 -> v0.3.0

Detailed changelogs:

- https://github.com/kubernetes-csi/external-resizer/blob/release-1.4/CHANGELOG/CHANGELOG-1.4.md
- https://github.com/csi-addons/volume-replication-operator/tree/v0.2.0
- https://github.com/csi-addons/volume-replication-operator/tree/v0.3.0

Signed-off-by: Mathieu Parent <mathieu.parent@insee.fr>
2022-01-26 17:38:53 +01:00
Sébastien Han 287d9a3fbe Merge pull request #9647 from rzarzynski/wip-extra-env-vars-2
osd: allow for injecting extra env. variables via ConfigMap
2022-01-26 17:27:19 +01:00
Radoslaw Zarzynski 17ca41f557 osd: allow for injecting extra env. variables via ConfigMap
This patch brings a mechanism to define arbitrary environment
variables in OSD containers (both prepare and main ones).
It bases on idea proposed by Sebastien Han to use an optional
`ConfigMap` instance named `aook-ceph-daemon-env-override`
as the source of these settings.

The need for the patch comes from the fact that, although
crimson finally exposes the same CLI interface as the classical
OSD, the broadly used development builds have ASan built in.
As ASan, by default, complains if it isn't the very first loaded
DSO, we need a way to set the `ASAN_OPTIONS` environment variable
to `verify_asan_link_order=0` to mitigate the early aborts.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-01-26 15:17:27 +00:00
Sébastien Han d6c8512c7e Merge pull request #9654 from leseb/nits
core: fix some nits
2022-01-26 16:06:51 +01:00
Sébastien Han 58c175de47 core: add a break line before printing the ceph config
It's easier to read the config if there is a new line before instead
having a truncated version like:

```
2022-01-26 06:39:41.421373 D | cephclient: config file @ /etc/ceph/ceph.conf: [global]
fsid                = 0650a1d5-d688-4575-97fc-7b3052a8a3dd
mon initial members = a
mon host            = [v2:10.104.192.209:3300,v1:10.104.192.209:6789]

[client.admin]
keyring = /var/lib/rook/rook-ceph/client.admin.keyring
```

Now we have:

```
2022-01-26 06:39:41.421373 D | cephclient: config file @ /etc/ceph/ceph.conf:
[global]
fsid                = 0650a1d5-d688-4575-97fc-7b3052a8a3dd
mon initial members = a
mon host            = [v2:10.104.192.209:3300,v1:10.104.192.209:6789]

[client.admin]
keyring = /var/lib/rook/rook-ceph/client.admin.keyring
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-26 15:19:16 +01:00
Sébastien Han e31e4b49b0 core: print the stdout on errors
If we fail fetching the ceph version let's also print the stdout since
ceph tends to put the error here too.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-26 15:16:22 +01:00
Travis Nielsen 4ef7981b88 Merge pull request #9649 from travisn/check-application-pool
pool: Check for application already being set
2022-01-25 15:39:07 -07:00
Travis Nielsen e0cfb4fce6 pool: check for application already being set
If the pool application is already set, skip setting it again
to avoid a warning message being logged that it is already
set.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-25 15:05:18 -07:00
Travis Nielsen 8fd1f1afe0 Merge pull request #9642 from travisn/helm-lint
helm: Update CI to the latest helm and fix the linter
2022-01-24 16:46:59 -07:00
Travis Nielsen 50afb9f026 helm: update to the latest helm version v3.8
The CI was building with helm 3.6.2, now updating to
the latest v3.8.0

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-24 14:12:22 -07:00
Travis Nielsen b1a0918d0e helm: fix linter issue with end tag
Recent versions of helm are failing the linter on the end tag
if the tag is "{{- end -}}". Instead, the correct end tag
is "{{- end }}".

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-24 14:11:28 -07:00
Blaine Gardner 18c1397bcc Merge pull request #9565 from thotz/rgw-tls-cert-fix-bucket-notifications
rgw: inject tls certs for bucket notification and topic operations
2022-01-24 11:56:09 -07:00
Jiffin Tony Thottan a97747cece rgw: inject tls certs for bucket notification and topic operations
The certs for accessing TLS enabled RGW is saved as secrets and inject
them if controllers for notification and topics if request is sent to
TLS enabled RGW endpoint.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2022-01-24 22:39:57 +05:30
Sébastien Han 16f84677ce Merge pull request #9640 from Juneezee/refactor/t.TempDir
test: use `T.TempDir` to create temporary test directory
2022-01-24 17:59:58 +01:00
Eng Zer Jun ed63571530 test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-24 23:52:54 +08:00