Commit Graph
13292 Commits
Author SHA1 Message Date
Blaine Gardner a1814af1d9 ceph: remove NFS and Cassandra operator code
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:07:02 -06:00
Blaine Gardner 7a14eb58a0 ceph: remove NFS and Cassandra from codegen
Remove NFS and Cassandra from codegen scripts, BUT leave generated code
to be removed later.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:35 -06:00
Blaine Gardner 34fdcd7175 docs: remove NFS and Cassandra design docs
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:35 -06:00
Blaine Gardner 0508fe3d06 docs: remove NFS and Cassandra refs from docs
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:35 -06:00
Blaine Gardner e684fa09e7 bot: remove NFS and Cassandra mergify/commitlint cfg
Remove NFS and Cassandra references from mergify and commitlint configs.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:34 -06:00
Blaine Gardner 61f6429ea4 test: remove NFS and Cassandra github actions
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:34 -06:00
Blaine Gardner 1b1736a3c5 ceph: remove NFS and Cassandra example manifests
Additionally remove NFS and Cassandra from CRD generator script.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:34 -06:00
Blaine Gardner 9531c252a5 ceph: remove NFS and Cassandra make targets
Remove all make-related references to NFS and Cassandra. This is limited
purely to make targets. Leave CRD and codegen script references to be
removed in a different commit.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:33 -06:00
Travis Nielsen fc699c2a6d Merge pull request #8623 from aruniiird/make-script-backward-compatible-with-python2
Making script backward compatible with python2
2021-08-31 11:54:43 -06:00
Sébastien Han d4dd0577f9 Merge pull request #8529 from sp98/id-mapping
ceph: add ClusterID and PoolID mappings between local and peer cluster
2021-08-31 17:32:14 +02:00
Santosh Pillai 3f8abec403 ceph: add ClusterID and PoolID mappings between local and peer cluster
During disaster recovery/migration of a cluster, as part of the failover, the
kubernetes artifacts like deployment, PVC, PV, etc will be restored to a new
cluster by the admin. Even if the kubernetes objects are restored the
corresponding RBD/CephFS subvolume cannot be retrieved during CSI operations as
the clusterID and poolID are not the same in both clusters

This PR creates a mapping between Cluster ID and RBD Pool ID between
local cluster and peer cluster.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-08-31 20:08:56 +05:30
Sébastien Han 00c111b4a1 Merge pull request #8618 from leseb/use-pointers
ceph: fix vault kv secret engine auto-detection
2021-08-31 16:13:11 +02:00
Arun Kumar Mohan 888d4ea706 ceph: making script backward compatible with python2
Variable name `exec`, was raising an error with python2.
Renamed it to `execErr`.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-08-31 19:40:32 +05:30
Sébastien Han d675969567 ceph: fix vault kv secret engine auto-detection
Passing struct by value essentially gives you a copy, so when modified
within a function, the scope is then reduced to that function. Using
pointers solves that you mutate the struct as many times as you want from
anywhere.
As a result, the auto-detection of the Vault KV backend was not working
correctly.
Also, added a ton of unit tests for Vault.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-31 15:29:04 +02:00
Satoru Takeuchi 58a43a4f23 Merge pull request #8611 from maya-r/typo-docs
docs: Correct field name.
2021-08-31 16:04:34 +09:00
YZ775andHiroya Onoe c38e1f1ddd ceph: avoid duplicate ownerReferences
If we call OwnerInfo.SetOwnerReference for an object multiple times,
it results in OwnerReference duplication.

Signed-off-by: Yuzuki Mimura <yuzuki725.m@gmail.com>
Co-authored-by: Hiroya Onoe <onoehiroya@gmail.com>
2021-08-31 04:58:59 +00:00
Maya Rashish 41988d7eda docs: correct field name
A single entry is a storageClassDeviceSet in the code, but in YAML
the name that appears is the one of the whole array.

Signed-off-by: Maya Rashish <mrashish@redhat.com>
2021-08-30 17:51:36 +03:00
Sébastien Han 67e3372259 Merge pull request #8609 from travisn/set-filesystem-status
ceph: Set the filesystem status when mirroring not enabled
2021-08-30 09:37:32 +02:00
Travis Nielsen 7cfae42a62 ceph: set the filesystem status when mirroring not enabled
When mirroring is enabled on the filesystem, the status was not
being set on the filesystem. Now the reconcile will ensure the
status is updated on the CR whether or not mirroring is enabled.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-27 17:28:25 -06:00
Travis Nielsen 33558fba3d Merge pull request #8603 from travisn/suppress-cassandra-errors
cassandra: Suppress integration test errors temporarily
2021-08-26 16:32:06 -06:00
Travis Nielsen 5489dd7b58 cassandra: suppress integration test errors temporarily
The Cassandra tests are failing in the CI, although they pass when
run locally on a developer cluster. Until the issue can be tracked
down we need to disable these checks to get to a green CI again.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-26 14:34:40 -06:00
Travis Nielsen b5ed5ab573 Merge pull request #8600 from travisn/test-branch-version
ceph: Test against release version in the branch
2021-08-26 13:07:55 -06:00
Travis Nielsen 23b772643b ceph: test against release version in the branch
The release version needs to be the same in the example/test manifests
as it is in the local build image. The github actions are different in
this regard than the Jenkins builds were. The Jenkins builds always
locally used the master tag instead of a release-specific tag.
Now that the github actions use the release-specific tag, the test
framework no longer should be using the master tag in release branches.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-26 12:10:05 -06:00
Blaine Gardner 7b862c1a8d Merge pull request #8506 from subhamkrai/shell-check
ci: add action for shellcheck linter
2021-08-26 09:42:12 -06:00
Travis Nielsen 42717d2fca Merge pull request #8566 from subhamkrai/merge-tolerations
ceph: merge toleration for osd/prepareOSD pod if specified both places
2021-08-26 08:55:27 -06:00
subhamkrai ebe21c8f68 ci: add action for shellcheck linter
we are adding new linter for shellcheck.
As we are writing more shell scripts this
will help maintain quality.

Also, doing all the changes required in
bash files to pass this shellcheck.

Closes: https://github.com/rook/rook/issues/8431
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-26 10:03:32 +05:30
Sébastien Han 9744f0f2c6 Merge pull request #8583 from leseb/cancel-retry
ceph: do not check ok-to-stop when OSDs are in CLBO
2021-08-25 17:38:06 +02:00
Travis Nielsen b528310b79 Merge pull request #8590 from travisn/mon-store-config
ceph: Consolidate the calls to set mon config
2021-08-25 08:31:20 -06:00
Travis Nielsen edb8aa1834 Merge pull request #8584 from parth-gr/string-instance1
core: convert util.NewSet() to sets.NewString()
2021-08-25 08:30:24 -06:00
subhamkrai e1f232eeb7 ceph: merge toleration for osd/prepareOSD pod if specified both places
earlier, `ApplyToPodSpec()` was only taking one toleration and ignoring
tolerations from `placement.ALL()`.

this commit merge toleration for Mgr,Mon,Osd pod
example, for osd it will merge
spec.placement.all and
storageDeviceClassSets.Placement(in case of pvc) or
spec.placement.osd(in case of non-pvc's).

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-25 19:34:28 +05:30
parth-gr b1cfeb9dbe core: removing util.set package files
We are no longer using package util.set
for creating instances,
and instead of that using sets.String package

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-25 13:15:14 +05:30
parth-gr 77ba1ef11b core: convert util.NewSet() to sets.NewString()
Converting util.NewSet() instance to use sets.NewString() instance

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-25 13:13:05 +05:30
Sébastien Han 41e915d411 Merge pull request #8493 from leseb/admission-controller
ceph: move the admission webhook to the operator
2021-08-25 09:30:54 +02:00
Sébastien Han 88048cc4e8 ceph: do not check ok-to-stop when OSDs are in CLBO
This handles the scenario where the OSDs have been created but not yet
started due to a wrong CR configuration.
For instance, when OSDs are encrypted and Vault is used to store
encryption keys, if the KV version is incorrect during the cluster
initialization the OSDs will fail to start and stay in CLBO until the
CR is updated again with the correct KV version so that it can start.

For this scenario, if the CRUSH map has no host registered yet it's fair
to assume the initialization broke and we need to fix it. So when don't
need to call ok-to-stop since it will always fail and eventually force
pass but let's not wait for nothing.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-25 09:30:38 +02:00
Travis Nielsen cdfe1982d1 ceph: consolidate the calls to set mon config
The mon config had two different implementations that have evolved
over the lifetime of the project. This is a simple refactor to remove
the SetConfig() option and stick with the MonStore as a single
implementation for updating the mon store.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-24 14:38:26 -06:00
Travis Nielsen 7af10af781 Merge pull request #8514 from thotz/obcupdateapi
ceph: add support for update() from lib-bucket-provisioner
2021-08-24 13:28:46 -06:00
Sébastien Han 764633216b Merge pull request #8587 from leseb/find-path
core: parse go files only subdirs
2021-08-24 19:11:53 +02:00
Sébastien Han 656dd0f334 ceph: move the admission webhook to the operator
Our admission webhooks will now run as part of the Operator container
and not an additional deployment. This has the advantage of consuming
fewer resources in the cluster and not having to manage affinities and
tolerations. This only drawback is that the Secret containing the
certificates is not mounted anymore and the content needs to be written
inside the Operator. This is not practical since we also need to watch
for the Secret content to change. Meaning that the certificates have
been renewed and the webhook server needs to use them.
A new approach is on its way to hopefully simplify this last issue and
implement a watcher for the Secret.
In the meantime, users need to use the cert-manager or renew
certificates manually. Additionally, they must update the
ValidatingWebhookConfiguration object with the new CA bundle.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 19:07:04 +02:00
Sébastien Han 06a5571149 core: parse go files only subdirs
Instead of using the root let's only lookup our packages. Otherwise we
will format files from ./.work.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 18:52:39 +02:00
Travis Nielsen 9a96afab05 Merge pull request #8582 from Madhu-1/fix-csidriver-panic
ceph: fix panic when recreating the csidriver object
2021-08-24 09:05:31 -06:00
Travis Nielsen 764abc66f0 Merge pull request #8418 from leseb/fix-7609
ceph: embed ceph-csi templates in rook binary
2021-08-24 08:55:41 -06:00
Sébastien Han 579fc27783 ceph: embed ceph-csi templates in rook binary
Thanks to Golang 1.16, we can now embed files in the Go binary. This
means we don't need to add the CSI templates files to the container
image. They are added in the Go binary at build time.

The existing location of the template must be in the package calling it.
So they moved to pkg/operator/ceph/csi/template. All the files have been
symlinked back to cluster/examples/kubernetes/ceph/csi/template.

Closes: https://github.com/rook/rook/issues/7609
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 16:22:20 +02:00
Madhu Rajanna d246dde443 ceph: remove variable declaration
remove csiDriver and csiClient variables
creation and reuse them from the method
receivers.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-24 19:43:18 +05:30
Madhu Rajanna e2dc418371 ceph: fix logging of csidriver
log the successful message about
starting CSIDriver after creating
the daemonset and deployment objects.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-24 19:41:04 +05:30
Sébastien Han 62f06f2d5d Merge pull request #8394 from leseb/build-go-matrix
ci: build on golang 1.16 and 1.17
2021-08-24 16:09:53 +02:00
Sébastien Han cd9754e2f1 core: do not format deepcopy-gen files
deepcopy-gen appends a header starting with `// +build` which makes
gofmt to complain in Go 1.17 so until this is fixed in
https://github.com/kubernetes/gengo/pull/210 let's skip those files for
formatting.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 12:03:17 +02:00
Sébastien Han 7c14b32435 ci: build on golang 1.16 and 1.17
The CI now runs the build test against Golang 1.16 and 1.17.
So 1.17 is been added as a supported version.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 11:18:51 +02:00
Madhu Rajanna d48f67a82c ceph: fix panic in reCreateCSIDriverInfo
Initialize the client and the csidriver object
before calling the reCreateCSIDriverInfo
function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-24 12:15:48 +05:30
Satoru Takeuchi 6582bac8e7 Merge pull request #8441 from leseb/signal
ceph: signal signals handling with context
2021-08-24 14:01:29 +09:00
Travis Nielsen 0e106f7a70 Merge pull request #8571 from parth-gr/legacy-PDB
ceph: Remove legacy PDBs methods
2021-08-20 10:27:35 -06:00