Commit Graph
4762 Commits
Author SHA1 Message Date
Sébastien Han f27fd207ce ceph: convert the CephCluster controller to the controller-runtime
This is the final conversion to controller-runtime conversion. This time the
CephCluster CRD has been converted to use the controller-runtime
library.
The controller incorporates all the previous watchers too, so the Node
and hot-plug configmap are been watched too.
Only the operator setting configmap is not being watcher since it's not
related to the CephCluster CRD.
Not only the patch converts to controller-runtime but also tries to
re-organize the tree of the repo to actually make the code more
readable and have better functions/methods/tests separations.

Closes: https://github.com/rook/rook/issues/4939
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-28 09:40:35 +02:00
Sébastien Han 008670bc79 Merge pull request #5344 from travisn/doc-unique-crush
Document unique crush name requirement
2020-04-27 19:53:06 +02:00
Blaine Gardner 6c98adee13 Merge pull request #5327 from SUSE/psp-rename-00
ceph: append 00 to rook-privileged PSP name
2020-04-27 13:23:46 -04:00
Travis Nielsen 0bddd562e0 ceph: document unique crush name requirement
CRUSH hierarchy requires unique names at each location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-27 11:09:56 -06:00
Travis Nielsen 10d5631fee Merge pull request #5246 from ashangit/rm_rgw_on_mon_config_db
ceph: remove specific rgw configuration on mon configuration database
2020-04-27 08:20:54 -06:00
Sébastien Han d1a72380bb Merge pull request #5338 from lazybetrayer/fix-log-collection-script
docs: fix log collection script
2020-04-27 09:07:14 +02:00
Wang Zhen c37b5d5a70 docs: fix log collection script
[skip ci]

Signed-off-by: Wang Zhen <lazybetrayer@gmail.com>
2020-04-27 11:50:21 +08:00
n.fraison 9c6ef9c278 ceph: remove specific mds configuration on mon configuration database
Remove specific mds config added to the mon configuration database when deleting a CephObjectstore
or when scaling down number of mds
Also remove mds CephX key

Signed-off-by: n.fraison <n.fraison@criteo.com>
2020-04-26 11:15:27 +02:00
Travis Nielsen 8727807e49 Merge pull request #5321 from cybozu-go/ceph-some-minor-fixes
update documents and cleanup code
2020-04-24 12:57:15 -06:00
Satoru Takeuchi f78efc50aa cleanup: use the constants of environment variables properly
There are some code where constants of environment variables
are defined but not used properly. Let's avoid to use string
literals as possible.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-24 17:31:00 +00:00
Blaine Gardner 2bf340ff0b ceph: append 00 to rook-privileged PSP name
Adding seccomp annotations to the PSP has made Kubernetes start choosing
the recommended Kubernetes "default" PSP over the Rook privileged PSP,
causing operator start errors. Append 00 (double zero) to the beginning
of the PSP name to make sure it is one of the first PSPs checked by
Kubernetes and will be matched before the "default" one.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2020-04-24 10:21:19 -06:00
Satoru Takeuchi 2d6f0ef2ff docs: convert unordered lists to ordered lists if appropriate
Some workflows are formatted as unordered lists. It's better to convert
these lists to ordered ones.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-24 13:02:34 +00:00
Satoru Takeuchi afb6dbe1ed ceph: fix a wrong comment and a warning message about populating device info
PopulateDeviceInfo may fail for reasons other than lsblk failure.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-24 13:00:15 +00:00
Travis Nielsen 382c6c3610 Merge pull request #5322 from travisn/metadata-osd
Metadatadevice should not be added to osd data device list
2020-04-23 15:03:15 -06:00
Travis Nielsen af09c308df Merge pull request #5297 from binoue/update-helm-chart
ceph: update helm chart manifests
2020-04-23 13:40:50 -06:00
Travis Nielsen 99dc363ab6 ceph: metadatadevice to not be added to osd data device list
When the metadataDevice is specified, it was a regression that it was
being interpreted also as a data device. ceph-volume was failing
when the metadataDevice was added as both a data and metadata device.
So we ensure it is not added to the list of data devices.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-23 12:40:11 -06:00
Travis Nielsen 946673f3e1 Merge pull request #5305 from travisn/cluster-test-example
Restore the cluster-test.yaml example
2020-04-23 10:16:35 -06:00
Sébastien Han bb0a1565a4 Merge pull request #5290 from Obleskar/patch-2
Docs: Update link for Minikube PVs
2020-04-23 18:16:19 +02:00
Sébastien Han 75357b8516 Merge pull request #5288 from denysvitali/master
docs: improve ceph finalizer patching (#1488)
2020-04-23 18:15:00 +02:00
Travis Nielsen 5789c2b197 Merge pull request #5289 from Madhu-1/csi-v2.1.0
CSI: update cephcsi to v2.1.0
2020-04-23 07:49:23 -06:00
binoue 18cde4596c ceph: update helm chart manifests
Currently, helm chart is little outdated and has differences with
example/common.yaml.
This commit modifies both manifests to use same resources.

Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
2020-04-23 14:21:32 +09:00
Obleskar 6a6be9d0c2 Docs: Update link for Minikube PVs
The hyperlink to the Minikube documentation on persistent volumes throws a 404. These docs have moved from https://github.com/kubernetes/minikube/blob/master/docs/persistent_volumes.md to https://github.com/kubernetes/minikube/blob/master/site/content/en/docs/handbook/persistent_volumes.md.

The anchor (#a-note-on-mounts-persistence-and-minikube-hosts) scrolls to the directory paths that Minikube preserves by default.

Alternatively, linking to the rendered site might be more stable.
https://minikube.sigs.k8s.io/docs/handbook/persistent_volumes/

Signed-off-by: Obleskar <benjamindeuson@gmail.com>
2020-04-22 16:47:58 -07:00
Travis Nielsen 7bfa69c76c Merge pull request #5304 from travisn/cluster-lookup
Lookup of the cluster cannot rely on namespace name
2020-04-22 16:20:33 -06:00
Travis Nielsen 8d72e788c1 ceph: lookup of the cluster cannot rely on namespace name
The name of a CephCluster is commonly the same as the namespace,
but not always. When looking up the ceph cluster we can look for
the first one in the namespace rather than requiring the name
to be the same as the namespace.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-22 13:42:40 -06:00
Denys Vitali 5693944737 docs: improve ceph finalizer patching (#1488)
- docs: improve ceph finalizer patching (#1488)
- docs: ceph-teardown: replace double grep w/ awk

Signed-off-by: Denys Vitali <denys.vitali@swisscom.com>
2020-04-22 16:47:14 +02:00
Travis Nielsen 36ebf9d890 Merge pull request #5286 from terorie/patch-1
doc: fix secret env keys on ObjectBucketClaim
2020-04-21 15:46:39 -06:00
Travis Nielsen 9d01f92d2c ceph: restore the cluster-test.yaml example
The cluster-test.yaml example is useful for small test clusters
where only one mon is required and certain settings can be disabled
when not running in production.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-21 15:41:39 -06:00
n.fraison 848d99265b ceph: factorize deletion of rgw CephX key and configuration in centralized mon database
Use same method for the deletion of rgw CephX key and configuration in centralized mon database
when removing a CephObjectStore or scaling down number of rgw

Signed-off-by: n.fraison <n.fraison@criteo.com>
2020-04-21 22:36:29 +02:00
n.fraison 6b9a70cb9a ceph: remove specific rgw configuration on mon configuration database
Remove specific rgw config added to the mon configuration database when deleting a CephObjectstore
or when scaling down number of rgw

Signed-off-by: n.fraison <n.fraison@criteo.com>
2020-04-21 22:36:11 +02:00
Sébastien Han b3fefafbb0 Merge pull request #5284 from travisn/toolbox-job
Toolbox job for running a ceph script
2020-04-21 08:58:54 +02:00
Travis Nielsen f8ffd2c2eb ceph: toolbox job for running a ceph script
As another option to run Ceph commands, a script can be executed
with Ceph commands as if running in the toolbox, but will instead
run in a job where the logs can be collected and analyzed
separately. This would be useful where automation is in place to collect
information about the cluster periodically or upon some failure
that is detected, without requiring an interactive shell.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-20 15:48:59 -06:00
Travis Nielsen c024f01d9a ceph: allow toolbox script to skip watching mons
The toolbox script is used for configuring the toolbox. If the
mons are to fail over while the toolbox is running, the new
list of mons needs to be updated. However, if run temporarily
as a one-time job, the script only needs to initialize the
ceph.conf and then exit.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-20 15:48:44 -06:00
Richard Patel 66feafa9a5 docs: fix secret env keys on ObjectBucketClaim
The documentation on the Ceph ObjectBucketClaim CRD
erroneously states that the generated Secret resource
contains keys `ACCESS_KEY_ID`, `SECRET_ACCESS_KEY`.

According to ceph-object.md, and when mounting the secret
into a container via secretRef, or inspecting it with kubectl,
the variables have an `AWS_` prefix.
Like so: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`.

This commit adds the `AWS_` prefix to both keys.

Signed-off-by: Richard Patel <me@terorie.dev>
2020-04-20 23:37:44 +02:00
Sébastien Han f792cd7c04 Merge pull request #5280 from travisn/mgr-module-async
Configure mgr modules in parallel of rest of reconcile loop
2020-04-20 14:07:21 +02:00
Sébastien Han f93ab8fd2d Merge pull request #5227 from leseb/external-key-priv
ceph: make admin key optional for external cluster
2020-04-20 12:30:16 +02:00
Madhu Rajanna 5684c4cc37 CSI: update cephcsi to v2.1.0
cephcsi v2.1.0 is released and doesnot need any deployment
template changes. This PR updates the cephcsi image
version to v2.1.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-20 14:58:25 +05:30
Sébastien Han 7c994441c3 ceph: make admin key optional for external cluster
Currently, we need to configure the Ceph external Admin keyi
in the Rook deployment to be able to connect to an external Ceph cluster.
If we wanted to run in a multi-tenant fashion
were several K8s/Rook clusters wanted to connect to the same external ceph cluster,
each K8s deployment would have the access to the External Ceph Admin key
and could potentially access or delete the Data
from pools that belong to other k8s/Rook Clusters.

Now the admin key is optional but the helper script create-external-cluster-resources.sh
will help create the necessary keys/users to connect to that cluster.

Closes: https://github.com/rook/rook/issues/4917 and https://github.com/rook/rook/pull/5227
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-20 09:59:04 +02:00
Sébastien Han 58b7b1e012 ceph: stop potential secret leak
If cluster info is not fully initialized we could leak some of the ceph
secrets in the operator logs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-20 09:56:27 +02:00
Travis Nielsen 56cd128eef Merge pull request #5279 from cybozu-go/ceph-allow-to-set-the-device-for-osd-in-the-integration-test
Ceph: make it possible to pass the integration test in the local environment
2020-04-17 19:25:11 -06:00
Travis Nielsen 8c4a8856d6 Merge pull request #5281 from travisn/nautilus-14.2.9
Set defaults to the Ceph v14.2.9 release
2020-04-17 16:25:59 -06:00
Travis Nielsen 348690ea64 ceph: update defaults to the ceph v14.2.9 release
With the release of ceph v14.2.9 we set the default examples to this version.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-17 12:54:06 -06:00
Travis Nielsen 9634574d5b ceph: configure mgr modules in parallel of osds
The mgr modules take some time to configure during every reconcile loop.
The OSDs were waiting until the mgr module config was completed. Since
the mgr module config is optional or at least not critical to the rest
of the reconcile we can do it in parallel.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-17 12:43:53 -06:00
Satoru Takeuchi a860aa445e ceph: describe the desired kernel version in the integration test
The integration test certainly fails with linux kernel v4.4.x, the default
kernel of Ubuntu 16.04. It succeeds with the linux kernel v4.8.x that can be
got by apt-get. Let's declare this as the desired version.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-17 23:24:03 +09:00
Satoru Takeuchi bf27b0f5c4 ceph: allow to set the device for OSD in the integration test
The device name for OSD is fixed to /dev/xvdc. It works fine in the CI.
However, the name would be different in the developer's local environment.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-17 23:24:03 +09:00
Travis Nielsen b465ad4cc2 Merge pull request #5269 from jmolmo/testcluster_refactor
ceph: refactor <StartTestCluster>
2020-04-16 23:23:21 -06:00
Travis Nielsen 1c1591c847 Merge pull request #5261 from cybozu-go/missing-requiresafereplicasize
Ceph: Add requireSafeReplicaSize in CephObjectStore CRD
2020-04-16 10:44:46 -06:00
Travis Nielsen 566530e90e Merge pull request #5263 from surajssd/surajssd/add-seccomp-annotation-to-psp
PSP: Add seccomp annotations
2020-04-16 08:48:23 -06:00
Juan Miguel Olmo Martínez 28afede309 ceph: refactor <StartTestCluster>
Refactor the StartTestCluster method to make it more readable.

I have put most of the parameters used in this function in the struct.
In this way the call to this function is more easy to understand and to use.
The addition of new parameters to this method started to be annoying.

[test ceph]

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2020-04-16 12:03:45 +02:00
Sébastien Han a62c67a00c Merge pull request #5218 from travisn/codegen-pkg
Codegen needs to copy to pkg subfolder
2020-04-16 09:11:04 +02:00
Sébastien Han f824f4fc28 Merge pull request #5267 from cybozu-go/tests-fix-k8s-install-scripts
Tests: fix k8s install scripts
2020-04-16 08:59:25 +02:00