Commit Graph
13292 Commits
Author SHA1 Message Date
binoue 18cde4596c ceph: update helm chart manifests
Currently, helm chart is little outdated and has differences with
example/common.yaml.
This commit modifies both manifests to use same resources.

Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
2020-04-23 14:21:32 +09:00
Obleskar 6a6be9d0c2 Docs: Update link for Minikube PVs
The hyperlink to the Minikube documentation on persistent volumes throws a 404. These docs have moved from https://github.com/kubernetes/minikube/blob/master/docs/persistent_volumes.md to https://github.com/kubernetes/minikube/blob/master/site/content/en/docs/handbook/persistent_volumes.md.

The anchor (#a-note-on-mounts-persistence-and-minikube-hosts) scrolls to the directory paths that Minikube preserves by default.

Alternatively, linking to the rendered site might be more stable.
https://minikube.sigs.k8s.io/docs/handbook/persistent_volumes/

Signed-off-by: Obleskar <benjamindeuson@gmail.com>
2020-04-22 16:47:58 -07:00
Travis Nielsen 7bfa69c76c Merge pull request #5304 from travisn/cluster-lookup
Lookup of the cluster cannot rely on namespace name
2020-04-22 16:20:33 -06:00
Travis Nielsen 8d72e788c1 ceph: lookup of the cluster cannot rely on namespace name
The name of a CephCluster is commonly the same as the namespace,
but not always. When looking up the ceph cluster we can look for
the first one in the namespace rather than requiring the name
to be the same as the namespace.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-22 13:42:40 -06:00
Denys Vitali 5693944737 docs: improve ceph finalizer patching (#1488)
- docs: improve ceph finalizer patching (#1488)
- docs: ceph-teardown: replace double grep w/ awk

Signed-off-by: Denys Vitali <denys.vitali@swisscom.com>
2020-04-22 16:47:14 +02:00
Travis Nielsen 36ebf9d890 Merge pull request #5286 from terorie/patch-1
doc: fix secret env keys on ObjectBucketClaim
2020-04-21 15:46:39 -06:00
Travis Nielsen 9d01f92d2c ceph: restore the cluster-test.yaml example
The cluster-test.yaml example is useful for small test clusters
where only one mon is required and certain settings can be disabled
when not running in production.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-21 15:41:39 -06:00
n.fraison 848d99265b ceph: factorize deletion of rgw CephX key and configuration in centralized mon database
Use same method for the deletion of rgw CephX key and configuration in centralized mon database
when removing a CephObjectStore or scaling down number of rgw

Signed-off-by: n.fraison <n.fraison@criteo.com>
2020-04-21 22:36:29 +02:00
n.fraison 6b9a70cb9a ceph: remove specific rgw configuration on mon configuration database
Remove specific rgw config added to the mon configuration database when deleting a CephObjectstore
or when scaling down number of rgw

Signed-off-by: n.fraison <n.fraison@criteo.com>
2020-04-21 22:36:11 +02:00
Sébastien Han b3fefafbb0 Merge pull request #5284 from travisn/toolbox-job
Toolbox job for running a ceph script
2020-04-21 08:58:54 +02:00
Travis Nielsen f8ffd2c2eb ceph: toolbox job for running a ceph script
As another option to run Ceph commands, a script can be executed
with Ceph commands as if running in the toolbox, but will instead
run in a job where the logs can be collected and analyzed
separately. This would be useful where automation is in place to collect
information about the cluster periodically or upon some failure
that is detected, without requiring an interactive shell.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-20 15:48:59 -06:00
Travis Nielsen c024f01d9a ceph: allow toolbox script to skip watching mons
The toolbox script is used for configuring the toolbox. If the
mons are to fail over while the toolbox is running, the new
list of mons needs to be updated. However, if run temporarily
as a one-time job, the script only needs to initialize the
ceph.conf and then exit.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-20 15:48:44 -06:00
Richard Patel 66feafa9a5 docs: fix secret env keys on ObjectBucketClaim
The documentation on the Ceph ObjectBucketClaim CRD
erroneously states that the generated Secret resource
contains keys `ACCESS_KEY_ID`, `SECRET_ACCESS_KEY`.

According to ceph-object.md, and when mounting the secret
into a container via secretRef, or inspecting it with kubectl,
the variables have an `AWS_` prefix.
Like so: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`.

This commit adds the `AWS_` prefix to both keys.

Signed-off-by: Richard Patel <me@terorie.dev>
2020-04-20 23:37:44 +02:00
Sébastien Han f792cd7c04 Merge pull request #5280 from travisn/mgr-module-async
Configure mgr modules in parallel of rest of reconcile loop
2020-04-20 14:07:21 +02:00
Sébastien Han f93ab8fd2d Merge pull request #5227 from leseb/external-key-priv
ceph: make admin key optional for external cluster
2020-04-20 12:30:16 +02:00
Madhu Rajanna 5684c4cc37 CSI: update cephcsi to v2.1.0
cephcsi v2.1.0 is released and doesnot need any deployment
template changes. This PR updates the cephcsi image
version to v2.1.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-20 14:58:25 +05:30
Sébastien Han 7c994441c3 ceph: make admin key optional for external cluster
Currently, we need to configure the Ceph external Admin keyi
in the Rook deployment to be able to connect to an external Ceph cluster.
If we wanted to run in a multi-tenant fashion
were several K8s/Rook clusters wanted to connect to the same external ceph cluster,
each K8s deployment would have the access to the External Ceph Admin key
and could potentially access or delete the Data
from pools that belong to other k8s/Rook Clusters.

Now the admin key is optional but the helper script create-external-cluster-resources.sh
will help create the necessary keys/users to connect to that cluster.

Closes: https://github.com/rook/rook/issues/4917 and https://github.com/rook/rook/pull/5227
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-20 09:59:04 +02:00
Sébastien Han 58b7b1e012 ceph: stop potential secret leak
If cluster info is not fully initialized we could leak some of the ceph
secrets in the operator logs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-20 09:56:27 +02:00
Travis Nielsen 56cd128eef Merge pull request #5279 from cybozu-go/ceph-allow-to-set-the-device-for-osd-in-the-integration-test
Ceph: make it possible to pass the integration test in the local environment
2020-04-17 19:25:11 -06:00
Travis Nielsen 8c4a8856d6 Merge pull request #5281 from travisn/nautilus-14.2.9
Set defaults to the Ceph v14.2.9 release
2020-04-17 16:25:59 -06:00
Travis Nielsen 348690ea64 ceph: update defaults to the ceph v14.2.9 release
With the release of ceph v14.2.9 we set the default examples to this version.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-17 12:54:06 -06:00
Travis Nielsen 9634574d5b ceph: configure mgr modules in parallel of osds
The mgr modules take some time to configure during every reconcile loop.
The OSDs were waiting until the mgr module config was completed. Since
the mgr module config is optional or at least not critical to the rest
of the reconcile we can do it in parallel.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-17 12:43:53 -06:00
Satoru Takeuchi a860aa445e ceph: describe the desired kernel version in the integration test
The integration test certainly fails with linux kernel v4.4.x, the default
kernel of Ubuntu 16.04. It succeeds with the linux kernel v4.8.x that can be
got by apt-get. Let's declare this as the desired version.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-17 23:24:03 +09:00
Satoru Takeuchi bf27b0f5c4 ceph: allow to set the device for OSD in the integration test
The device name for OSD is fixed to /dev/xvdc. It works fine in the CI.
However, the name would be different in the developer's local environment.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-17 23:24:03 +09:00
Travis Nielsen b465ad4cc2 Merge pull request #5269 from jmolmo/testcluster_refactor
ceph: refactor <StartTestCluster>
2020-04-16 23:23:21 -06:00
Travis Nielsen 1c1591c847 Merge pull request #5261 from cybozu-go/missing-requiresafereplicasize
Ceph: Add requireSafeReplicaSize in CephObjectStore CRD
2020-04-16 10:44:46 -06:00
Travis Nielsen 566530e90e Merge pull request #5263 from surajssd/surajssd/add-seccomp-annotation-to-psp
PSP: Add seccomp annotations
2020-04-16 08:48:23 -06:00
Juan Miguel Olmo Martínez 28afede309 ceph: refactor <StartTestCluster>
Refactor the StartTestCluster method to make it more readable.

I have put most of the parameters used in this function in the struct.
In this way the call to this function is more easy to understand and to use.
The addition of new parameters to this method started to be annoying.

[test ceph]

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2020-04-16 12:03:45 +02:00
Sébastien Han a62c67a00c Merge pull request #5218 from travisn/codegen-pkg
Codegen needs to copy to pkg subfolder
2020-04-16 09:11:04 +02:00
Sébastien Han f824f4fc28 Merge pull request #5267 from cybozu-go/tests-fix-k8s-install-scripts
Tests: fix k8s install scripts
2020-04-16 08:59:25 +02:00
zoetrope babb55cc15 ceph: add requireSafeReplicaSize to CRD
add requireSafeReplicaSize field to CRD in upgrade-from-v1.2-crds.yaml and ceph_manifets.go

Signed-off-by: zoetrope <a.ikezoe@gmail.com>
2020-04-16 15:18:00 +09:00
Satoru Takeuchi 76a2b8c230 tests: prevent the installation failure due to downgrade packages
kubeadm installer tries to install the fixed versions of some packages.
So if the newer versions are already installed, this script fails.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-16 09:18:49 +09:00
Satoru Takeuchi ddf02d3032 tests: make swapon and swapoff reliable
After running `swapoff -a`, systemd re-enable swap soon.
We should stop the corresponding unit to avoid this problem.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-16 09:18:49 +09:00
Travis Nielsen c2878f693e build: codegen needs to copy to pkg subfolder
The codegen output is moving the apis and client folders to the root
folder instead of under pkg.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-15 16:21:50 -06:00
Travis Nielsen 753a5f56a6 Merge pull request #4904 from jmolmo/issue_4342_doc
Ceph:Update Helm Chart documentation
2020-04-15 15:42:46 -06:00
Suraj Deshmukh 2f6205b2f6 PSP: Add seccomp annotations
This commit adds seccomp annotations to the PSP shipped with Rook
Operator. In this way the operator when deployed will be more secure and
will only be allowed the syscalls defined the runtime's seccomp profile.

Signed-off-by: Suraj Deshmukh <surajd.service@gmail.com>
2020-04-15 20:10:30 +05:30
Sébastien Han 5adfc7b43d Merge pull request #5085 from leseb/rbd-mirror-crd
ceph: extract rbd-mirror to its own crd
2020-04-15 10:58:22 +02:00
Sébastien Han 2ad0d344b7 Merge pull request #5256 from travisn/review-cleanup
Reduce verbose logging for ok-to-stop failure
2020-04-15 10:30:08 +02:00
zoetrope f00092de73 ceph: Add requireSafeReplicaSize in CephObjectStore CRD
Add requireSafeReplicaSize field in CephObjectStore CRD.

Closes: https://github.com/rook/rook/issues/5260
Signed-off-by: zoetrope <a.ikezoe@gmail.com>
2020-04-15 17:01:16 +09:00
Sébastien Han 2d06e600c4 Merge pull request #5247 from cybozu-go/update-pg_num-related-documents
ceph: Update pg_num related documents
2020-04-15 09:39:06 +02:00
Sébastien Han 2bd017102b ci: add retry when deleting rbd image
Sometimes the image still has watchers so let's retry the deletion.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-15 09:12:57 +02:00
Sébastien Han d16cebea70 ci: modify upgrade to master
We now install 1.2 by default, then we upgrade from 1.2 to 1.3 and 1.3
to master.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-15 09:12:57 +02:00
Sébastien Han c1b8a7aa9e ceph: extract rbd-mirror to its own crd
Previously, the rbd-mirror daemon was integrated into the `CephCluster`
CRD. This wasn't really practical since we would have to wait for the
whole orchestration to be done to actually set it up. The same goes for
any CR update. Let's say you want to change the number of daemons, Rook
would go through mons, mgrs and osds until it get to rbd-mirror.
This triggers an undesired full orchestration.

With its own CRD this component just gains a lot more flexibility.

Closes: https://github.com/rook/rook/issues/5084
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-15 09:12:56 +02:00
Travis Nielsen 4060167852 ceph: document example for skipping osd creation
When an OSD has been previously configured with ceph-volume, the
OSD will be skipped on the next installation if the device is not
fully wiped. Added an example to the docs to detect this case.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-14 16:12:04 -06:00
Travis Nielsen 668bca27b1 ceph: reduce verbose logging for ok-to-stop failures
When a daemon is not safe-to-stop the operator will print the full
stack trace rather than a simple error message that the daemon
cannot be stopped.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-14 14:53:56 -06:00
Satoru Takeuchi 372e862478 ceph: add the description of too few PGs per OSD warning in the document
Since users often encounter "too few PGs per OSD" warning, it's worth treating
this as a common issue.

Closes: https://github.com/rook/rook/issues/1329
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-14 18:47:02 +00:00
Satoru Takeuchi 6d0ca4892c ceph: remove the redundant description of pg_autoscaler
The description of pg_autoscaler is spread over the two documents.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-14 18:24:07 +00:00
Satoru Takeuchi da48fa517c ceph: add the description of Octopus and the existing pools in pg_autoscaler
pg_autoscaler is enabled by default in Octopus. In addition, enabling this
module don't enable the autoscaler of the existing pools.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-14 18:24:07 +00:00
Travis Nielsen 8c8b74afc3 Merge pull request #4740 from rajatsingh25aug/csiPV-provision
Ceph: PV provisioning fails upon upgrade
2020-04-14 10:35:06 -06:00
Travis Nielsen 0edaf70093 Merge pull request #5251 from travisn/test-env-vars
Convert integration test CLI params to env vars
2020-04-14 10:23:32 -06:00