Commit Graph
4559 Commits
Author SHA1 Message Date
Alexander Trost f7ed68dd67 Merge pull request #4876 from galexrt/makefile_auto_doc
makefile: auto generate target help menu
2020-03-27 18:12:27 +01:00
Sébastien Han 6f70c9bd5a Merge pull request #5101 from cybozu-go/check-lv-availability
ceph: check LV availability by "ceph-volume lvm list"
2020-03-27 16:43:48 +01:00
Travis Nielsen cf7a910df6 Merge pull request #5062 from travisn/test-14.2.8
Run Ceph integration tests on latest nautilus
2020-03-27 09:21:28 -06:00
Alexander Trost 976011e923 makefile: auto generate target help menu
This uses grep, sort and awk to auto generate the `make help` menu based
on comments after each target.
Removed `clean.all` and `prune.all` targets as they are not available
anymore.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-03-27 16:07:56 +01:00
Sébastien Han 9c343ae94e Merge pull request #5032 from umangachapagain/config-override
Ceph: add CSI configurations to ConfigMap
2020-03-27 15:21:24 +01:00
Sébastien Han 720b068946 Merge pull request #4697 from umangachapagain/objectUserIntegrationTest
Ceph: integration test creates ObjectStoreUser before ObjectStore
2020-03-27 15:21:09 +01:00
Travis Nielsen 9cecf9e4d5 ceph: better attempt in integration tests to wipe the disks
The integration tests are failing to run when the disks
are not properly cleaned up and the bluestore label
is still found. Now we run sgdisk to more completely
zap the disk.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 07:51:10 -06:00
Travis Nielsen eb1e4beff8 ceph: run integration tests on latest nautilus
Run the tests on latest nautilus with the tag ceph/ceph:v14 so we
are constantly testing on the latest release.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 07:48:54 -06:00
morimoto-cybozu 0c3439c707 ceph: check LV availability by "ceph-volume lvm list"
This commit adds support for LVs to the device availability check
in the OSD prepare pod.
The availability of an LV is checked by "ceph-volume lvm list".
If it returns non-empty result, the LV is in use and not available.

Closes: https://github.com/rook/rook/issues/5075
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2020-03-27 13:30:08 +00:00
Umanga Chapagain ac9244fc1a Ceph: integration test creates ObjectStoreUser before ObjectStore
This fix is added to ensure that everything works as expected
when user tries to create ObjectStoreUser before creating
ObjectStore itself. ObjectStoreUser will wait for ObjectStore
to be up and running.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2020-03-27 18:16:28 +05:30
morimoto-cybozu ab83738aac ceph: fix device path passed to "ceph-volume inventory"
This commit fixes the argument for "ceph-volume inventory".
When a device "/dev/mapper/foo" is being checked for its availability,
the argument should not be "/dev/foo" nor "/dev/dm-1".

Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2020-03-27 11:24:47 +00:00
Umanga Chapagain 0e932c15eb Ceph: add CSI configurations to ConfigMap
This commit adds all the CSI configurations to ConfigMap.
This configMap can be used in combination with Env Vars
to configure Ceph CSI drivers in rook.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2020-03-27 15:17:30 +05:30
Sébastien Han 9dbf86b85e Merge pull request #5069 from rohan47/pending-crashcollector-pod
Remove crashcollector pod automatically when the node is deleted.
2020-03-27 09:56:23 +01:00
Travis Nielsen 444b70e96d Merge pull request #5100 from travisn/go-modules-update
Ensure go.sum is updated with every PR
2020-03-26 16:36:45 -06:00
Travis Nielsen adba744320 build: ensure go.sum is updated with every build
Every developer should commit all the files that need to be changed
in a given set of changes. The go.sum file was not being generated
as expected since the update to go modules. Since the Jenkinsfile
was running make mod.check, the build was ending up with a modified
file, which caused a dirty tag on the build. Now it is expected
that each PR will merge with the corresponding go.sum changes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-26 15:42:06 -06:00
rohan47 3a80d7982c Ceph: Remove crashcollector pod automatically when the node is deleted.
The CrashCollector pod remains in pending state indefinitely after
running on a k8s node that was deleted. The code now deletes the
deployment after the node is deleted.

Signed-off-by: rohan47 <rohgupta@redhat.com>
2020-03-27 02:20:13 +05:30
Travis Nielsen cd33eea36a Merge pull request #5094 from leseb/misc-repo-stuff
Enable mergify for backport commits to release-1.3
2020-03-26 10:59:25 -06:00
Sébastien Han a46c1b3491 mergify: prepare for release-1.3
incoming!

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 17:34:26 +01:00
Sébastien Han c4cd5eaa2e make: remove leftover help flag
This flag is gone since we move to go modules.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 17:34:26 +01:00
Travis Nielsen ac673b03e7 Merge pull request #5093 from leseb/unblock-fs
k8sutil: mitigate UpdateDeploymentAndWait condition to help controllers
2020-03-26 10:06:26 -06:00
Sébastien Han c62d89c7bc k8sutil: mitigate UpdateDeploymentAndWait condition
If a deployment stays in pending we should give early by looking at
ProgressDeadlineExceeded, this will reduce the time to wait from 20 min
to 10 min because ProgressDeadlineExceeded default is 600 seconds.

Prior to this patch we would wait 20min since we take
currentDeployment.Spec.ProgressDeadlineSeconds which is typically 600
then retry every 2 seconds, which makes it 20min total.

Closes: https://github.com/rook/rook/issues/5090
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 14:15:45 +01:00
Sébastien Han 55aee177f3 ceph: don't hardcode ContinueUpgradeAfterChecksEvenIfNotHealthy
Let's just read the given value from the spec.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 14:07:46 +01:00
Sébastien Han 80dd249ecd ceph: mds controller do not set owner ref twice
The owner ref is set right after the deployment is created one level on
top.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 13:00:14 +01:00
Sébastien Han 339a054ce6 Merge pull request #5087 from travisn/pool-model-refactor
Simplify internal pool implementation by removing unnecessary model types
2020-03-26 09:39:01 +01:00
Travis Nielsen 5254de1a8c ceph: simplify pool model to v1 types
The pools had some legacy structs that translated between the
ceph.v1 types used by the CRDs and the internal implementation
of the pools. This simplifies the pool implementation by removing
the intermediate model and leaving us only with the ceph v1
pool types and no unnecessary translation.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-25 16:59:12 -06:00
Sébastien Han f7600066da Merge pull request #5067 from leseb/misc-controller-runtime
Misc controller runtime fixes
2020-03-25 21:31:04 +01:00
Sébastien Han 8db14885b5 ceph: controller fix misleading debug log
They are cases where we let the exponential backoff retry and some where
we force our own retry. Let's properly log this information.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 19:12:17 +01:00
Sébastien Han c84d66de1b ceph: controller, reconcile faster
Let's not wait for the CephCluster to be done reconciling but instead
check for the Ceph cluster status, if it's closed to "ok" then we
proceed so HEALTH_OK and HEALTH_WARN are accepted.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 18:45:19 +01:00
Sébastien Han e57073c4f6 Merge pull request #5080 from cybozu-go/ceph-minor-cleanups
ceph: some minor cleanups
2020-03-25 17:40:11 +01:00
Sébastien Han b5e50481ff ceph: controller: retry after 10sec even if no cluster
Even if there is no CephCluster we still want to retry every 10sec as
one will likely show up soon.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 17:24:30 +01:00
Sébastien Han a044b27650 ci: fix MockExecuteCommandWithOutput mock definition
With the recent exex update we missed this one.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 17:24:30 +01:00
Sébastien Han 7b544f04d4 ceph: reconcile more often when cluster is not ready
Let's not apply the exponential backoff when waiting for the cluster to
be ready, let's only apply it when there is no cluster.

Closes: https://github.com/rook/rook/issues/5059
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 17:24:30 +01:00
Sébastien Han a699512911 ceph: rollback to objectmatcher 1.1.0
With 1.1.1, the object matcher seems not to preserve annotation's order,
this is tracked here: https://github.com/banzaicloud/k8s-objectmatcher/issues/24

1.1.0 does not have that issue, so let's go back with 1.1.0.

Closes: https://github.com/rook/rook/issues/5047
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 17:24:30 +01:00
Sébastien Han 9a772a5b8c rook: bump to the latest stable controller-runtime
Let's use 0.5.1

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-25 17:24:29 +01:00
Satoru Takeuchi 8a13310ddb ceph: Consider the various paths of sgdisk
The path of sgdisk depends on the systems.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-03-25 12:27:14 +00:00
Satoru Takeuchi bb83122d18 ceph: fix a wrong comment
"comment" here should mean the comments of `pg_autoscaler`
in cluster/examples/kubernetes/ceph/cluster.yaml. However, it's
ambiguous. It's straightforward to use "add".

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-03-25 10:37:05 +00:00
Travis Nielsen 006b835711 Merge pull request #5058 from travisn/object-no-pools
Allow creation of object store with pre-existing pools
2020-03-24 09:19:55 -06:00
Sébastien Han b9a6f4cc35 Merge pull request #5078 from umangachapagain/servicemonitor-fix
Ceph: fix updates for servicemonitor
2020-03-24 15:00:15 +01:00
Umanga Chapagain db3735656c Ceph: fix updates for servicemonitor
with older implementation, servicemonitor was not getting
updated due to missing resource version. This fix adds
resource version to the servicemonitor definition and
ensures that the object is properly created or
updated.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2020-03-24 16:30:44 +05:30
Travis Nielsen 5d2db6a9f1 ceph: allow creation of object store with pre-existing pools
The pools may already be created by the mgr module. If the pool specs
are not specified in the object CR, skip pool creation for the
object store. The pools are required to exist and the reconcile
will fail until they do exist.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-23 23:54:20 -06:00
Travis Nielsen 9d3b1cb869 Merge pull request #5074 from cybozu-go/ceph-set-pgnum-of-block-pools-to-the-default-value
ceph: set pgnum of block pools to the default value
2020-03-23 23:27:55 -06:00
Satoru Takeuchi 167cbd7244 ceph: set pgnum of block pools to the default value
Rook should always set the pgnum of the new block pool
to the default value("0"). Current implementation accidentally
works fine because newPool.Number is always 0 here.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-03-24 03:06:25 +00:00
Travis Nielsen 4e4dc94c97 Merge pull request #5072 from travisn/mon-affinity-duplicated-master
ceph: avoid duplication of mon anti-affinity
2020-03-23 17:35:45 -06:00
Travis Nielsen a328f78a73 ceph: avoid duplication of mon anti-affinity
The default pod anti-affinity for the mons that rook adds
automatically intended to be appended to any anti-affinity that
is specified in the cluster CR.

There is a bug in the ApplyToPodSpec() method that has long existed.
The issue is that when antiaffinity is appended, it will append not only
to the pod spec, but will modify the original placement spec. Thus, each
mon that is started will have one more antiaffinity clause than the previous mon.

This condition is rarely hit or noticed because it commonly is only hit
in the canary pods. Since these pods are immediately deleted,
there are no side effects of the duplicate affinity clauses. The place
where it becomes an issue is when the mons are backed by a PVC. In this case,
the mons do not have node affinity and will hit the code path that appends
to the antiaffinity and thus modifies the original antiaffinity.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-23 16:28:09 -06:00
Travis Nielsen da3bf49b61 Merge pull request #5043 from cybozu-go/remove-unnecessary-napespace-fields
remove unnecessary namespace fields
2020-03-20 17:18:21 -06:00
Travis Nielsen 90c8934d31 Merge pull request #5046 from leseb/ganesha-crd-controller-runtime
ceph: convert NFS controller to the controller-runtime
2020-03-20 14:28:42 -06:00
Sébastien Han f64500a954 ceph: failed reconcile if ceph version is not found
This is critical to get this information so that the label can be set
properly and also the configuration done appropriately.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-20 17:38:03 +01:00
Sébastien Han 0e2f84f998 ceph: convert NFS controller to the controller-runtime
The CRD watcher has been replaced by the new controller-runtime
framework.
This brings robustness in our operator, meaning that any resources that
are modified will be reconciled into the desired state.

Closes: https://github.com/rook/rook/issues/4941
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-20 17:38:03 +01:00
Sébastien Han e96dc646a2 rook: add ExecuteCommandWithEnv
We can now execute commands and pass env variables to the executor.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-20 17:37:58 +01:00
Travis Nielsen 9edf205c37 Merge pull request #5035 from leseb/file-crd-controller-runtime
ceph: convert Filesystem controller to the controller-runtime
2020-03-19 18:25:41 -06:00