Commit Graph
4615 Commits
Author SHA1 Message Date
Travis Nielsen a6a7e46bfc Merge pull request #5104 from rohantmp/rbacFix
Fix RBAC and handle error to enable deleting of drain-canaries
2020-04-03 09:17:19 -06:00
Travis Nielsen e84b52d530 Merge pull request #5157 from leseb/canary-liveness
ceph: do not set liveness check on canary pod
2020-04-03 07:42:53 -06:00
Rohan CJ 03e92da102 Ceph: handle not-found error on delete.
Previously, delete was returning an error causing the drain-canary
to reconcile even though it no longer existed.

Signed-off-by: Rohan CJ <rohantmp@gmail.com>
2020-04-03 19:10:45 +05:30
Rohan CJ ecef80eda5 Ceph: Use both "apps" and "extensions" for deployment RBAC.
Some versions of k8s can optionally enable extensions after deprecation.
This could lead to rbac denials when trying to perform operations on deployments

Signed-off-by: Rohan CJ <rohantmp@gmail.com>
2020-04-03 19:10:45 +05:30
Travis Nielsen 3bd9c0eca9 Merge pull request #5158 from sp98/minior-integration-test-changes
Integration Tests: moving GetCleanupPod() and GetCleanupVerificationPod() methods out of CephManifests interface
2020-04-03 07:37:46 -06:00
Sébastien Han 6199861b97 ceph: do not set liveness check on canary pod
The canary pod does not run any daemon so the liveness check probe will
fail.

Closes: https://github.com/rook/rook/issues/5150
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-03 13:23:31 +02:00
Santosh Pillai 1673291589 ceph: Integration Tests: moving GetCleanupPod() and GetCleanupVerificationPod() methods out of CephManifests interface
The CephManifests interface is intended for manifests that install the product (RBAC, CRDs, operator, etc). These cleanup methods are more about test code. The test code needs to behave the same whether it is a normal install or an upgrade. We don't need to version it differently for upgrades between the CephManifestsMaster and CephManifestsV1_1 implementations

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2020-04-03 13:54:52 +05:30
Travis Nielsen 5f44522146 Merge pull request #5134 from sp98/mon-dir-cleanup
Cluster CleanupPolicy: Delete correct mon directories under the dataDirHostPath
2020-04-02 23:48:53 -06:00
Santosh Pillai e1a76bfb96 ceph: Cluster CleanupPolicy: Delete correct mon directories under the dataDirHostPath
In case of multiple clusters, we don't want to delete all the mon directories under the dataDirHostPath during cluster cleanup.
This PR deletes the mon directory only if the montior secret key matches.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2020-04-03 10:01:36 +05:30
Travis Nielsen 843390a4fc Merge pull request #5152 from cybozu-go/ceph-remove-unnecessary-whitespace-in-the-example
ceph: remove unnecessary whitespaces in the example
2020-04-02 16:10:47 -06:00
Satoru Takeuchi e4812f36d0 ceph: remove unnecessary whitespaces in the example
Fix the following error message due to unnecessary whitespaces.

```console
$ kubectl create -f cluster-on-pvc.yaml
error: error parsing cluster-on-pvc.yaml: error converting YAML to JSON: yaml: line 85: mapping values are not allowed in this context
```

Closes: #5149
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-02 21:36:22 +00:00
Travis Nielsen c79325f92c Merge pull request #5138 from cybozu-go/support-topology-spread-constraints
ceph: support topoplogySpreadConstraints
2020-04-02 12:00:45 -06:00
Satoru Takeuchi 06b10e248c ceph: support topoplogySpreadConstraints
Support topologySpreadConstraints to spread OSD pods among failure
domains as even as possible.

https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/

Without this feature, the number of OSD pods would be different
between the failure domains very much. At worst, there is
a possibility that a failure domain has many OSDs and other
failure domains have no OSDs. It can partly be mitigated
with podAntiAffinity. However, it only works if there are at most
one OSD per node. For more detail, please refer to the following
document.

https://github.com/rook/rook/blob/master/cluster/examples/kubernetes/ceph/cluster-on-pvc.yaml#L59

Closes: https://github.com/rook/rook/issues/4387
Signed-off-by: Hiroshi Muraoka <h.muraoka714@gmail.com>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-04-02 16:40:58 +00:00
Travis Nielsen 52ef25827b Merge pull request #5133 from travisn/v1.3-doc-updates
Updated docs need for the Rook v1.3 release
2020-04-02 10:32:55 -06:00
Travis Nielsen b51291af00 Merge pull request #5140 from cloudical-io/go_114
build: allow go 1.14
2020-04-02 10:29:51 -06:00
Travis Nielsen f30c8066d1 docs: update docs for Rook v1.3 release
Various updates are needed for the v1.3 release.
- The pending release notes were missing some features
- Added a section to the upgrade guide for breaking changes to OSDs
- Clarifications around Ceph prereqs and min version
- Other misc clarifications

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-04-02 10:26:00 -06:00
Blaine Gardner 9d41b38095 Ceph: upgrade docs: update upgrade docs for v1.3
Update both documents and `upgrade-from-v1.2*.yaml` files for Rook v1.3
release.

Of note:
- note when the upgrade to CSI driver v2.0 should be updated
- note that CSI image envs can be removed to have Rook use defaults
- update Ceph version upgrade section
- update supported Ceph version info
- update CRDs before upgrade b/c new 'status' fields
- remove special CSI update info left from v1.2

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
(cherry picked from commit 2ccff83d94)
2020-04-02 10:26:00 -06:00
Travis Nielsen 6b38dd3a6a Merge pull request #4933 from leseb/rm-pid-host
ceph: do not use host PID anymore
2020-04-02 07:29:04 -06:00
Sébastien Han 58f1971172 ceph: do not use host PID anymore
Thanks to https://github.com/ceph/ceph/pull/33633, when running on
Octopus, we don't need to use `--pid=host`. This means we are not using
the host PID but use the PID namespace of the pod.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-01 15:42:53 -06:00
Travis Nielsen 14fcbcb151 Merge pull request #5124 from ShyamsundarR/ceph-pool-compression
ceph: Add ability to specify compression mode for pools
2020-04-01 14:29:00 -06:00
ShyamsundarR de97b6a1d6 ceph: Add ability to specify compression mode for pools
This commit enables configuring a compression_mode for pools
that takes the values as specified for bluestore OSD
compression_mode values,
https://docs.ceph.com/docs/master/rados/configuration/bluestore-config-ref/#inline-compression

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-01 14:24:30 -04:00
Alexander Trost a85df12b73 build: allow go 1.14
This adds v1.14 to the supported Golang versions in the makefile.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-04-01 15:11:35 +02:00
ShyamsundarR 7ce1c4fc71 build: Fix ability to build with podman
This commit corrects choosing DOCKERCMD in image.mk file to choose
the environment variable if present, else default to "docker".

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-01 08:19:51 -04:00
Sébastien Han ed37a6a699 Merge pull request #5128 from leseb/liveness-other-daemons
ceph: add liveness probe to mon, mds and osd daemons
2020-04-01 10:53:12 +02:00
Sébastien Han 040193bb5a ceph: remove DaemonType type
This type was a string already and was just making us doing string()
calls all the time to it's not worth it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-01 09:08:18 +02:00
Sébastien Han 7788901a88 ceph: add liveness probe to mon, mds and osd daemons
Now Kubernetes will perform liveness checks on mon, mds and osd daemons.
The command will:

* call the socket (check for existence)
* execute a command and check the return code (success if 0)

This handles the case where the daemon is stuck locally and
unresponsive. It's unlikely but not impossible.
These checks bring more robustness to the implementation.

rbd-mirror and nfs have been leftover for the following reason. The
rbd-mirror socket name is different from other daemons (could be fixed
though): /run/ceph/ceph-client.rbd-mirror.a.1.94362516231272.asok also,
the command to call would need to be changed from "status" to "rbd
mirror status" so we can keep this for a later.
The nfs ganesha has no socket only a PID file which doesn't mean much.
No PID means the process does not run so Kubernetes will already handle
this and the pod will crash loop.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-31 15:20:42 +02:00
Sébastien Han 6062771687 Merge pull request #4901 from travisn/octopus-tests
Run integration tests against octopus
2020-03-31 10:59:11 +02:00
Sébastien Han 04c78135bb Merge pull request #5125 from travisn/ceph-test-versions
Update K8s versions for ceph tests with K8s 1.18
2020-03-31 09:13:28 +02:00
Travis Nielsen ba3e601a50 ceph: update k8s versions for ceph tests on 1.18
With 1.18 in the test matrix we need to update which versions
will run the different ceph test suites.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 16:51:02 -06:00
Travis Nielsen dcc1e2a291 Merge pull request #5117 from travisn/test-k8s-1.18
Test rook on K8s 1.18
v1.3.0-beta.0
2020-03-30 16:48:04 -06:00
Travis Nielsen 9039c674ce Merge pull request #5123 from travisn/dirty-build
Tidy the go modules before the release build
2020-03-30 16:05:24 -06:00
Travis Nielsen 0a5a17cf51 build: tidy the go modules before the release build
The go modules are changed by the install of the junit
tool for the unit tests. Before the release build we need
to ensure that the modules are tidied.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 15:33:28 -06:00
Travis Nielsen 32d6c78e12 Merge pull request #5121 from travisn/git-restore-cmd
Use git checkout instead of git restore
2020-03-30 13:37:12 -06:00
Travis Nielsen 0e55aa348b tests: reduce unhelpful debug logging
Removed some extremely verbose debug logging that makes
the logs difficult to read when debug is enabled.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 13:32:47 -06:00
Travis Nielsen 360e5552b6 ceph: run integration tests against octopus
With the release of Octopus we need to run the integration tests
on Octopus in order to have validation as a supported platform.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 13:32:47 -06:00
Travis Nielsen 928e901ec5 Merge pull request #5113 from leseb/rgw-port-sdn
ceph: use a different port for rgw on sdn
2020-03-30 13:32:30 -06:00
Dmitry Yusupov c2f175e266 Merge pull request #5120 from dyusupov/master
edgefs: TZ needs to be exposed
2020-03-30 12:30:35 -07:00
Travis Nielsen defa687b3d Merge pull request #4959 from sp98/hostPath-cleanUp
Cleanup cluster.Spec.DataDirHostPath on cluster deletion
2020-03-30 13:07:08 -06:00
Travis Nielsen 0e1ccce61a build: use git checkout instead of git restore
The CI is using an older version of git that doesn't
recognize git restore.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 13:05:25 -06:00
Dmitry Yusupov 080510bb6f edgefs: TZ needs to be exposed
/etc/timezone now exposed by operator

Signed-off-by: Dmitry Yusupov <dmitry.yusupov@nexenta.com>
2020-03-30 11:42:45 -07:00
Travis Nielsen f3e1a3407b Merge pull request #5115 from travisn/dirty-build
Check for modified files in build
2020-03-30 12:23:39 -06:00
Travis Nielsen 9a615eef99 build: check for modified files in build
The master build is publishing with the dirty tag so we
need to ensure that the files do not remain modified.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 12:15:53 -06:00
Sébastien Han f136105951 ceph: use a different port for rgw on sdn
When running on SDN, the container is not privileged and the network
stack is not exposed so running the gw on port 80 will not work (not
enough privileged).

Closes: https://github.com/rook/rook/issues/5106
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-30 19:37:11 +02:00
Travis Nielsen a3a7a10236 build: test rook on k8s 1.18
With the release of k8s 1.18 we now will run the integration
tests on 1.14 through 1.18 in master.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 11:14:14 -06:00
Santosh Pillai 2bfd7c42bc ceph: cleanup cluster.Spec.DataDirHostPath on cluster deletion
In order to ensure proper clean up of all the rook-ceph data when the cluster is deleted, we need to clean up the dataDirHostPath (var/lib/rook)

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2020-03-30 19:36:01 +05:30
Elise Gafford 2ca0d8f93d ceph: add cleanupPolicy to cephcluster spec
In order to perform full automation of cephcluster deletion safely
and protect against catastrophic data loss, the end user must be
able to signify that they intend to irrecoverably delete the data
in their cluster. The cleanupPolicy field of the cluster spec is
intended to communicate this. This patch does not implement
automated deletion, but only creates the spec field and the
safety feature of halting orchestration other than deletion on a
cluster with a set cleanup policy value.

Partially-fixes: #3222
Signed-off-by: Elise Gafford <egafford@redhat.com>
2020-03-30 19:11:54 +05:30
Sébastien Han bd850252e5 Merge pull request #5110 from moricho/fix/codegen
Fix `codegen` to find boilerplate outside of the $GOPATH
2020-03-30 15:02:34 +02:00
Sébastien Han bc6186dabe Merge pull request #5108 from AdlerFleurant/operator-openshift-env-fix
ceph: Removed 2nd env variable ROOK_HOSTPATH_REQUIRES_PRIVILEGED
2020-03-30 14:53:40 +02:00
Adler Fleurant 70e7f39db0 ceph: Removed 2nd env variable ROOK_HOSTPATH_REQUIRES_PRIVILEGED
It is not necessary to have 2 same environment variables and not desired as
changing only one may lead to the second variable being applied with unexpected
value. It is better to have only 1 ROOK_HOSTPATH_REQUIRES_PRIVILEGED environment
variable.

Closes: https://github.com/rook/rook/issues/5107
Signed-off-by: Adler Fleurant <Adler.Fleurant@PicoChange.com>
2020-03-30 07:50:30 -04:00
moricho f556add260 codegen: use the bopilerplate for the header of the generated codes
This specifies `--output-base` and `--go-header-file` option at executing `code-generator`.

Signed-off-by: moricho <ikeda.morito@gmail.com>
2020-03-30 17:54:56 +09:00