Commit Graph
4588 Commits
Author SHA1 Message Date
Sébastien Han 040193bb5a ceph: remove DaemonType type
This type was a string already and was just making us doing string()
calls all the time to it's not worth it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-04-01 09:08:18 +02:00
Sébastien Han 7788901a88 ceph: add liveness probe to mon, mds and osd daemons
Now Kubernetes will perform liveness checks on mon, mds and osd daemons.
The command will:

* call the socket (check for existence)
* execute a command and check the return code (success if 0)

This handles the case where the daemon is stuck locally and
unresponsive. It's unlikely but not impossible.
These checks bring more robustness to the implementation.

rbd-mirror and nfs have been leftover for the following reason. The
rbd-mirror socket name is different from other daemons (could be fixed
though): /run/ceph/ceph-client.rbd-mirror.a.1.94362516231272.asok also,
the command to call would need to be changed from "status" to "rbd
mirror status" so we can keep this for a later.
The nfs ganesha has no socket only a PID file which doesn't mean much.
No PID means the process does not run so Kubernetes will already handle
this and the pod will crash loop.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-31 15:20:42 +02:00
Sébastien Han 04c78135bb Merge pull request #5125 from travisn/ceph-test-versions
Update K8s versions for ceph tests with K8s 1.18
2020-03-31 09:13:28 +02:00
Travis Nielsen ba3e601a50 ceph: update k8s versions for ceph tests on 1.18
With 1.18 in the test matrix we need to update which versions
will run the different ceph test suites.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 16:51:02 -06:00
Travis Nielsen dcc1e2a291 Merge pull request #5117 from travisn/test-k8s-1.18
Test rook on K8s 1.18
v1.3.0-beta.0
2020-03-30 16:48:04 -06:00
Travis Nielsen 9039c674ce Merge pull request #5123 from travisn/dirty-build
Tidy the go modules before the release build
2020-03-30 16:05:24 -06:00
Travis Nielsen 0a5a17cf51 build: tidy the go modules before the release build
The go modules are changed by the install of the junit
tool for the unit tests. Before the release build we need
to ensure that the modules are tidied.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 15:33:28 -06:00
Travis Nielsen 32d6c78e12 Merge pull request #5121 from travisn/git-restore-cmd
Use git checkout instead of git restore
2020-03-30 13:37:12 -06:00
Travis Nielsen 928e901ec5 Merge pull request #5113 from leseb/rgw-port-sdn
ceph: use a different port for rgw on sdn
2020-03-30 13:32:30 -06:00
Dmitry Yusupov c2f175e266 Merge pull request #5120 from dyusupov/master
edgefs: TZ needs to be exposed
2020-03-30 12:30:35 -07:00
Travis Nielsen defa687b3d Merge pull request #4959 from sp98/hostPath-cleanUp
Cleanup cluster.Spec.DataDirHostPath on cluster deletion
2020-03-30 13:07:08 -06:00
Travis Nielsen 0e1ccce61a build: use git checkout instead of git restore
The CI is using an older version of git that doesn't
recognize git restore.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 13:05:25 -06:00
Dmitry Yusupov 080510bb6f edgefs: TZ needs to be exposed
/etc/timezone now exposed by operator

Signed-off-by: Dmitry Yusupov <dmitry.yusupov@nexenta.com>
2020-03-30 11:42:45 -07:00
Travis Nielsen f3e1a3407b Merge pull request #5115 from travisn/dirty-build
Check for modified files in build
2020-03-30 12:23:39 -06:00
Travis Nielsen 9a615eef99 build: check for modified files in build
The master build is publishing with the dirty tag so we
need to ensure that the files do not remain modified.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 12:15:53 -06:00
Sébastien Han f136105951 ceph: use a different port for rgw on sdn
When running on SDN, the container is not privileged and the network
stack is not exposed so running the gw on port 80 will not work (not
enough privileged).

Closes: https://github.com/rook/rook/issues/5106
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-30 19:37:11 +02:00
Travis Nielsen a3a7a10236 build: test rook on k8s 1.18
With the release of k8s 1.18 we now will run the integration
tests on 1.14 through 1.18 in master.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-30 11:14:14 -06:00
Santosh Pillai 2bfd7c42bc ceph: cleanup cluster.Spec.DataDirHostPath on cluster deletion
In order to ensure proper clean up of all the rook-ceph data when the cluster is deleted, we need to clean up the dataDirHostPath (var/lib/rook)

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2020-03-30 19:36:01 +05:30
Elise Gafford 2ca0d8f93d ceph: add cleanupPolicy to cephcluster spec
In order to perform full automation of cephcluster deletion safely
and protect against catastrophic data loss, the end user must be
able to signify that they intend to irrecoverably delete the data
in their cluster. The cleanupPolicy field of the cluster spec is
intended to communicate this. This patch does not implement
automated deletion, but only creates the spec field and the
safety feature of halting orchestration other than deletion on a
cluster with a set cleanup policy value.

Partially-fixes: #3222
Signed-off-by: Elise Gafford <egafford@redhat.com>
2020-03-30 19:11:54 +05:30
Sébastien Han bd850252e5 Merge pull request #5110 from moricho/fix/codegen
Fix `codegen` to find boilerplate outside of the $GOPATH
2020-03-30 15:02:34 +02:00
Sébastien Han bc6186dabe Merge pull request #5108 from AdlerFleurant/operator-openshift-env-fix
ceph: Removed 2nd env variable ROOK_HOSTPATH_REQUIRES_PRIVILEGED
2020-03-30 14:53:40 +02:00
Adler Fleurant 70e7f39db0 ceph: Removed 2nd env variable ROOK_HOSTPATH_REQUIRES_PRIVILEGED
It is not necessary to have 2 same environment variables and not desired as
changing only one may lead to the second variable being applied with unexpected
value. It is better to have only 1 ROOK_HOSTPATH_REQUIRES_PRIVILEGED environment
variable.

Closes: https://github.com/rook/rook/issues/5107
Signed-off-by: Adler Fleurant <Adler.Fleurant@PicoChange.com>
2020-03-30 07:50:30 -04:00
moricho f556add260 codegen: use the bopilerplate for the header of the generated codes
This specifies `--output-base` and `--go-header-file` option at executing `code-generator`.

Signed-off-by: moricho <ikeda.morito@gmail.com>
2020-03-30 17:54:56 +09:00
Travis Nielsen a50bab382a Merge pull request #5105 from travisn/flex-test-enable-agent
Reenable flex driver in the integration tests
2020-03-27 19:59:46 -06:00
Travis Nielsen bef11804b8 flex: enable flex driver in the integration tests
The flex agent was mistakenly disabled in the CSI config change
in this PR: https://github.com/rook/rook/pull/5032. The agent must
be enabled for integration tests.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 16:26:22 -06:00
Travis Nielsen 90e0c44e72 Merge pull request #5102 from travisn/test-dirty-build
Ensure files are not modified during build
2020-03-27 14:32:26 -06:00
Travis Nielsen 816d8286e4 build: ensure files are not modified during build
The recent change to go modules is causing a file to change
during the build, which results in a dirty tag to be
generated.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 14:14:15 -06:00
Travis Nielsen 7bbc559e86 Merge pull request #5098 from travisn/metadata-pool-pgs-master
Set pg_num for rgw metadata pools to lower default
2020-03-27 13:02:24 -06:00
Alexander Trost f7ed68dd67 Merge pull request #4876 from galexrt/makefile_auto_doc
makefile: auto generate target help menu
2020-03-27 18:12:27 +01:00
Sébastien Han 6f70c9bd5a Merge pull request #5101 from cybozu-go/check-lv-availability
ceph: check LV availability by "ceph-volume lvm list"
2020-03-27 16:43:48 +01:00
Travis Nielsen cf7a910df6 Merge pull request #5062 from travisn/test-14.2.8
Run Ceph integration tests on latest nautilus
2020-03-27 09:21:28 -06:00
Alexander Trost 976011e923 makefile: auto generate target help menu
This uses grep, sort and awk to auto generate the `make help` menu based
on comments after each target.
Removed `clean.all` and `prune.all` targets as they are not available
anymore.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-03-27 16:07:56 +01:00
Travis Nielsen e39637930d ceph: set pg_num for rgw metadata pools to lower default
The PG count on metadata pools should default to rgw_rados_pool_pg_num_min
instead of the more general default pg count. This means rgw pools
will default to 8 PGs instead of 32 PGs, which means a lot more pools
can be created before hitting the default PG limit.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 09:06:59 -06:00
Sébastien Han 9c343ae94e Merge pull request #5032 from umangachapagain/config-override
Ceph: add CSI configurations to ConfigMap
2020-03-27 15:21:24 +01:00
Sébastien Han 720b068946 Merge pull request #4697 from umangachapagain/objectUserIntegrationTest
Ceph: integration test creates ObjectStoreUser before ObjectStore
2020-03-27 15:21:09 +01:00
Travis Nielsen 9cecf9e4d5 ceph: better attempt in integration tests to wipe the disks
The integration tests are failing to run when the disks
are not properly cleaned up and the bluestore label
is still found. Now we run sgdisk to more completely
zap the disk.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 07:51:10 -06:00
Travis Nielsen eb1e4beff8 ceph: run integration tests on latest nautilus
Run the tests on latest nautilus with the tag ceph/ceph:v14 so we
are constantly testing on the latest release.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-27 07:48:54 -06:00
morimoto-cybozu 0c3439c707 ceph: check LV availability by "ceph-volume lvm list"
This commit adds support for LVs to the device availability check
in the OSD prepare pod.
The availability of an LV is checked by "ceph-volume lvm list".
If it returns non-empty result, the LV is in use and not available.

Closes: https://github.com/rook/rook/issues/5075
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2020-03-27 13:30:08 +00:00
Umanga Chapagain ac9244fc1a Ceph: integration test creates ObjectStoreUser before ObjectStore
This fix is added to ensure that everything works as expected
when user tries to create ObjectStoreUser before creating
ObjectStore itself. ObjectStoreUser will wait for ObjectStore
to be up and running.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2020-03-27 18:16:28 +05:30
morimoto-cybozu ab83738aac ceph: fix device path passed to "ceph-volume inventory"
This commit fixes the argument for "ceph-volume inventory".
When a device "/dev/mapper/foo" is being checked for its availability,
the argument should not be "/dev/foo" nor "/dev/dm-1".

Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2020-03-27 11:24:47 +00:00
Umanga Chapagain 0e932c15eb Ceph: add CSI configurations to ConfigMap
This commit adds all the CSI configurations to ConfigMap.
This configMap can be used in combination with Env Vars
to configure Ceph CSI drivers in rook.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2020-03-27 15:17:30 +05:30
Sébastien Han 9dbf86b85e Merge pull request #5069 from rohan47/pending-crashcollector-pod
Remove crashcollector pod automatically when the node is deleted.
2020-03-27 09:56:23 +01:00
Travis Nielsen 444b70e96d Merge pull request #5100 from travisn/go-modules-update
Ensure go.sum is updated with every PR
2020-03-26 16:36:45 -06:00
Travis Nielsen adba744320 build: ensure go.sum is updated with every build
Every developer should commit all the files that need to be changed
in a given set of changes. The go.sum file was not being generated
as expected since the update to go modules. Since the Jenkinsfile
was running make mod.check, the build was ending up with a modified
file, which caused a dirty tag on the build. Now it is expected
that each PR will merge with the corresponding go.sum changes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-03-26 15:42:06 -06:00
rohan47 3a80d7982c Ceph: Remove crashcollector pod automatically when the node is deleted.
The CrashCollector pod remains in pending state indefinitely after
running on a k8s node that was deleted. The code now deletes the
deployment after the node is deleted.

Signed-off-by: rohan47 <rohgupta@redhat.com>
2020-03-27 02:20:13 +05:30
Travis Nielsen cd33eea36a Merge pull request #5094 from leseb/misc-repo-stuff
Enable mergify for backport commits to release-1.3
2020-03-26 10:59:25 -06:00
Sébastien Han a46c1b3491 mergify: prepare for release-1.3
incoming!

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 17:34:26 +01:00
Sébastien Han c4cd5eaa2e make: remove leftover help flag
This flag is gone since we move to go modules.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 17:34:26 +01:00
Travis Nielsen ac673b03e7 Merge pull request #5093 from leseb/unblock-fs
k8sutil: mitigate UpdateDeploymentAndWait condition to help controllers
2020-03-26 10:06:26 -06:00
Sébastien Han c62d89c7bc k8sutil: mitigate UpdateDeploymentAndWait condition
If a deployment stays in pending we should give early by looking at
ProgressDeadlineExceeded, this will reduce the time to wait from 20 min
to 10 min because ProgressDeadlineExceeded default is 600 seconds.

Prior to this patch we would wait 20min since we take
currentDeployment.Spec.ProgressDeadlineSeconds which is typically 600
then retry every 2 seconds, which makes it 20min total.

Closes: https://github.com/rook/rook/issues/5090
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-03-26 14:15:45 +01:00