Commit Graph
7961 Commits
Author SHA1 Message Date
subhamkrai cb0ca66a6b ci: enable gosec linter in golangci-lint
golangci-lint linter gosec showing more errors
than gosec gh. This commit resolve
new errors. And, removing
nosec comments from autogenerated files.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-10-19 11:50:06 +05:30
Umanga Chapagain 6bb76189ad build: fixes "requires" for kube-controller-manager
using v0.0.0 is causing issues with unknown version even
though the replace block uses correct version. So updating
requires block to match replaces block version.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2020-10-19 11:24:41 +05:30
Ahmad Nurus S 72bf7873c9 nfs: implement disk quota
Signed-off-by: Ahmad Nurus S <prksu.sh@gmail.com>
2020-10-19 10:20:12 +07:00
Alexander Trost 268329709e Merge pull request #6440 from prksu/update-rook-nfs-doc
nfs: update documentation and example
2020-10-18 01:33:28 +02:00
Travis Nielsen ff23835bd8 Merge pull request #6445 from cybozu-go/add-a-maintainer
ceph: add a maintainer
2020-10-16 09:14:11 -06:00
MG 1063c387f9 docs: ceph disaster recovery
updated by removing the paranthesis around the arguments  to pass the value
Eg: ROOK_CEPH_MON_INITIAL_MEMBERS

Signed-off-by: MG <gmadhavi@gmail.com>
2020-10-16 23:03:11 +08:00
Satoru Takeuchi 625f3ca4b4 ceph: add a maintainer
Add a new Ceph maintainer.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-15 22:59:01 +00:00
Satoru Takeuchi a188a80289 ci: clarify the supported vm-driver of minikube scripts
We can't use vm-drivers other than "docker" or "none" in minikube
scripts. It's because there is no device for OSD.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-15 20:11:31 +00:00
Travis Nielsen 662bde6d4a Merge pull request #6437 from cybozu-go/ceph-the-number-of-provisioner-pods
ceph: reduce the number of provisioner pods in small K8s cluster
2020-10-15 13:23:08 -06:00
Satoru Takeuchi e22bedbd9b ceph: reduce the number of provisioner pods in small K8s cluster
The default number of provisioner pods are 2. When there is only one node,
for example the CI environment, one of these becomes Pending state.
Although it's harmless, it's better to reduce this value to 1.

Closes: https://github.com/rook/rook/issues/6294

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-15 18:25:27 +00:00
Ahmad Nurus S c22189ba74 nfs: update documentation and example
Signed-off-by: Ahmad Nurus S <prksu.sh@gmail.com>
2020-10-16 01:11:44 +07:00
Sébastien Han 4181338c7a Merge pull request #5994 from tenzen-y/fix-basetestdir-function
ci: cleanup base directory code
2020-10-15 18:52:51 +02:00
tenzen-y 356f21569f ci: cleanup base directory code
Fix the following minor bugs.

- The wrong description of TEST_BASE_DIR environment variable.
- baseTestDir() don't handle error properly

Signed-off-by: tenzen-y <toyonomajyutushi@yahoo.co.jp>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-15 14:14:23 +00:00
Sébastien Han f0b2c6eacf Merge pull request #6184 from cybozu-go/ceph-raw-mode-on-lv-backed-pvc
ceph: raw mode OSD on LV-backed PVC
2020-10-15 09:46:56 +02:00
Sébastien Han 0dca391659 Merge pull request #5991 from guni1192/ci-fix-helm-path
ci: fix helm path
2020-10-15 09:45:52 +02:00
Takashi IIGUNI 27c7146526 ci: simplify helm path setting
Download the pre-defined helm by default to make the integration test simpler.
As a result, we can make PATH/TEST_HELM_PATH environment variable optional.
In addition, the default helm path can be simplified that doesn't include arch and os.

Signed-off-by: Takashi IIGUNI <iiguni.tks@gmail.com>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-15 10:09:00 +09:00
Daichi Sakaue b9f29349ed ceph: use raw mode instead of lvm mode even if lv-backed PVC
It can simplify OSD management in many ways and can avoid many problems
that come from LVM.

Closes: https://github.com/rook/rook/issues/5939
Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-14 01:46:25 +00:00
Sébastien Han 4736ddcc87 Merge pull request #6379 from leseb/fix-5591
ceph: add support for stretched cluster crush rule
2020-10-13 17:40:01 +02:00
Sébastien Han 86bcfa744b Merge pull request #6213 from leseb/gh-action-minikube
ci: add integration test for external python script
2020-10-13 16:55:33 +02:00
Sébastien Han 7afa0e125c Merge pull request #6362 from travisn/stretch-cluster-design
ceph: Design doc for stretch clusters
2020-10-13 16:51:34 +02:00
Sébastien Han b70b098405 ceph: add support for stretched cluster crush rule
The pool spec has now a new property called "replicasPerFailureDomain"
which essentially represents the number of replicas to store in each
failure domain.
Assuming the failure domain is a datacenter (if the cluster is
stretched) then you will have 2 replicas per datacenter where each
replica ends up on a different host. This gives you a total of 4
replicas and for this, the "size" must be set to 4.

Closes: https://github.com/rook/rook/issues/5591
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-13 16:33:34 +02:00
Sébastien Han bf9eb38f2a ci: add integration test for external python script
The CI runs a github action that builds and deploys a Rook cluster and
then execute various tests on the create-external-cluster-resources.py
script.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-13 16:31:59 +02:00
Sébastien Han 6c73493f8a Merge pull request #6420 from travisn/disallow-two-mons
ceph: Require an odd number of mons and sufficient nodes for mons
2020-10-13 16:10:19 +02:00
Travis Nielsen fe602e0902 Merge pull request #6426 from leseb/add-cv-log-to-non-pvc
ceph: add ceph-volume log to non-pvc scenario
2020-10-13 07:48:47 -06:00
Travis Nielsen 23e4b755b4 ceph: design doc for stretch clusters
Design document to support stretch clusters with an arbiter mon
based on the new design for stretch clusters in Ceph.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-12 15:33:33 -06:00
Travis Nielsen a4b1ca3faa docs: clarify cluster-on-pvc scenario for ceph
A frequently asked question is how to configure the cluster in dynamic
environments, or basically to use the cluster-on-pvc configuration.
This adds more details to the examples in the docs.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-12 15:21:28 -06:00
Travis Nielsen c7e0338d5e ceph: require an odd number of mons and sufficient nodes
The ceph mons should never be started with an even number in quorum.
The desired state should always be an odd number of mons to ensure
a healthy majority quorum. The operator now rejects a request for
an even number of mons.

If there are not a sufficient number of nodes for mons to be on a
unique host, the operator also returns an early error instead of
getting stuck waiting for a pending mon. This is not foolproof since
there might be fewer available nodes for the mons, but at least it
is a quick check for the common case.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-12 15:21:27 -06:00
Travis Nielsen ede7808960 Merge pull request #6421 from cybozu-go/ci-update-k8s-version-for-stability
ci: update K8s version for stability
2020-10-12 08:03:59 -06:00
Sébastien Han 8da0a146c9 Merge pull request #6427 from toabctl/ceph-watcher-debug-message-improvements
ceph: Improve watcher debug message when node is not tolerable
2020-10-12 15:52:54 +02:00
Thomas Bechtold ede4b9cb1a ceph: improve watcher debug message when node is not tolerable
When a node is not tolerable, also output the node name so it's clear
which not it is which is not tolerable.

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-10-12 14:19:17 +02:00
Sébastien Han b623133fb6 ceph: add ceph-volume log to non-pvc scenario
When deploying on non-PVC, we still need to gather c-v logs on failures.
Now if the bootstrap fails, the c-v log will be printed.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-12 14:10:17 +02:00
Sébastien Han 010a2804f6 Merge pull request #6392 from aruniiird/add-upgrade-option-to-external-script
ceph: adding '--upgrade' option to external script
2020-10-12 09:55:15 +02:00
Sébastien Han 0b253ef7f9 Merge pull request #6416 from cybozu-go/ci-ceph-fix-readme
ci: ceph fix tests/README.md
2020-10-12 09:13:38 +02:00
Satoru Takeuchi d8211135ea ci: update K8s version for stability
K8s v1.16.13 is used in CI. In this version, there is a serious bug
which causes a continuous restart of both kube-scheduler
and kube-controller-manager. It results in the instability of CI.

This bug is fixed in v1.16.14 or later by the following PR.
https://github.com/kubernetes/kubernetes/pull/93208

Both 6415 and 6390, which only happen in v1.16.13, would be
caused by this issue.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-11 02:46:56 +00:00
Satoru Takeuchi 90c03c814a ci: fix tests/README to use proper test command
In local environment, We should use `go test` than the test binary
under "_output/".

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-09 22:20:09 +00:00
Travis Nielsen e229a4f12e Merge pull request #6418 from travisn/doc-clarifications
docs: Clarifications for the readme and other documentation
2020-10-09 14:11:29 -06:00
Travis Nielsen 2f86051b1c docs: remove obsolete reference to v1alpha2 in design doc
The storageClassDeviceSets unnecessarily references the obsolete v1alpha2
version.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:53:45 -06:00
Travis Nielsen c1d553c318 docs: move ceph common issues subtopics under ceph tools
The ceph tools were split across different areas of the table of contents
that was confusing. Now the Ceph tools and troubleshooting guides are all
found under the Ceph tools.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:40:50 -06:00
Travis Nielsen 13da75a1db docs: clarify that the flex driver is disabled by default
The ceph flex driver has been disabled by default for some time.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:24:20 -06:00
Travis Nielsen 193def940d docs: remove unused documentation for test clusters
Rook should not be maintaining documentation for starting a K8s cluster
such as minikube or kubeadm. For simplicity, we just link to their documentation
to help them get started. Also removed is a section in the dev guide that
nobody has used for debugging operators locally.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:23:20 -06:00
Travis Nielsen efed0eff08 docs: update main readme for graduation
There are several small updates to the readme, including changing
a reference of incubation status to graduation, removing the
framework as an independently listed status since the operator
versioning is the critical point that depends on the framework,
moving the security reporting under bug reports, and reordering
cassandra to be alphabetically before cockroach.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:21:32 -06:00
Satoru Takeuchi a198c44a47 ci: export findRookRoot
Export findRookRoot() because it's useful to run any scripts
besides deploy_admission_controller_test.sh. In addition,
remove the unnecessary argument of findRookRoot().

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-09 15:49:00 +00:00
Arun Kumar Mohan fbb8af12e1 ceph: adding '--upgrade' option to external script
An '--upgrade' option is added to the option list, which upgrades the user
permissions (caps) to match newer version. Will try to detect
'rgw-pool-prefix' or user can provide it through the argument.

Segregated the command line arguments into three groups, common, output and
upgrade

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2020-10-09 18:59:25 +05:30
Sébastien Han 7bdbc0a4fc Merge pull request #6397 from leseb/fix-6134
ceph: tune osd tcmalloc thread cache bytes
2020-10-08 18:56:04 +02:00
Sébastien Han fe6b321bc6 Merge pull request #6395 from leseb/log-err-exec-file
ceph: log stderr in error when exec with output file
2020-10-08 17:56:00 +02:00
Sébastien Han eba561f13e Merge pull request #6408 from leseb/fix-bz-1885971
ceph: reduce s3 max retry
2020-10-08 17:30:27 +02:00
Travis Nielsen fd5c1982d1 Merge pull request #6407 from varshar16/update-nfs-ganesha-rados-url
ceph: update nfs ganesha rados config object
2020-10-08 08:56:23 -06:00
Sébastien Han d01a35cadd ceph: reduce s3 max retry
When rgw is not responding the aws-sdk retries using exponential
backoff. Having 20 retries means we would wait for a while (hours) to
eventually fail. This is a problem as we cannot reflect the status of
the bucket properly.
Now we use 5 max retries which means around 8 seconds.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-08 16:20:00 +02:00
Varsha Rao 7fd114612b docs: update ceph-nfs-crd about config object
Instead of creating config object per daemon. Now all daemons in a cluster
share a single config object.

Signed-off-by: Varsha Rao <varao@redhat.com>
2020-10-08 14:42:25 +05:30
Varsha Rao f5d125f8c9 ceph: update ganesha config object name
In cephadm and volume/nfs module, all ganesha daemons within a cluster watch
single config object. Each ganesha daemon in rook watches it's own config
object. This patch updates the ganesha config object name to maintain
uniformity.

Signed-off-by: Varsha Rao <varao@redhat.com>
2020-10-08 14:42:25 +05:30