Commit Graph
2411 Commits
Author SHA1 Message Date
Travis Nielsen 3e7a7626db Merge pull request #6599 from cloudical-io/issue_6593
ceph: allow pod labels to be added to CSI components
2020-11-12 07:54:14 -07:00
Sébastien Han c886ed7fcb Merge pull request #6544 from varshar16/wip-update-ceph-nfs-config-name
ceph: Add 'ganesha-' to nfs-ganesha config object name
2020-11-12 15:08:57 +01:00
Varsha Rao 8fadaeeb09 ceph: add 'ganesha-' to nfs-ganesha config object name
Since ceph octopus version, the expected nfs-ganesha config object name by
mgr/volumes/nfs[1] plugin is "conf-nfs.ganesha-<clustername>". For older
versions the name is "conf-<clustername>.<nodeid>".

[1] https://github.com/ceph/ceph/blob/master/src/pybind/mgr/volumes/fs/nfs.py#L648-L655

Signed-off-by: Varsha Rao <varao@redhat.com>
2020-11-12 18:09:46 +05:30
Alexander Trost 34930020ab ceph: allow pod labels to be added to CSI components
This adds the functionality to add custom pod labels to the CSI
components through the operator configuration way of env vars or config
map.

Resolves #6593

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-11-12 13:06:24 +01:00
Sébastien Han 0c72601d31 Merge pull request #6595 from ml8mr/fix_crd_url
docs: link to master branch for ceph-csi-snapshot
2020-11-12 09:32:05 +01:00
Pete Birley 152a05c85e ceph: update to helm 3 for the rook chart
This updates the chart to make use of helm3 which has been released
for some time, and also permits CRDs to be installed pror to other objects
allowing the chart to be deployed at the same time as CRs for rook objects.

Co-authored-by: Pete Birley <pete@port.direct>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-11 14:41:24 -07:00
Travis Nielsen bf1d3888f4 ceph: factor out CRDs from the common.yaml and helm chart
The CRDs need to be installed and updated separately from the
rest of the manifests. Helm charts also do not have a way to update
CRDs, therefore, any updates to the CRD schema must be done separately
from the helm chart. Now the CRDs are created in a new crd.yaml that must
be created along with common.yaml and before creating the operator.

The helm chart still contains the CRDs for initial creation, though
the helm chart will never update or remove them.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-11 12:16:20 -07:00
Travis Nielsen 61539b6699 Merge pull request #6559 from BlaineEXE/upgrade-test-1.5
ceph: upgrade test Rook only from v1.4 to master
2020-11-11 10:13:39 -07:00
Travis Nielsen 6c19f1f63d Merge pull request #6490 from xiaods/issue-6460-fix
fix: issue#6460 remove helm2 section, upgrade helm3
2020-11-11 07:57:48 -07:00
Travis Nielsen 55770d4fc8 Merge pull request #6594 from cybozu-go/doc-ceph-clarify-when-operator-stop-orchestration
docs: clarify the meaning of cleanup policy of ceph cluster crd
2020-11-11 07:52:46 -07:00
Satoru Takeuchi 21cef906f7 docs: clarify the meaning of cleanup policy of ceph cluster crd
The description of cleanupPolicy is a bit ambiguous. First, we should clarify
that both skipping orchestration and data cleanup will happen iff
"yes-really-destroy-data" is set to `confirmation`. Second, We should also
describe wiping devices will happen even if user don't specify `sanitizeDisks`
explicitly.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-11-11 05:15:41 +00:00
Blaine Gardner 2f034824cb ceph: upgrade test Rook only from v1.4 to master
Since Rook no longer supports legacy filestore devices, there is
no need to keep testing upgrades from v1.2 all the way to master.
We can now just test v1.4 to master.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-11-10 15:44:05 -07:00
Mike Latimer afd6c421bb docs: link to master branch for ceph-csi-snapshot
Due to upstream code motion, the current link to the upstream
ceph-csi-snapshot documentation no longer points to the correct
location of their CRDs. Fix this by just using the master branch.

Signed-off-by: Mike Latimer <mlatimer@suse.com>
2020-11-10 15:10:54 -07:00
Travis Nielsen b9f692a56e ceph: disable the discovery daemon by default
The discovery daemon is not needed in most scenarios, therefore we disable it
by default. More and more clusters are moving to the cluster-on-pvc scenario
which certainly does not need the local discovery. Even where clusters are not
running on PVCs, the discovery is not needed since the device discovery is again
performed in the osd prepare job.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-10 13:16:52 -07:00
Satoru Takeuchi c7990777d2 ceph: make the placement of admission controller configurable
If users want to restrict the nodes where Ceph daemons should exist,
it's better to make the placement of admission controller configurable
as other daemons.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-11-09 11:54:52 +00:00
Richard Patel 865c3b263e docs: better Ceph toolbox kubectl commands
Signed-off-by: Richard Patel <me@terorie.dev>
2020-11-08 03:38:48 +01:00
Satoru Takeuchi aa85577ff3 Merge pull request #6567 from gseidel/precise-three-nodes
docs: precise three nodes
2020-11-07 04:32:43 +09:00
Gerhard Seidel 042e10bcf6 docs: precise three nodes
When following the quick start, it may not work, because the user don't know that he need three nodes. And three nodes may also not be correct, because the control plane is also a node. So we should precise what the user need to run his cluster correctly.

Closes: https://github.com/rook/rook/issues/6540
Signed-off-by: Gerhard Seidel <gseidel.message@googlemail.com>
2020-11-06 16:22:59 +01:00
Sébastien Han e3d032dbc1 Merge pull request #6489 from travisn/stretch-mons
ceph: Stretch cluster configuration
2020-11-06 15:16:39 +01:00
Mateusz Gozdek 8ba3762fa4 docs: fix bunch of typos
Found by running the following command:

codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-06 10:01:04 +01:00
Travis Nielsen a5b9fbc994 ceph: update base image and examples to v15.2.5
For the v1.5 release we use the latest version of ceph by default,
which is now v15.2.5

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 15:50:23 -07:00
Travis Nielsen b91f4211c9 ceph: configure a stretched cluster
In clusters where only two datacenters (or similar failure domains)
are available, a different mon and osd approach is needed to deal
with the network partitions or some other reason for one of the failure
domains going down. The Ceph stretched cluster makes the mons aware
of the failure domains by configuring one as the arbiter in a third
zone, while keeping two replicas of the data in each of the data
zoens.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 09:17:52 -07:00
Jared Watts 8d445c919d Merge pull request #6202 from prksu/nfs-quota
nfs: implement disk quota
2020-11-02 08:34:54 -08:00
Deshi XiaoandAlexander Trost 0ada78ecb0 docs: fix issue#6460 remove helm2 section
1. only helm3 support,deprecated helm2.x docs
2. remove useless helm3 comments
3. update documentation for helm-operator.md

Co-authored-by: Alexander Trost <Galexrt@googlemail.com>
Signed-off-by: Deshi Xiao <xiaods@gmail.com>
2020-10-31 10:52:24 +08:00
Travis Nielsen 44bf443dca Merge pull request #6495 from LalitMaganti/preserve-fs
ceph: add option to preserve filesystem on CRD deletion
2020-10-30 16:56:38 -06:00
Lalit Maganti c6aec79c4f ceph: add option to preserve filesystem on CRD deletion
Due to #6492, preservePoolsOnDelete is not useful at all for CephFS;
after the filesystem is deleted, the leftover pools cannot be
reassocaited with a newly created filesystem without wiping all
metadata. The only way we can actually preserve data is keeping around
the entire filesystem.

This commit implements a `preserveFilesystemOnDelete` option which work
similar to the existing pool preservation option but instead keeps the
whole CephFS while taking it down and removing all MDSes.

This commit also changes all documentation to refer to this new option
with the intent of essentially deprecating `preservePoolsOnDelete`. IMO,
keeping around `preservePoolsOnDelete` is actively harmful because it
lulls users into thinking their data will be safe but, in reality,
recovering from this situation is highly complex and has large potential
for data loss.

Signed-off-by: Lalit Maganti <lalitm@google.com>
2020-10-30 17:04:18 +00:00
Sébastien Han ea1d71cbfb ceph: add vault kms support for osd encryption
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/

The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:

security:
  kms:
    tokenSecretName: <name of the secret containing a Vault token, used
    to authenticate>
    connectionDetails: < a map of strings containing connection
    information>

Refer to the ceph-cluster-crd documentation to lear more.

Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-30 16:16:33 +01:00
Travis Nielsen 1c562512e1 Merge pull request #6501 from xiaods/issue-6480-fix
Remove provisioner from nfs docs
2020-10-29 10:24:58 -06:00
Travis Nielsen 22a1630249 Merge pull request #5483 from cloudandheat/feature/configurable-crush-root
ceph: make root= CRUSH label value configurable
2020-10-29 10:21:02 -06:00
Deshi Xiao 512544b849 docs: tuning docs for nfs setup
1. remove missing provisioner.yaml
2. modified statement description

Signed-off-by: Deshi Xiao <xiaods@gmail.com>
2020-10-29 21:22:09 +08:00
Jonas Schäfer 7ace6ac255 ceph: make root= CRUSH label value configurable
The CRUSH map does not allow duplicate values (nor keys) for the
labels. That means that if the currently hardcoded label value
`default` occurs anywhere else in the tree (for example, because
some of the topology labels provided by the cloud provider use it),
the cluster cannot sucessfully spawn.

By giving the user control over the label value used for the root
CRUSH map label, it is possible to adapt the cluster to this
situation.

To be able to actually use the `default` value, the root=default
node which is created by default by ceph itself has to be removed;
since that node is accompanied by a replicated crush rule, we have
to remove that rule, too.

The integration tests are modified to sometimes use the custom
crushRoot (based on an arbitrary criterium) to get coverage across
the various suites. Separate tests could be added, but were not
deemed necessary at this point.

Fixes #4993.

Signed-off-by: Jonas Schäfer <jonas.schaefer@cloudandheat.com>
2020-10-29 09:22:41 +01:00
subhamkraiandtravisn 1b2c15041f ceph: update deprecated CRD apiextensions.k8s.io/v1beta1 to v1
the apiextensions.k8s.io/v1beta1 version of CustomResourceDefinition
is deprecated in Kubernetes v1.16 and will no longer be supported from
v1.19. For now, we changing only for ceph and it's related documented.

Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: travisn <tnielsen@redhat.com>
Signed-off-by: subhamkrai <srai@redhat.com>
2020-10-29 06:26:10 +05:30
Tareq Sharafy d2a6518a5c docs: disaster recovery guide for PVCs
Signed-off-by: Tareq Sharafy <tareq.sha@gmail.com>
2020-10-24 23:07:18 +03:00
Sébastien Han 1a5d296e8e Merge pull request #6481 from cloudical-io/issue_6430
docs: update ceph monitoring examples
2020-10-23 12:07:26 +02:00
Alexander Trost 30fc31fb31 docs: update ceph monitoring examples
This updates the Ceph docs regarding the Grafana dashboards new version
requirement.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-10-23 10:33:48 +02:00
Travis Nielsen 66636f5420 Merge pull request #6447 from MG40/docs
docs: Ceph disaster recovery
2020-10-20 16:33:03 -06:00
Ahmad Nurus S 72bf7873c9 nfs: implement disk quota
Signed-off-by: Ahmad Nurus S <prksu.sh@gmail.com>
2020-10-19 10:20:12 +07:00
MG 1063c387f9 docs: ceph disaster recovery
updated by removing the paranthesis around the arguments  to pass the value
Eg: ROOK_CEPH_MON_INITIAL_MEMBERS

Signed-off-by: MG <gmadhavi@gmail.com>
2020-10-16 23:03:11 +08:00
Ahmad Nurus S c22189ba74 nfs: update documentation and example
Signed-off-by: Ahmad Nurus S <prksu.sh@gmail.com>
2020-10-16 01:11:44 +07:00
Sébastien Han f0b2c6eacf Merge pull request #6184 from cybozu-go/ceph-raw-mode-on-lv-backed-pvc
ceph: raw mode OSD on LV-backed PVC
2020-10-15 09:46:56 +02:00
Daichi Sakaue b9f29349ed ceph: use raw mode instead of lvm mode even if lv-backed PVC
It can simplify OSD management in many ways and can avoid many problems
that come from LVM.

Closes: https://github.com/rook/rook/issues/5939
Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-14 01:46:25 +00:00
Sébastien Han b70b098405 ceph: add support for stretched cluster crush rule
The pool spec has now a new property called "replicasPerFailureDomain"
which essentially represents the number of replicas to store in each
failure domain.
Assuming the failure domain is a datacenter (if the cluster is
stretched) then you will have 2 replicas per datacenter where each
replica ends up on a different host. This gives you a total of 4
replicas and for this, the "size" must be set to 4.

Closes: https://github.com/rook/rook/issues/5591
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-13 16:33:34 +02:00
Travis Nielsen a4b1ca3faa docs: clarify cluster-on-pvc scenario for ceph
A frequently asked question is how to configure the cluster in dynamic
environments, or basically to use the cluster-on-pvc configuration.
This adds more details to the examples in the docs.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-12 15:21:28 -06:00
Travis Nielsen c7e0338d5e ceph: require an odd number of mons and sufficient nodes
The ceph mons should never be started with an even number in quorum.
The desired state should always be an odd number of mons to ensure
a healthy majority quorum. The operator now rejects a request for
an even number of mons.

If there are not a sufficient number of nodes for mons to be on a
unique host, the operator also returns an early error instead of
getting stuck waiting for a pending mon. This is not foolproof since
there might be fewer available nodes for the mons, but at least it
is a quick check for the common case.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-12 15:21:27 -06:00
Travis Nielsen c1d553c318 docs: move ceph common issues subtopics under ceph tools
The ceph tools were split across different areas of the table of contents
that was confusing. Now the Ceph tools and troubleshooting guides are all
found under the Ceph tools.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:40:50 -06:00
Travis Nielsen 13da75a1db docs: clarify that the flex driver is disabled by default
The ceph flex driver has been disabled by default for some time.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:24:20 -06:00
Travis Nielsen 193def940d docs: remove unused documentation for test clusters
Rook should not be maintaining documentation for starting a K8s cluster
such as minikube or kubeadm. For simplicity, we just link to their documentation
to help them get started. Also removed is a section in the dev guide that
nobody has used for debugging operators locally.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:23:20 -06:00
Travis Nielsen efed0eff08 docs: update main readme for graduation
There are several small updates to the readme, including changing
a reference of incubation status to graduation, removing the
framework as an independently listed status since the operator
versioning is the critical point that depends on the framework,
moving the security reporting under bug reports, and reordering
cassandra to be alphabetically before cockroach.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-10-09 12:21:32 -06:00
Travis Nielsen fd5c1982d1 Merge pull request #6407 from varshar16/update-nfs-ganesha-rados-url
ceph: update nfs ganesha rados config object
2020-10-08 08:56:23 -06:00
Varsha Rao 7fd114612b docs: update ceph-nfs-crd about config object
Instead of creating config object per daemon. Now all daemons in a cluster
share a single config object.

Signed-off-by: Varsha Rao <varao@redhat.com>
2020-10-08 14:42:25 +05:30