Commit Graph
13292 Commits
Author SHA1 Message Date
Alexander Trost 34930020ab ceph: allow pod labels to be added to CSI components
This adds the functionality to add custom pod labels to the CSI
components through the operator configuration way of env vars or config
map.

Resolves #6593

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-11-12 13:06:24 +01:00
Sébastien Han 0c72601d31 Merge pull request #6595 from ml8mr/fix_crd_url
docs: link to master branch for ceph-csi-snapshot
2020-11-12 09:32:05 +01:00
Blaine Gardner 90ec68d9ff Merge pull request #6524 from LalitMaganti/master
ceph: associate newly created pools to existing filesystem
2020-11-11 16:15:13 -07:00
Blaine Gardner dd560f2305 Merge pull request #6610 from BlaineEXE/fix-rgw-deploy-version
ceph: fill in rgw deployment version
2020-11-11 16:12:51 -07:00
Travis Nielsen 91bd0ee3e9 Merge pull request #6556 from travisn/helm3-crd
ceph: Update helm version to 3.4
2020-11-11 16:01:23 -07:00
Lalit Maganti 1d1a9f111a ceph: associate newly created pools to existing filesystem
When new pools are added to an already existing filesystem, we would
create them but not actually associate them to the filesystem. Ensure
that we do this.

Also, while we're here, we refactor the CreateFilesystem test: before,
we were not properly testing creating the filesystem from scratch
because we would return valid JSON for *every* ceph command. Change this
by making the first run return an error so that we get some better
coverage.

After this, we also add a check for the correctness of adding new pools
to the CreateFilesystem unittest.

Fixes #5876

Signed-off-by: Lalit Maganti <lalitm@google.com>
2020-11-11 22:16:37 +00:00
Blaine Gardner 98d5c73d5e ceph: fill in rgw deployment version
The RGW deployment's ceph-version label should now display the same
version as the image it is using.

It previously displayed an empty version: 0.0.0-0.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-11-11 15:13:31 -07:00
Pete Birley 152a05c85e ceph: update to helm 3 for the rook chart
This updates the chart to make use of helm3 which has been released
for some time, and also permits CRDs to be installed pror to other objects
allowing the chart to be deployed at the same time as CRs for rook objects.

Co-authored-by: Pete Birley <pete@port.direct>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-11 14:41:24 -07:00
Travis Nielsen bf1d3888f4 ceph: factor out CRDs from the common.yaml and helm chart
The CRDs need to be installed and updated separately from the
rest of the manifests. Helm charts also do not have a way to update
CRDs, therefore, any updates to the CRD schema must be done separately
from the helm chart. Now the CRDs are created in a new crd.yaml that must
be created along with common.yaml and before creating the operator.

The helm chart still contains the CRDs for initial creation, though
the helm chart will never update or remove them.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-11 12:16:20 -07:00
Travis Nielsen 61539b6699 Merge pull request #6559 from BlaineEXE/upgrade-test-1.5
ceph: upgrade test Rook only from v1.4 to master
2020-11-11 10:13:39 -07:00
Travis Nielsen 6142efe14b Merge pull request #6579 from subhamkrai/nfs-log
ceph: change nfs-ganesha log level
2020-11-11 10:09:52 -07:00
subhamkrai 7d1b5f7416 ceph: change nfs-ganesha log level
this commit allows changing the default
log level from nfs.yaml file by adding
new command.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-11-11 20:34:07 +05:30
Travis Nielsen 6c19f1f63d Merge pull request #6490 from xiaods/issue-6460-fix
fix: issue#6460 remove helm2 section, upgrade helm3
2020-11-11 07:57:48 -07:00
Travis Nielsen 55770d4fc8 Merge pull request #6594 from cybozu-go/doc-ceph-clarify-when-operator-stop-orchestration
docs: clarify the meaning of cleanup policy of ceph cluster crd
2020-11-11 07:52:46 -07:00
Satoru Takeuchi 21cef906f7 docs: clarify the meaning of cleanup policy of ceph cluster crd
The description of cleanupPolicy is a bit ambiguous. First, we should clarify
that both skipping orchestration and data cleanup will happen iff
"yes-really-destroy-data" is set to `confirmation`. Second, We should also
describe wiping devices will happen even if user don't specify `sanitizeDisks`
explicitly.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-11-11 05:15:41 +00:00
Blaine Gardner 2f034824cb ceph: upgrade test Rook only from v1.4 to master
Since Rook no longer supports legacy filestore devices, there is
no need to keep testing upgrades from v1.2 all the way to master.
We can now just test v1.4 to master.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2020-11-10 15:44:05 -07:00
Blaine Gardner 6a1f49fa10 Merge pull request #6589 from travisn/disable-discovery
ceph: Disable the discovery daemon by default
2020-11-10 15:14:54 -07:00
Mike Latimer afd6c421bb docs: link to master branch for ceph-csi-snapshot
Due to upstream code motion, the current link to the upstream
ceph-csi-snapshot documentation no longer points to the correct
location of their CRDs. Fix this by just using the master branch.

Signed-off-by: Mike Latimer <mlatimer@suse.com>
2020-11-10 15:10:54 -07:00
Travis Nielsen b9f692a56e ceph: disable the discovery daemon by default
The discovery daemon is not needed in most scenarios, therefore we disable it
by default. More and more clusters are moving to the cluster-on-pvc scenario
which certainly does not need the local discovery. Even where clusters are not
running on PVCs, the discovery is not needed since the device discovery is again
performed in the osd prepare job.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-10 13:16:52 -07:00
Sébastien Han 969225c0c2 Merge pull request #6483 from cybozu-go/ceph-make-the-placement-of-admission-controller-configurable
ceph: make the placement of admission controller configurable
2020-11-09 14:32:02 +01:00
Satoru Takeuchi c7990777d2 ceph: make the placement of admission controller configurable
If users want to restrict the nodes where Ceph daemons should exist,
it's better to make the placement of admission controller configurable
as other daemons.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-11-09 11:54:52 +00:00
Sébastien Han b72382f51e Merge pull request #6576 from terorie/patch-1
docs: better Ceph toolbox kubectl commands
2020-11-09 10:20:27 +01:00
Richard Patel 865c3b263e docs: better Ceph toolbox kubectl commands
Signed-off-by: Richard Patel <me@terorie.dev>
2020-11-08 03:38:48 +01:00
Travis Nielsen 6f13dd6559 Merge pull request #6574 from travisn/build-fixes-from-1.5
Integration test fixes for release branch to preserve in master
2020-11-06 12:58:30 -07:00
Satoru Takeuchi aa85577ff3 Merge pull request #6567 from gseidel/precise-three-nodes
docs: precise three nodes
2020-11-07 04:32:43 +09:00
Travis Nielsen f1287bdb19 ceph: skip csi tests on k8s 1.13
The csi tests require 1.14 or newer to run

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-06 11:16:32 -07:00
Travis Nielsen c029b1513d ceph: remove nullable attribute from pre-1.16 schema
The nullable attribute was causing the integration tests to fail
on 1.13 and earlier where it is not supported in the schema.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit cc8f406ce9)
2020-11-06 10:47:26 -07:00
Travis Nielsen 556de488c0 nfs: skip running tests on older than k8s 1.14
NFS tests are not supported on older than K8s 1.14.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 4adc54e6f1)
2020-11-06 10:47:26 -07:00
Travis Nielsen cf9fdb683f Merge pull request #6564 from invidian/codespell-pipeline
ci: .github/workflows: add codespell pipeline
2020-11-06 09:25:21 -07:00
Gerhard Seidel 042e10bcf6 docs: precise three nodes
When following the quick start, it may not work, because the user don't know that he need three nodes. And three nodes may also not be correct, because the control plane is also a node. So we should precise what the user need to run his cluster correctly.

Closes: https://github.com/rook/rook/issues/6540
Signed-off-by: Gerhard Seidel <gseidel.message@googlemail.com>
2020-11-06 16:22:59 +01:00
Sébastien Han e3d032dbc1 Merge pull request #6489 from travisn/stretch-mons
ceph: Stretch cluster configuration
2020-11-06 15:16:39 +01:00
Sébastien Han f938447398 Merge pull request #6562 from leseb/mergify-1.5
bot: update mergify to release 1.5 branch
2020-11-06 15:16:17 +01:00
Travis Nielsen 67ce7e4c3b Merge pull request #6561 from Madhu-1/cephfs-priv-pod
ceph: add privileged securityContext to cephfs deployment
2020-11-06 07:13:33 -07:00
Mateusz Gozdek 8912e491d8 ci: .github/workflows: add codespell pipeline
As a follow up to #6563 which fixed all the typos found in the
repository, to avoid adding more obvious typos.

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-06 12:43:29 +01:00
Satoru Takeuchi afcfe40b5e Merge pull request #6563 from invidian/typos
docs: fix bunch of typos
2020-11-06 18:57:45 +09:00
Mateusz Gozdek 8ba3762fa4 docs: fix bunch of typos
Found by running the following command:

codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-06 10:01:04 +01:00
Sébastien Han 1b45127056 bot: mergify add release 1.5
Update mergify to auto backport PR with label on release 1.5 branch and
auto merge.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-06 08:58:46 +01:00
Sébastien Han 896d721737 bot: mergify remove release-1.2 actions
1.2 is getting old and automatic backport will likely fail anyways.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-06 08:54:50 +01:00
Madhu Rajanna 8f5196a559 ceph: add privileged securityContext to cephfs deployment
cephfs csi driver still uses the ceph CLI for all the
ceph fs operations, when cephcsi gets a request from
the kubernetes it will internally starts threads for
each call. in cephcsi we are setting the pids
limit to max, to set this value the container has
to be running as the privileged one.

if we go with default pid limit we will face core dump
due to resource crunch.

without privileged access we will see Failed to set new
PID limit to -1: open /sys/fs/cgroup/pids/kubepods.slice/
kubepods-besteffort.slice/kubepods-besteffort-pod1029ed37_bebc_466d_9e53_79e7323a67ed.slice/crio-9ef6768d027916f414bf30b54ef13eebad8efd0aa4741d97618affa838d7b4c0.scope/pids.max: permission denied

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-06 09:40:18 +05:30
Travis Nielsen 372b74b4d7 Merge pull request #6558 from travisn/latest-octopus
ceph: Update base image and examples to v15.2.5
v1.5.0-alpha.0
2020-11-05 16:40:53 -07:00
Travis Nielsen a5b9fbc994 ceph: update base image and examples to v15.2.5
For the v1.5 release we use the latest version of ceph by default,
which is now v15.2.5

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 15:50:23 -07:00
Travis Nielsen 6e2679cbff Merge pull request #6545 from leseb/fix-6532
ceph: add unit test for ceph-volume batch mode and warn about unsupported ceph version
2020-11-05 11:42:01 -07:00
Sébastien Han 64fe9f7685 ceph: introduced unsupported releases
These are a collection of unstable ceph release that are not recommended
for production. It is recommended to rollback to the previous or
  upgrading to the next one if possible.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-05 18:35:57 +01:00
Sébastien Han 8631eef1f7 ceph: correctly populate variable
The wrong variable was referenced in the print.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-05 18:06:47 +01:00
Sébastien Han bc5c8d32f1 ceph: fix crush-class osd on non-PVC
We need to use storeConfig instead device config as the later never
seems to be populated.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-05 18:06:47 +01:00
Sébastien Han 6b211eacba ceph: add unit test for osd batch mode
We now have much better coverage on the c-v on non-pvc scenario.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-11-05 18:06:11 +01:00
Travis Nielsen 7c3cdbce50 ceph: use constants for k8s topology labels
Instead of directly defining our own constants, we should be using the k8s
constants for the well-known topology labels.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 09:23:58 -07:00
Travis Nielsen e72b69f8e0 ceph: validate pools for stretch clusters
Pools in stretch clusters will all use the same crush rule that
is generated by the operator when configuring the cluster for
stretch mode, with replica 4 and two replicas per failure domain.
Pools cannot create new crush rules, so we require that replica be
4 when the pools is created, EC is not allowed, and no new
crush rule will be created.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 09:23:57 -07:00
Travis Nielsen b91f4211c9 ceph: configure a stretched cluster
In clusters where only two datacenters (or similar failure domains)
are available, a different mon and osd approach is needed to deal
with the network partitions or some other reason for one of the failure
domains going down. The Ceph stretched cluster makes the mons aware
of the failure domains by configuring one as the arbiter in a third
zone, while keeping two replicas of the data in each of the data
zoens.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 09:17:52 -07:00
Travis Nielsen 3b188a447a ceph: reorganize admission controller validation helpers
The validation code for the admission controller is found in the
ceph.rook.io/v1 package, corresponding to each of the types.
To be better organized, the validators are now found in a source
file of the corresponding type name.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-05 09:17:52 -07:00