Commit Graph
6516 Commits
Author SHA1 Message Date
Travis Nielsen 3bbc358d32 Merge pull request #7440 from travisn/mgr-labels
ceph: Set active mgr label on mgr services
2021-03-19 11:16:33 -06:00
Travis Nielsen c78e6a9e20 ceph: set active mgr label on mgr services
Other resources such as the service monitor need to match
the labels on the services to the active mgr, thus the active
mgr name is now set on the labels for the metrics service
and dashboard service.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-18 16:37:01 -06:00
Travis Nielsen 4d06ad1155 Merge pull request #7415 from subhamkrai/placement
ceph: placement in case of both PVC and non-PVC's
2021-03-18 12:33:02 -06:00
Sébastien Han e7c3e2c064 Merge pull request #7435 from aruniiird/fix-create-external-script-random-failure
ceph: fix for the random script failures because of open file issue
2021-03-18 16:57:35 +01:00
Arun Kumar Mohan 822a5194da ceph: fix for the random script failures because of open file issue
Script's tests were failing randomly because of an unclosed file
exceptions.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-03-18 20:59:24 +05:30
Sébastien Han c43058a98c Merge pull request #7355 from leseb/more-disks-ci
ceph: add more osds to the ci
2021-03-18 10:56:19 +01:00
Sébastien Han f13563d674 Merge pull request #7432 from leseb/csi-block-owner
ceph: fix csi blocker owner deletion
2021-03-18 10:38:09 +01:00
Sébastien Han 461dcd7457 ceph: fix csi blocker owner deletion
Small regression introduced in https://github.com/rook/rook/commit/26c8fd9bd14178d74dcc28f009e5c2f658224436
which was overriding the BlockOwnerDeletion from the ownerref.
Now, if set we don't modify it.
This fixes deployments on OCP.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-18 09:50:52 +01:00
Sébastien Han 67c669f183 Merge pull request #7425 from subhamkrai/gh-multi-k8s
ci: run gh action on multiple k8s version
2021-03-18 09:20:42 +01:00
subhamkraiandTravis Nielsen 319e4a41a4 ceph: placement in case of both PVC and non-PVC's
In the case of PVC,
We are giving lower priority to all placement.
We want deviceSet placement to applied and
override in case of overlapping settings and
we are merging nodeAffinity if applied in both
all placement and deviceSet.

In case of non-PVC,
we apply spec.placement

Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-18 10:32:57 +05:30
subhamkrai 07df8220d8 ci: run gh action on multiple k8s version
this commit enables gh action to run

Flex test on only 'v1.15.12',
Smoke and Helm tests on both 'v1.15.12','v1.20.2',
Mgr, Multicluster and upgrade tests on only 'v1.20.2'

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-18 09:03:56 +05:30
Travis Nielsen 7b3244a324 Merge pull request #7390 from travisn/refactor-tests
ceph: Refactor integration tests for simplification
2021-03-17 13:51:37 -06:00
Travis Nielsen 65fb9e0d92 Merge pull request #7419 from fritchie/maxsize-optional
ceph: mark maxSize as optional
2021-03-17 13:45:53 -06:00
Travis Nielsen deb7c21f98 ceph: install pre-1.16 crds with helm chart
The helm chart was not installing the CRDs on pre-1.16 k8s due
to a regression in the conditionals for the chart that was introduced
with 7103.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 13:10:53 -06:00
Travis Nielsen bdbf264e68 ceph: integration tests in github actions skip cleanup
The integration tests run independently in the github actions so there is
no need to cleanup from every test. The cleanup is still needed in the
Jenkins tests where all the suites run serially.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 11:26:11 -06:00
Travis Nielsen c23238cddb ceph: refactor integration tests for simplification
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 11:26:10 -06:00
Travis Nielsen f50e49be3a ceph: object store user initialization check for nil
If the object store status is not yet initialized, the object store user should
fail its initialization and requeue the reconcile.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:26 -06:00
Travis Nielsen 64e28af741 ceph: allow flex driver and discovery to be enabled with configmap
For testing purposes, we need to configure the flex driver
and discovery daemon with the operator settings configmap.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:26 -06:00
Travis Nielsen 933113eac8 ceph: remove obsolete storeType setting
All clusters are configured with bluestore as the store type
therefore we can remove the settings where mentioned in the
docs and dead code.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:25 -06:00
Frank Ritchie 7fcd0ab3d9 ceph: mark maxSize as optional
This is a follow up to:

https://github.com/rook/rook/pull/7397

Looks like I forgot to mark maxSize as optional.

Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
2021-03-17 09:43:31 -04:00
Sébastien Han 913e2c97f7 Merge pull request #7417 from anmolsachan/add_osd_slow_ops_alert
ceph: add osd slow ops alert
2021-03-16 16:21:03 +01:00
Travis Nielsen 32c628108c Merge pull request #7416 from leseb/debug-ci
ci: debug runners on failures
2021-03-16 08:57:32 -06:00
Travis Nielsen 014793c7ee Merge pull request #7340 from rohan47/clusterrole-multus
ceph: updated rbac for multus in helm charts
2021-03-16 08:49:12 -06:00
Travis Nielsen 4d969a406f Merge pull request #7414 from sp98/skip-cluster-cleanup
ceph: skip cleanup if cluster is not configured correctly.
2021-03-16 08:24:24 -06:00
rohan47 f3a75216a8 ceph: updated rbac for multus in helm charts
The rbac required for multus was present in role which makes
only the network-attachment-definitions in the rook cluster namespace
accessible to the operator. Moved it to clusterrole for cluster wide
access to NAD.

Signed-off-by: rohan47 <rohgupta@redhat.com>
2021-03-16 19:40:17 +05:30
Santosh Pillai 7bd05d0d7c ceph: skip cleanup if cluster is not configured correctly
deletion of cluster is stuck when cluster is not configured. The PR skips the cleanup and removes the finalizers if the cluster was not configured, that is, if clusterInfo could not be loaded.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-03-16 18:59:43 +05:30
Satoru Takeuchi 756d4ecb89 Merge pull request #7259 from cybozu-go/ceph-improve-owner-reference-management-2
ceph: improve owner reference management
2021-03-16 20:22:57 +09:00
Satoru Takeuchi 26c8fd9bd1 ceph: improve owner reference management
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-16 10:29:33 +00:00
Anmol Sachan f84cbb47fe ceph: add osd slow ops alert
This commit adds an alert that notifies the users about OSD
requests taking too long to process.

Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
2021-03-16 15:40:28 +05:30
Sébastien Han ab50e75e08 Merge pull request #7402 from subhamkrai/yaml-ignore
ceph: ignore monitoring folder from yamllint
2021-03-16 10:46:17 +01:00
Sébastien Han 88c0eb57cd ci: debug runners on failures
If one of the suite fails, it is sometimes handy to ssh into the
instance to precisely understand what's going on.
So by applying a debug label so as "debug-smoke-suite" you will be able
to ssh into the runner to debug. By default it looks
like the runner will be cancelled automatically after 6 hours as per the
official doc: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes

I've had a different experience, and somehow 45min seems to be the limit
but I might be wrong.

Once the label is applied, go into the check and the once you are
interested in and search the step "setup tmate session for debugging".
It should output lines with the connection details.

To get the list of all labels see: https://github.com/rook/rook/labels?q=debug-
Example run is available here: https://github.com/rook/rook/pull/7416/checks?check_run_id=2120103989

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 10:43:38 +01:00
Sébastien Han a062d20059 Merge pull request #7103 from leseb/auto-gen-crds
ceph: auto generate crds files
2021-03-16 10:40:03 +01:00
Sébastien Han 31db03fece ceph: auto-gen crds
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:

* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`

will be autogenerated for us.

We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.

It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:

```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```

The corresponding API type markers are:

```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```

For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 09:16:24 +01:00
Sébastien Han 7291ce950e ceph: fix variable name
Fix linter complain.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 08:35:36 +01:00
subhamkrai acfbc78392 ceph: ignore the monitoring folder from yamllint
this commit adds the monitoring folder to
yamllint ignore list.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-16 12:33:37 +05:30
Travis Nielsen 53cc160817 Merge pull request #7397 from fritchie/pool-quota-units
ceph: set pool quota with k8s quantity format
2021-03-15 18:09:48 -06:00
Frank Ritchie 9ae699d632 ceph: set pool quota with k8s quantity format
This is a follow up to:

https://github.com/rook/rook/pull/7264

which added pool quotas. It was stated that it would be nice to
be able to set pool max_bytes quotas using standard k8s quantity
formats rather than an integer representing the number of bytes.

I was too busy at the time to make the changes. This PR adds the
functionality.

Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
2021-03-15 19:26:41 -04:00
Blaine Gardner 6324746c18 Merge pull request #7409 from BlaineEXE/fix-ci
ci: update minikube github action to fix apt-get
2021-03-15 10:39:26 -06:00
Blaine Gardner b744a08c54 ci: update minikube github action to fix apt-get
Fix an apt-get update issue by bumping to the latest (fixed) version of
manusa/actions-setup-minikube (v2.3.1).

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-03-15 10:10:34 -06:00
Sébastien Han f2c0546837 Merge pull request #7391 from leseb/enable-auto-pg-repair
ceph: enable pg auto repair
2021-03-15 15:18:40 +01:00
Sébastien Han 24d7cd79d3 Merge pull request #7392 from cybozu-go/ceph-suppress-golanglint-complaints
ceph: suppress golanglint-ci complaints
2021-03-15 09:27:55 +01:00
Satoru Takeuchi fb93c9f64f Merge pull request #7394 from cybozu-go/ci-describe-the-location-of-log
ci: describe the location of test logs
2021-03-15 10:09:46 +09:00
Satoru Takeuchi 04eed746d9 ci: describe the location of test logs
Although the integration test creates test logs, the location is not
documented yet.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-12 20:01:35 +00:00
Satoru Takeuchi 4766954dfc ceph: suppress golanglint-ci complaints
Suppress the following complaints.

```
$ golangci-lint run -E gosec
pkg/operator/test/client.go:244:13: G404: Use of weak random number generator (math/rand instead of crypto/rand) (gosec)
        randIdx := rand.Intn(len(nodes.Items))
                   ^
tests/framework/installer/ceph_manifests_v1.5.go:44:19: G107: Potential HTTP request made with variable url (gosec)
        response, err := http.Get(url)
                         ^
pkg/daemon/ceph/client/pool.go:182:6: ineffectual assignment to stats (ineffassign)
        var stats = new(PoolStatistics)
            ^
pkg/operator/k8sutil/pod_test.go:34:2: ineffectual assignment to container (ineffassign)
        container, err := GetMatchingContainer([]v1.Container{}, expectedName)
        ^
```

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-12 15:34:27 +00:00
Sébastien Han a6ef1ff490 ceph: enable pg auto repair
As a recommendation from the core team we should turn on PG auto repair
when running on Bluestore OSDs. The Quincy release will enable this by
default but in the meantime let's enable it here.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-12 09:18:00 +01:00
Travis Nielsen 93db66692d Merge pull request #6790 from subhamkrai/yaml-check
ci: add gh action for yaml linter
2021-03-11 18:06:48 -07:00
subhamkrai b46e31dbee ci: add gh action for yaml linter
this commit adds gh action for yaml linter.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-11 08:23:18 +05:30
Travis Nielsen c0eacc89d5 Merge pull request #7235 from travisn/mgr-sidecar
ceph: Allow two mgr daemons and actively reconcile the mgr services with a sidecar
2021-03-10 10:35:40 -07:00
Travis Nielsen c06a0090c3 Merge pull request #7354 from cybozu-go/ceph-fix-improper-json-parsing-in-radosgw-admin
ceph: fix improper json parsing
2021-03-10 10:13:19 -07:00
Satoru Takeuchi a89d35e5b6 ceph: fix improper json parsing in radosgw-admin
Sometimes `radosgw-admin` succeeds after showing logs to stderr. We should
skip non-json strings if parsing output as json.

Here is an example.

```
2021-02-26 04:10:44.190418 I | op-bucket-prov: creating Ceph user "ceph-user-aSzNqgE7"
E0226 04:11:37.901310       8 controller.go:199] error syncing 'logging/loki-bucket': error provisioning bucket: Provision: can't create ceph user: error creating ceph user "ceph-user-aSzNqgE7": failed to unmarshal json. 2021-02-26T04:11:21.425+0000 7f6714be4980  1 robust_notify: If at first you don't succeed: (110) Connection timed out
2021-02-26T04:11:21.426+0000 7f6714be4980  0 ERROR: failed to distribute cache for ceph-hdd-object-store.rgw.meta:users.uid:ceph-user-aSzNqgE7
2021-02-26T04:11:32.168+0000 7f6714be4980  1 robust_notify: If at first you don't succeed: (110) Connection timed out
2021-02-26T04:11:32.168+0000 7f6714be4980  0 ERROR: failed to distribute cache for ceph-hdd-object-store.rgw.meta:users.keys:23Z8GUEXR0TJDO86PSJR
{
    "user_id": "ceph-user-aSzNqgE7",
    "display_name": "ceph-user-aSzNqgE7",
...
    "mfa_ids": []
}: invalid character '-' after top-level value: failed to unmarshal json.
```

In this case, some logs like "robust_notify:..." was shown in stderr.
Unmarsharing was failed due to tried to parse these logs as json.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-10 16:06:56 +00:00