Commit Graph
927 Commits
Author SHA1 Message Date
Travis Nielsen 721acd1a8a Revert "ceph: added rook-ceph-default service account"
This reverts commit 737fb099fe.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-08 11:46:12 -06:00
Travis Nielsen 69bac95f83 Merge pull request #7540 from leseb/mon-maintenance
ceph: allow monitor maintenance mode
2021-04-08 09:06:27 -06:00
Sébastien Han e8c304d9b3 ceph: allow monitor maintenance mode
If the health spec of the CephCluster CRD has a timeout set to 0 like
so:

```
  healthCheck:
    daemonHealth:
      mon:
        disabled: false
        interval: 45s
        timeout: 0
```

And the mon goes out of quorum then Rook will not fail over the mon.
This is interesting when doing maintenance on a monitor and we don't
want to create a new one.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-08 16:27:53 +02:00
Travis Nielsen 665856b90a ceph: enable pacific as a supported ceph version
With the Ceph Pacific release coming this week we add support
in Rook for Pacific with the Rook v1.6 release coming soon.
The integration tests will now run across nautilus, octopus,
and pacific to cover all supported Ceph versions. The default
examples still specify Octopus until there is more bake time
for Pacific.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-07 06:59:04 -06:00
Sébastien Han 62d45239d4 ceph: use minutes instead of seconds
So we don't have to do the math :)

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-07 12:31:21 +02:00
rohan47 06aa82620b ceph: added rgw app label to appLabels for deleting stuck rook pods
Added the missing rgw app label to appLabels for deleting stuck rook pods

Signed-off-by: rohan47 <rohgupta@redhat.com>
2021-04-07 05:08:14 +05:30
Sébastien Han c411c1182e Merge pull request #7518 from travisn/obsolete-osd-dirs
ceph: Remove obsolete osd dirs config
2021-04-06 11:17:55 +02:00
Travis Nielsen c577eb80ee ceph: remove obsolete osd dirs config
OSD directories were removed in v1.2, this is some more cleanup
of dir configuration that is no longer needed.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-01 12:42:07 -06:00
Sébastien Han 51e7310e09 ceph: add dualstack support on pacific
With Pacific comes the support for dualstack where ceph daemons can
listen on both ipv4 and ipv6 stacks.
A new field in the network spec has been added: `dualStack`

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-31 18:04:29 +02:00
Blaine Gardner 0f6a5bac76 Merge pull request #7442 from sp98/mon-drain-on-failover
ceph: prevent voluntary mon drain while another mon is failing over
2021-03-30 09:40:14 -06:00
Travis Nielsen 3c37def52f Merge pull request #7406 from parth-gr/service-account
ceph: added rook-ceph-default service account
2021-03-30 08:06:40 -06:00
Sébastien Han 0c799c3b8f Merge pull request #7476 from subhamkrai/ceph-versions
ceph: update cephCluster CR with ceph versions output
2021-03-30 09:10:01 +02:00
Sébastien Han ec5ac91fe8 Merge pull request #7465 from leseb/type-duration
ceph: use metav1.Duration for time CRD field
2021-03-30 09:01:15 +02:00
Santosh Pillai 4af14f268c ceph: prevent voluntary mon drain while another mon is failing over
With simultaneous node drains, a mon can go down while another mon is failing over. This results in two mons down and ceph become inaccessible. The commit makes maxUnavailable=0 while a mon is failing over and updates it back to maxUnavailable=1 after the failover. This update action is best effort. Any errors while updating the maxUnavailable mon pdb is only logged

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-03-30 12:21:58 +05:30
Santosh Pillai 42121e909e ceph: update mon PDBs
- Update mon PDB to use maxUnavailable=1 instead of minAvailable. The maxUnavailable will always be 1 irrespective of the number of mons in the cluster
- Move mon PDB reconcile logic from Disruption Controller to Cluster Controller

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-03-30 12:21:21 +05:30
Blaine Gardner 1df4336a68 Merge pull request #7386 from BlaineEXE/update-osds-in-parallel
ceph: Update osds in parallel
2021-03-29 16:32:07 -06:00
Blaine Gardner 795124b7a8 ceph: update osds in parallel
Update OSDs in parallel per the design in
design/ceph/update-osds-in-parallel.md

The max number of OSDs updated in parallel is currently fixed at 20.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-03-29 10:55:28 -06:00
Blaine Gardner 660d2fe442 Merge pull request #7493 from BlaineEXE/codespell-tidy
test: document or clean up codespell flags
2021-03-29 10:07:53 -06:00
subhamkrai 05d4c2776c ceph: update cephCluster CR with ceph versions output
update cephCluster CR with ceph versions output.
this output will contain ceph version of the
ceph daemons which will help with upgrade status.

ceph versions command output
```
ceph versions
{
    "mon": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 3
    },
    "mgr": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "osd": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "mds": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 2
    },
    "rbd-mirror": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "rgw": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "overall": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 9
    }
}
```

CR status
```
status:
  ceph:
   ---
    versions:
      mds:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 2
      mgr:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
      mon:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 3
      osd:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
      overall:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 9
      rbd-mirror:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
      rgw:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
```

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-29 21:04:35 +05:30
parth-grandTareq Sharafy 737fb099fe ceph: added rook-ceph-default service account
When a private docker registry is used and an image pull secret is specified in the chart, the pods with default Service Account fail to pull the image due to authentication issues.
Added rook-ceph-default service account and modify the pods specifications by adding the serviceAccountName.

Closes: https://github.com/rook/rook/issues/6673
Co-authored-by: Tareq Sharafy <tareq.sha@gmail.com>
Signed-off-by: parth-gr <partharora1010@gmail.com>
2021-03-29 20:20:40 +05:30
Blaine Gardner 71c41180be test: document or clean up codespell flags
Document codespell flags that are necessary. Remove codespell flags with
minor code changes if possible.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-03-26 15:00:49 -06:00
Sébastien Han 30dd2218d0 Merge pull request #7488 from leseb/bz-1941647-2
ceph: remove extra slash in curl
2021-03-26 16:22:37 +01:00
Sébastien Han d9646d09ce ceph: remove extra slash in curl
When no VAULT_BACKEND_PATH is specified we default to the API one which
is set to "secret/", we could use a different variable but we don't want
to break the library support. Bonus point this will also fix issue with
users passing a slash at the end of the path.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-26 15:41:13 +01:00
Satoru Takeuchi 9eb3160d8b ceph: validate all owner references
Remaining work of #7259

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-26 06:43:54 +00:00
Sébastien Han d8de013443 ceph: use metav1.Duration for time CRD field
Previously, `interval` was just a string so no special validation was
done by the OpenAPI validator. Now that it is advertised as metav1.Duration
 the server will introspect it correctly. Underneath the type is still a
 string so no update issue to be worried about.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-25 12:12:31 +01:00
Sébastien Han 52c56cfde4 Merge pull request #7466 from leseb/multipath-encryption
ceph: fix init container "expand-encrypted-bluefs" multipath
2021-03-24 15:25:50 +01:00
Sébastien Han 5762ca43b2 ceph: fix init container "expand-encrypted-bluefs" multipath
When running encrypted OSDs on multipath the expand container was
failing to find the underlying device of the encrypted block. This was
because the underlying block was located in /dev/mapper and this path
wasn't mounted in the init container "expand-encrypted-bluefs".

This will manifest like so:

```
Underlying device for crypt device ocs-deviceset-block-dev-from-san-1-data-0-5qkmm-block-dmcrypt disappeared.
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-24 10:35:12 +01:00
Sébastien Han a394640286 ceph: remove temp encryption key
Once we are done opening the block device, we should remove the key even
though it is mounted on a tmpfs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-22 17:00:12 +01:00
Travis Nielsen c78e6a9e20 ceph: set active mgr label on mgr services
Other resources such as the service monitor need to match
the labels on the services to the active mgr, thus the active
mgr name is now set on the labels for the metrics service
and dashboard service.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-18 16:37:01 -06:00
subhamkraiandTravis Nielsen 319e4a41a4 ceph: placement in case of both PVC and non-PVC's
In the case of PVC,
We are giving lower priority to all placement.
We want deviceSet placement to applied and
override in case of overlapping settings and
we are merging nodeAffinity if applied in both
all placement and deviceSet.

In case of non-PVC,
we apply spec.placement

Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-18 10:32:57 +05:30
Travis Nielsen c23238cddb ceph: refactor integration tests for simplification
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 11:26:10 -06:00
Travis Nielsen 64e28af741 ceph: allow flex driver and discovery to be enabled with configmap
For testing purposes, we need to configure the flex driver
and discovery daemon with the operator settings configmap.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:26 -06:00
Travis Nielsen 933113eac8 ceph: remove obsolete storeType setting
All clusters are configured with bluestore as the store type
therefore we can remove the settings where mentioned in the
docs and dead code.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:25 -06:00
Travis Nielsen 4d969a406f Merge pull request #7414 from sp98/skip-cluster-cleanup
ceph: skip cleanup if cluster is not configured correctly.
2021-03-16 08:24:24 -06:00
Santosh Pillai 7bd05d0d7c ceph: skip cleanup if cluster is not configured correctly
deletion of cluster is stuck when cluster is not configured. The PR skips the cleanup and removes the finalizers if the cluster was not configured, that is, if clusterInfo could not be loaded.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-03-16 18:59:43 +05:30
Satoru Takeuchi 756d4ecb89 Merge pull request #7259 from cybozu-go/ceph-improve-owner-reference-management-2
ceph: improve owner reference management
2021-03-16 20:22:57 +09:00
Satoru Takeuchi 26c8fd9bd1 ceph: improve owner reference management
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-16 10:29:33 +00:00
Sébastien Han 31db03fece ceph: auto-gen crds
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:

* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`

will be autogenerated for us.

We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.

It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:

```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```

The corresponding API type markers are:

```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```

For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 09:16:24 +01:00
Sébastien Han 7291ce950e ceph: fix variable name
Fix linter complain.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 08:35:36 +01:00
Travis Nielsen 51996be952 ceph: remove extra mgr deployments if count is reduced
If the count of mgr daemons is reduced, we remove the extra one.
The check is made during each reconcile.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:18 -07:00
Travis Nielsen 38efe8ceba ceph: get active mgr with new pacific command
In Pacific the simple command to retrieve the active mgr is with
ceph mgr stat instead of retrieving the full ceph mgr dump.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:18 -07:00
Travis Nielsen a1c6a706f1 ceph: configure resources for the mgr sidecar
The mgr sidecar container has a new mgr-sidecar section
for specifying resources if desired.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
Travis Nielsen e7331b0191 ceph: multiple mgrs have antiaffinity across hosts or zones
If there are multiple mgrs running, the operator will automatically
add pod antiaffinity across hosts by default. For stretch clusters,
antiaffinity across the stretch failure domain (e.g. zones)
will be added. Required antiaffinity will be created unless the
test setting allowMultiplePerNode is set to true.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
Travis Nielsen 142ba63b6a ceph: enforce anti-affinity to multiple mgrs
If there are two mgr daemons, they should be running on different
hosts or zones, depending on the failure domain.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
Travis Nielsen 388ff3e78b ceph: start sidecar to monitor active mgr
The mgr daemon may be failed over by ceph if the active mgr is not
responding and the standby mgr is available. If the active mgr changes
the services for the dashboard and metrics will be updated with a
label selector for the new active mgr. The services cannot direct
traffic to the standby mgr or else they will be incorrectly redirected
to the active mgr.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
Travis Nielsen d93ff428ec ceph: reconcile the mgr services with the active mgr
The active mgr should match the labels on the services that
are available for prometheus and the dashboard. The selector
labels must be updated whenever there is a new active mgr.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:29:16 -07:00
ushen 7e691af439 ceph: allow user decide num of mgrs
Allow user to decide how many mgrs to be used
by make mgr replicas configurable

Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
2021-03-10 08:29:14 -07:00
Sébastien Han 33be24d1a1 ceph: fix vault when used with kv version 2
The API path when the KV enfine is used in version is different and must
be adapted. We need to append a 'data' path, thus the path and response
are changing.

v1 is https://vault.local.com:8200/v1/test-kv1/key
v2 is https://vault.local.com:8200/v1/test-kv2/data/key

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-09 16:36:26 +01:00
Sébastien Han 3dde2609fa ceph: add raw mode for non-pvc osd
For simple OSD scenario that don't require the following:

* an encrypted osd
* an osd with a backing device
* multiple osd on the same block device

The OSD will be prepared using the ceph-volume raw mode which does
not use LVM on non-pvc case.

Closes: https://github.com/rook/rook/issues/4768
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-03 14:11:26 +01:00
Travis Nielsen 15d0537cec ceph: only raise conditions that represent current state
The conditions on the cephcluster CR were being set to false
when not in progress, which is misleading to the purpose of
conditions. The conditions will now always represent current
state of the cluster. Transient conditions that show progress of
a reconcile will be removed from the status.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-02 14:15:47 -07:00