Commit Graph
330 Commits
Author SHA1 Message Date
Shachar Sharon 9766e9b8fb ceph: allow passing 'osd-crush-initial-weight'
Ceph support the option '--osd-crush-initial-weight' upon OSD start,
which sets an explicit weight (in TiB units) to specific OSD. Allow
passing this option all the way from the user (similar to
'DeviceClass'), for the special case where end users wants it cluster
to have non-even balance over specific OSDs (e.g., one of the OSDs is
placed over a partition alongside OS-partition).

ROOK issue: https://github.com/rook/rook/issues/7448

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2021-04-20 20:50:54 +03:00
Travis Nielsen 56e7af3ee0 Merge pull request #7331 from thotz/healthcheckwhensslenabled
ceph: fix healthcheck in case ssl enabled for rgw
2021-04-20 08:32:17 -06:00
Jiffin Tony Thottan bcbe707bcb ceph: fix healthcheck incase ssl enabled for rgw
In case ssl enabled for RGW, bucket healthcheck won't work since access is denied from the server.
In that case configure S3 agent using "InsecureSkipVerify: true" option.

Fixes: 7288
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-04-20 14:23:48 +05:30
Travis Nielsen 9da066eb3f ceph: allow pool replica min size 0 for EC pools
In v1.6 EC pools are being disallowed by the schema since the minimum
is set to 1 on the replicated.size. We expect the replicated.size to be
0 if an EC pool is being created.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-16 12:53:07 -06:00
Sébastien Han 20c302f72d ceph: use replicaset instead of deployments for rgw instances
Since Pacific, we can deploy multiple object gateway using the same
keying and they will appear in the service map separetly. So from now on
and on upgrades to Pacific Rook will remove all extra deployments to
only keep 1. In this single deployment the number of replica will be set
the desired `instances` count from the CRD spec.

You will now see gateways like this on Pacific:

```
rgw: 3 daemons active (1 hosts, 1 zones)
```

See an upgrade operator logs from Octopus to Pacific:

```
2021-04-14 10:05:16.883145 I | ceph-object-controller: setting multisite settings for object store "my-store"
2021-04-14 10:05:17.006492 I | ceph-object-controller: Multisite for object-store: realm=my-store, zonegroup=my-store, zone=my-store
2021-04-14 10:05:17.006516 I | ceph-object-controller: multisite configuration for object-store my-store is complete
2021-04-14 10:05:17.006523 I | ceph-object-controller: creating object store "my-store" in namespace "rook-ceph"
2021-04-14 10:05:17.006533 I | cephclient: getting or creating ceph auth key "client.rgw.my.store.a"
2021-04-14 10:05:17.298176 I | ceph-object-controller: object store "my-store" deployment "rook-ceph-rgw-my-store-a" started
2021-04-14 10:05:17.307913 I | ceph-object-controller: object store "my-store" deployment "rook-ceph-rgw-my-store-a" already exists. updating if needed
2021-04-14 10:05:17.317441 I | op-k8sutil: updating deployment "rook-ceph-rgw-my-store-a" after verifying it is safe to stop
2021-04-14 10:05:17.317463 I | op-mon: checking if we can stop the deployment rook-ceph-rgw-my-store-a
2021-04-14 10:05:25.552648 I | op-k8sutil: finished waiting for updated deployment "rook-ceph-mgr-a"
2021-04-14 10:05:25.552662 I | op-mon: checking if we can continue the deployment rook-ceph-mgr-a
2021-04-14 10:05:25.555077 I | op-mgr: setting services to point to mgr "a"
2021-04-14 10:05:25.608535 I | op-osd: start running osds in namespace "rook-ceph"
2021-04-14 10:05:25.608587 I | op-osd: wait timeout for healthy OSDs during upgrade or restart is "10m0s"
2021-04-14 10:05:25.618392 I | op-osd: start provisioning the OSDs on PVCs, if needed
2021-04-14 10:05:25.620512 I | op-osd: no storageClassDeviceSets or volumeSources are defined to configure OSDs on PVCs
2021-04-14 10:05:25.620542 I | op-osd: start provisioning the OSDs on nodes, if needed
2021-04-14 10:05:25.628416 I | op-osd: 1 of the 1 storage nodes are valid
2021-04-14 10:05:25.760702 I | op-k8sutil: Removing previous job rook-ceph-osd-prepare-minikube to start a new one
2021-04-14 10:05:25.773213 I | op-k8sutil: batch job rook-ceph-osd-prepare-minikube still exists
2021-04-14 10:05:26.595673 I | op-mgr: successful modules: prometheus
2021-04-14 10:05:27.129569 I | op-mgr: successful modules: mgr module(s) from the spec
2021-04-14 10:05:27.776720 I | op-k8sutil: batch job rook-ceph-osd-prepare-minikube deleted
2021-04-14 10:05:27.781881 I | op-osd: started OSD provisioning job for node "minikube"
2021-04-14 10:05:27.786693 I | op-osd: OSD orchestration status for node minikube is "starting"
2021-04-14 10:05:28.617937 I | op-mgr: successful modules: balancer
2021-04-14 10:05:28.706307 W | cephclient: all OSDs are running on the same host. not performing upgrade check. running in best-effort
2021-04-14 10:05:28.710245 I | op-osd: updating OSD 0 on node "minikube"
2021-04-14 10:05:31.596511 I | op-mgr: the dashboard secret was already generated
2021-04-14 10:05:31.926125 I | op-mgr: setting ceph dashboard "admin" login creds
2021-04-14 10:05:32.190343 E | op-mgr: failed modules: "dashboard". failed to initialize dashboard: failed to set login credentials for the ceph dashboard: failed to set login creds on mgr: failed to complete command for set dashboard creds: Invalid command: unused arguments: ["P.c(b6V$0pfK#)'70c5z"]
dashboard set-login-credentials <username> :  Set the login credentials. Password read from -i <file>
Traceback (most recent call last):
  File "/usr/bin/ceph", line 1310, in <module>
    retval = main()
  File "/usr/bin/ceph", line 1256, in main
    outf.write(outbuf)
TypeError: a bytes-like object is required, not 'str'
.
2021-04-14 10:05:35.665419 I | op-k8sutil: finished waiting for updated deployment "rook-ceph-rgw-my-store-a"
2021-04-14 10:05:35.665553 I | op-mon: checking if we can continue the deployment rook-ceph-rgw-my-store-a
2021-04-14 10:05:35.675293 I | ceph-object-controller: config map "rook-ceph-rgw-my-store-mime-types" for object store "my-store" already exists, not overwriting
2021-04-14 10:05:35.691549 I | ceph-object-controller: found more rgw deployments 3 than desired 3 in object store "my-store", scaling down
2021-04-14 10:05:35.691580 I | op-k8sutil: removing deployment rook-ceph-rgw-my-store-c if it exists
2021-04-14 10:05:35.698382 I | op-k8sutil: Removed deployment rook-ceph-rgw-my-store-c
2021-04-14 10:05:35.704568 I | op-k8sutil: "rook-ceph-rgw-my-store-c" still found. waiting...
2021-04-14 10:05:44.881129 I | op-osd: OSD orchestration status for node minikube is "orchestrating"
2021-04-14 10:05:44.882149 I | op-osd: OSD orchestration status for node minikube is "completed"
2021-04-14 10:05:45.756762 I | op-k8sutil: "rook-ceph-rgw-my-store-c" still found. waiting...
2021-04-14 10:05:45.782039 W | cephclient: all OSDs are running on the same host. not performing upgrade check. running in best-effort
2021-04-14 10:05:45.785188 I | op-osd: updating OSD 1 on node "minikube"
2021-04-14 10:05:54.768851 W | cephclient: all OSDs are running on the same host. not performing upgrade check. running in best-effort
2021-04-14 10:05:54.773191 I | op-osd: updating OSD 2 on node "minikube"
2021-04-14 10:05:55.843879 I | op-k8sutil: "rook-ceph-rgw-my-store-c" still found. waiting...
2021-04-14 10:06:05.255136 I | op-osd: finished running OSDs in namespace "rook-ceph"
2021-04-14 10:06:05.255155 I | ceph-cluster-controller: done reconciling ceph cluster in namespace "rook-ceph"
2021-04-14 10:06:05.560201 W | ceph-cluster-controller: upgrade orchestration completed but somehow we still have more than one Ceph version running. map[ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable):3 ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific (stable):6]:
2021-04-14 10:06:05.885001 I | op-k8sutil: "rook-ceph-rgw-my-store-c" still found. waiting...
2021-04-14 10:06:08.321209 I | exec: timeout waiting for process radosgw-admin to return. Sending interrupt signal to the process
2021-04-14 10:06:10.645672 I | ceph-spec: object "rook-ceph-rgw-my-store-c" matched on delete, reconciling
2021-04-14 10:06:11.940994 I | op-k8sutil: confirmed rook-ceph-rgw-my-store-c does not exist
2021-04-14 10:06:11.972935 I | ceph-spec: object "rook-ceph-rgw-my-store-c-keyring" matched on delete, reconciling
2021-04-14 10:06:11.974394 I | ceph-object-controller: deleting rgw CephX key and configuration in centralized mon database for "rook-ceph-rgw-my-store-c"
2021-04-14 10:06:13.721964 I | ceph-object-controller: successfully deleted rgw config for "client.rgw.my.store.c" in mon configuration database
2021-04-14 10:06:13.721989 I | cephclient: deleting ceph auth "client.rgw.my.store.c"
2021-04-14 10:06:14.073700 I | ceph-object-controller: completed deleting rgw CephX key and configuration in centralized mon database for "rook-ceph-rgw-my-store-c"
2021-04-14 10:06:14.073724 I | op-k8sutil: removing deployment rook-ceph-rgw-my-store-b if it exists
2021-04-14 10:06:14.083950 I | op-k8sutil: Removed deployment rook-ceph-rgw-my-store-b
2021-04-14 10:06:14.090226 I | op-k8sutil: "rook-ceph-rgw-my-store-b" still found. waiting...
2021-04-14 10:06:24.174846 I | op-k8sutil: "rook-ceph-rgw-my-store-b" still found. waiting...
2021-04-14 10:06:30.505574 I | ceph-spec: object "rook-ceph-rgw-my-store-b" matched on delete, reconciling
2021-04-14 10:06:32.201263 I | op-k8sutil: confirmed rook-ceph-rgw-my-store-b does not exist
2021-04-14 10:06:32.219559 I | ceph-object-controller: deleting rgw CephX key and configuration in centralized mon database for "rook-ceph-rgw-my-store-b"
2021-04-14 10:06:32.220219 I | ceph-spec: object "rook-ceph-rgw-my-store-b-keyring" matched on delete, reconciling
2021-04-14 10:06:33.821969 I | ceph-object-controller: successfully deleted rgw config for "client.rgw.my.store.b" in mon configuration database
2021-04-14 10:06:33.821992 I | cephclient: deleting ceph auth "client.rgw.my.store.b"
2021-04-14 10:06:34.182091 I | ceph-object-controller: completed deleting rgw CephX key and configuration in centralized mon database for "rook-ceph-rgw-my-store-b"
2021-04-14 10:06:34.188659 I | ceph-object-controller: successfully scaled down rgw deployments to 1 in object store "my-store"
2021-04-14 10:06:34.188676 I | ceph-object-controller: enabling rgw dashboard
2021-04-14 10:06:49.487902 I | exec: timeout waiting for process radosgw-admin to return. Sending interrupt signal to the process
2021-04-14 10:06:49.494605 W | ceph-object-controller: failed to enable dashboard for rgw. failed to create user "dashboard-admin": failed to create s3 user: signal: interrupt
2021-04-14 10:06:49.494676 I | ceph-object-controller: created object store "my-store" in namespace "rook-ceph"
```

Running pods:

```
...
...
rook-ceph-rgw-my-store-a-6756bdbbd4-7265v            1/1     Running     1          41m
rook-ceph-rgw-my-store-a-6756bdbbd4-l5ccr            1/1     Running     1          41m
rook-ceph-rgw-my-store-a-6756bdbbd4-m679c            1/1     Running     1          41m
...
...
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-14 13:38:29 +02:00
Umanga Chapagain 68c2db5d3a ceph: add monitoring key to cephcluster.spec.labels
CephCluster.Spec.Labels takes a key "monitoring". This is used
to inject labels into all the monitoring resources created by
the Rook operator.

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2021-04-12 12:02:27 +05:30
Blaine Gardner 20bc478844 Merge pull request #7547 from BlaineEXE/add-status-subresources
ceph: add status subresource to CRDS
2021-04-07 16:00:17 -06:00
Blaine Gardner b8aeef9f77 ceph: add status subresource to CRDS
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-07 15:36:41 -06:00
Blaine Gardner 3f447c8e08 ceph: allow empty compressionMode in CRD pools
Allow empty ("") compressionMode in CRDs' 'pool' spec.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-07 13:46:44 -06:00
Sébastien Han 51e7310e09 ceph: add dualstack support on pacific
With Pacific comes the support for dualstack where ceph daemons can
listen on both ipv4 and ipv6 stacks.
A new field in the network spec has been added: `dualStack`

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-31 18:04:29 +02:00
Sébastien Han 5415291fce Merge pull request #7502 from leseb/addition-column
ceph: add a few missing crd fields
2021-03-30 11:31:13 +02:00
Sébastien Han 670945cc0f ceph: add a few missing crd fields
Since the recent auto crds generation, I missed a few fields, re-adding
now:

* additionalPrinterColumns
* min and max

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-30 10:32:59 +02:00
Sébastien Han 0c799c3b8f Merge pull request #7476 from subhamkrai/ceph-versions
ceph: update cephCluster CR with ceph versions output
2021-03-30 09:10:01 +02:00
Sébastien Han ec5ac91fe8 Merge pull request #7465 from leseb/type-duration
ceph: use metav1.Duration for time CRD field
2021-03-30 09:01:15 +02:00
Blaine Gardner 795124b7a8 ceph: update osds in parallel
Update OSDs in parallel per the design in
design/ceph/update-osds-in-parallel.md

The max number of OSDs updated in parallel is currently fixed at 20.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-03-29 10:55:28 -06:00
subhamkrai 05d4c2776c ceph: update cephCluster CR with ceph versions output
update cephCluster CR with ceph versions output.
this output will contain ceph version of the
ceph daemons which will help with upgrade status.

ceph versions command output
```
ceph versions
{
    "mon": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 3
    },
    "mgr": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "osd": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "mds": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 2
    },
    "rbd-mirror": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "rgw": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 1
    },
    "overall": {
        "ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable)": 9
    }
}
```

CR status
```
status:
  ceph:
   ---
    versions:
      mds:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 2
      mgr:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
      mon:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 3
      osd:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
      overall:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 9
      rbd-mirror:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
      rgw:
        ceph version 15.2.9 (357616cbf726abb779ca75a551e8d02568e15b17) octopus (stable): 1
```

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-29 21:04:35 +05:30
Sébastien Han d8de013443 ceph: use metav1.Duration for time CRD field
Previously, `interval` was just a string so no special validation was
done by the OpenAPI validator. Now that it is advertised as metav1.Duration
 the server will introspect it correctly. Underneath the type is still a
 string so no update issue to be worried about.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-25 12:12:31 +01:00
subhamkraiandTravis Nielsen 319e4a41a4 ceph: placement in case of both PVC and non-PVC's
In the case of PVC,
We are giving lower priority to all placement.
We want deviceSet placement to applied and
override in case of overlapping settings and
we are merging nodeAffinity if applied in both
all placement and deviceSet.

In case of non-PVC,
we apply spec.placement

Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-18 10:32:57 +05:30
Frank Ritchie 7fcd0ab3d9 ceph: mark maxSize as optional
This is a follow up to:

https://github.com/rook/rook/pull/7397

Looks like I forgot to mark maxSize as optional.

Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
2021-03-17 09:43:31 -04:00
Sébastien Han 31db03fece ceph: auto-gen crds
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:

* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`

will be autogenerated for us.

We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.

It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:

```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```

The corresponding API type markers are:

```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```

For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 09:16:24 +01:00
Sébastien Han 7291ce950e ceph: fix variable name
Fix linter complain.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 08:35:36 +01:00
Frank Ritchie 9ae699d632 ceph: set pool quota with k8s quantity format
This is a follow up to:

https://github.com/rook/rook/pull/7264

which added pool quotas. It was stated that it would be nice to
be able to set pool max_bytes quotas using standard k8s quantity
formats rather than an integer representing the number of bytes.

I was too busy at the time to make the changes. This PR adds the
functionality.

Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
2021-03-15 19:26:41 -04:00
Travis Nielsen a1c6a706f1 ceph: configure resources for the mgr sidecar
The mgr sidecar container has a new mgr-sidecar section
for specifying resources if desired.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
Travis Nielsen e7331b0191 ceph: multiple mgrs have antiaffinity across hosts or zones
If there are multiple mgrs running, the operator will automatically
add pod antiaffinity across hosts by default. For stretch clusters,
antiaffinity across the stretch failure domain (e.g. zones)
will be added. Required antiaffinity will be created unless the
test setting allowMultiplePerNode is set to true.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
ushen 7e691af439 ceph: allow user decide num of mgrs
Allow user to decide how many mgrs to be used
by make mgr replicas configurable

Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
2021-03-10 08:29:14 -07:00
subhamkrai 40a99a20ff ceph: webhook check if one of port or securePort are non-zero
this commit adds new crd to admission webhooks to check
if one of the port or securePort are set to non-zero
value in objectstore crd.

If both are set to 0 in crd we'll see error like
```
Error from server (invalid create: either of port or securePort
fields should be not be zero): error when creating "object-test.yaml":
admission webhook
"cephobjectstore-wh-rook-ceph-admission-controller-rook-ceph.rook.io"
denied the request: invalid create: either of port or securePort fields
should be not be zero.
```

Signed-off-by: subhamkrai <srai@redhat.com>
2021-03-03 21:22:04 +05:30
Travis Nielsen 15d0537cec ceph: only raise conditions that represent current state
The conditions on the cephcluster CR were being set to false
when not in progress, which is misleading to the purpose of
conditions. The conditions will now always represent current
state of the cluster. Transient conditions that show progress of
a reconcile will be removed from the status.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-02 14:15:47 -07:00
Blaine Gardner efa7d66d45 ceph: remove driveGroup support
Remove features and design that supports adding OSDs to Ceph clusters
via `spec:driveGroups`. Update the Ceph upgrade doc that informs users
who currently use Drive Groups (we believe there are none of these
users) how to migrate to using the `spec:storage` config.

Resolves https://github.com/rook/rook/issues/7275

Revert "ceph: fix drive group deployment failure"
This reverts commit 76f1d9944e.

Revert "ceph: osd: add drive groups spec to cluster CR"
This reverts commit 7117fc12b7.

Revert "design: ceph orchestrator module add/remove OSDs"
This reverts commit 178187d035.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-02-25 17:08:47 -07:00
Travis Nielsen ade67eb7ce Merge pull request #7302 from BlaineEXE/update-to-ceph-15.2.9
ceph: update build and examples to use v15.2.9
2021-02-25 13:56:27 -07:00
Blaine Gardner 818342cbfb ceph: update build and examples to use v15.2.9
There are still references to v15.2.7 in integration tests to allow
testing OSDs on partitions.

The newest dated Ceph image is ceph/ceph:v15.2.9-20210224

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-02-24 15:15:26 -07:00
Travis Nielsen 9d15c104cc ceph: codegen update for pool quotas
The pool quotas were added in #7264, however the codegen CI
action did not catch that an update was needed until it
was backported.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-22 11:14:11 -07:00
Frank Ritchie a0488dac73 ceph: add ability to set pool quotas
This will allow setting ceph pool quotas in bytes and/or objects.

Ceph documentation: https://docs.ceph.com/en/latest/rados/operations/pools/#set-pool-quotas

Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
2021-02-19 13:49:25 -05:00
Travis Nielsen 8ce98c4b72 ceph: add tolerations for all daemons to the cleanup job
The cleanup job will need to run wherever the ceph daemons
were running. The scheduling is done by rook on the specific
nodes where the daemons were running, so there is no need for
other placement such as node or pod affinity.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-16 10:25:39 -07:00
subhamkrai 6158b6608c ceph: do not merge nodeAffinity
we don't want to merge the node affinity if specified in both
storageClassDeviceSet and placement. Now, we are passing new
boolean argument in `ApplyToPodSpec` which will be false in
case of osd and prepare pods because we don't want to overlap
placement of osd on PVC's and non-PVC's.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-02-16 12:25:28 +05:30
Travis Nielsen 043f947321 build: remove yugabytedb operator
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-06 00:03:41 -07:00
Sébastien Han c0123cf182 ceph: add cephfs mirroring support
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-28 19:21:18 +01:00
Santosh Pillai 4237aa7ecc ceph: configure health check timeout for OSDs when upgrading
added waitTimeoutForHealthyOSD in the cluster crd that defines the time (in minutes) the operator would wait before an OSD can be stopped for
upgrade. This PR also removes the ok-to-continue logic for osds as its already handled by ok-to-stop. The default value is 10 minutes.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-01-27 11:13:05 +05:30
Travis Nielsen 53ed11f15b build: remove the edgefs operator from rook
The EdgeFS operator has been deprecated for some time in Rook.
If the replacement is added back to Rook it can be completed
according to the new guidelines in the documentation.
https://rook.io/docs/rook/master/storage-providers.html

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-25 17:51:24 -07:00
Travis Nielsen 2294851ca3 build: remove the cockroachdb operator from rook
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-20 17:28:58 -07:00
Sébastien Han 758226294b ceph: convert CephClient CRD to controller-runtime
This was the last remaining CRD to not use the controller-runtime
library.
Small additions were added with the transition:

* the Kubernetes Secret that contains the CephX key has now an owner
  reference to the CephClient object
* the secret name is present in the Status field of the CephClient:

```
status:
  info:
    secretName: rook-ceph-client-glance
  phase: Ready
```

The controller will reconcile on CR updates and also if the Kubernetes
Secret is deleted.

Closes: https://github.com/rook/rook/issues/4938
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-15 16:17:38 +01:00
Sébastien Han 67c12a803e Merge pull request #6883 from jshen28/allow-disable-liveness-check-for-mds
ceph: allow disable liveness check for mds daemonset
2021-01-11 11:20:10 +01:00
ushen d3416fcf2c ceph: allow disable liveness check for mds daemonset
Allow to disable liveness probes for mds pod

Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
2021-01-11 17:55:51 +08:00
Julien Girardin d63d9b7d4d ceph: change external rgw detection, not relying on cluster
After #6217, internal rgw spawning on external cluster was still broken,
operator claiming:
```
ceph-object-controller: failed to reconcile failed to create object
store deployments: failed to reconcile external endpoint:
failed to create or update object store "arch-cloud" endpoint:
failed to create endpoint "rook-ceph-rgw-arch-cloud". Endpoints
"rook-ceph-rgw-arch-cloud" is invalid: subsets[0]:
Required value: must specify `addresses` or `notReadyAddresses`
```

To spawn internal rgw for external cluster, changes detection of
what mean 'internal' for rgw and only use the externalRgwEndpoints
list for that independently of the status of the cluster
Now there is multiple posibilities:

 * internal cluster and internal rgw pods: "normal case"
 * external cluster and internal rgw pods: <= now working with the PR
 * external cluster and external rgw pods: the external case
 * internal cluster and external rgw pods: <= new case that could exist

Signed-off-by: Julien Girardin <jugirardin@free.fr>
2021-01-08 11:28:41 +01:00
Travis Nielsen 3beb625f08 ceph: update examples and operator to Ceph v15.2.8
The latest octopus release is now out with v15.2.8. We update the
operator base image and the examples to run with this version.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-17 18:21:15 -07:00
Travis Nielsen eebb367e42 ceph: allow arbiter to have separate placement from other mons
The arbiter may have unique requirements for placement settings. Thus, if the arbiter
placement is specified in the cluster CR, that placement will be applied to the arbiter
mon and the other mons will retain their placement from the other mon or all
sections of the placement.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-07 10:58:20 -07:00
Travis Nielsen c083711c3d ceph: merge required node affinity for stretch clusters
Stretch clusters require certain node affinity for the mons and
must also be merged with the required node affinity in the placement
of the cluster CR. All conditions, both requested and generated must
be met, rather than one overriding the other.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-04 09:20:34 -07:00
Sébastien Han 8aaff235bb ceph: ability to set external services ports
When the cluster is external we want to expose manager service port
along with the endpoints.
This allows us to connect but Rook will keep on using 9283 as a facing
port for Prometheus and more.

```
monitoring:
  enabled: true
  ...
  ...
  externalMgrPrometheusPort: 9283
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-12-04 09:52:09 +01:00
Travis Nielsen 75af038309 Merge pull request #6726 from travisn/ceph-15.2.7
ceph: Update examples and base image to v15.2.7
2020-12-01 14:05:37 -07:00
Blaine Gardner 87ee8dce29 Merge pull request #6679 from leseb/add-log-collector
ceph: add log collector
2020-12-01 12:08:50 -07:00
Travis Nielsen 0a35a08f2f ceph: update examples and base image to v15.2.7
With the release of the latest octopus v15.2.7 we update the base
of the operator image and set the examples to use the same release
to pick up the security and other bug fixes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-12-01 12:05:39 -07:00