Commit Graph
163 Commits
Author SHA1 Message Date
subhamkrai a30338ae6e Revert "osd: add callback function in osd removal"
This reverts commit 927af7c7f7.

Signed-off-by: subhamkrai <srai@redhat.com>
2023-12-07 21:01:04 +05:30
subhamkrai 927af7c7f7 osd: add callback function in osd removal
Adding callback function in the osd removal method
as in downstream there is requirement of adding extra
check before proceeding with osd removal.

Signed-off-by: subhamkrai <srai@redhat.com>
2023-11-30 10:48:24 +05:30
guoguangwu 235ac293ff core: import packages only once
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-16 13:40:17 +08:00
sp98 11b8d10a5a osd: replace existing OSDs to use new store
This follow up the #12507
- fixes replacing of encrypted OSDs.
- Updates OSD status at the end of reconcile

Signed-off-by: sp98 <sapillai@redhat.com>
2023-09-01 21:07:38 +05:30
sp98 886bb357ef osd: replace osd to use new backend store
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk

Signed-off-by: sp98 <sapillai@redhat.com>
2023-08-17 21:40:55 +05:30
Travis Nielsen 5f2a8bca61 Merge pull request #12561 from henrydavies1/remove-duplicate-ParseMonEndpoints
mon: delete ParseMonEndpoints from mon package
2023-08-03 08:00:46 -06:00
Henry Davies 2c8949e195 mon: delete ParseMonEndpoints from mon package
The function ParseMonEndpoints existed twice, once in the mon package
and once in the controller package. It has now been deleted from the
mon package. Usages in the mon package now reference the function
in the controller package.

Signed-off-by: Henry Davies <henrydavies@hotmail.co.uk>
2023-08-01 19:36:28 +01:00
sp98 c13d34f8d3 osd: configure backend store
Configure osd backend store from the cephCluster spec

Signed-off-by: sp98 <sapillai@redhat.com>
2023-07-24 09:16:30 +05:30
Blaine Gardner 5a1d1f2175 Merge pull request #12069 from BlaineEXE/multus-golang-tester
test: add multus validation test routine to rook binary
2023-05-02 19:37:29 +02:00
Blaine Gardner 0f6e7ee921 test: add multus validation test routine to rook binary
Add a more involved multus validation test to the Rook binary. Because
this is intended to be end-user runnable, make sure operator-only
commands are hidden.

Build this into the rook binary instead of creating a separate binary
for ease, and because any binary built with the kube api becomes 40+
megabytes. We save quite a bit of space by including this in the Rook
binary, which is good for keeping container layers as small as possible.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2023-05-02 10:23:00 -06:00
Javier 884d5e9855 osd: allow to use filter device
this feature is to allow the use of filters using the deviceFilter flag
by skipping devices that do not match the filter

Closes: #10340
Signed-off-by: Javier <sjavierlopez@gmail.com>
2023-04-21 13:13:59 -06:00
Redouane Kachach f00bd790a8 mgr: using dynamic mgr_role label to implement mgr HA
Closes: https://github.com/rook/rook/issues/11844

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2023-03-27 18:38:55 +02:00
Redouane Kachach 456a0c328c Revert "mgr: remove mgr sidecar"
This reverts commit ff75ec96b7.

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2023-03-07 13:12:50 +01:00
Redouane Kachach ff75ec96b7 mgr: remove mgr sidecar
With the new mgr HA implementation (based on readiness probe) we
don't need the mgr sidecar (live-watch) anymore. This commit is
intended to remove all the related code.

Signed-off-by: Redouane Kachach <rkachach@redhat.com>
2023-02-21 17:25:39 +01:00
parth-gr a84daf9bf0 core: change io/ioutil package to use io and os package
few functions got change as they were deprecated
for ex: ioutil.Readfile change to os.Readfile
ioutil.TempFile change to os.CreateTemp
And fixed golang-ci-lint-issues

Signed-off-by: parth-gr <paarora@redhat.com>
2023-02-17 20:38:29 +05:30
Travis Nielsen 403335177a core: read mon secret from file instead of env var
Environment variables are not recommended for secrets in pods since
they can be easily leaked if the environemnt variables are logged.
By mounting the mon secret as a file, the mgr and osd prepare pods
can read the mon secret from a file for better security.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-12-14 16:24:44 -07:00
Travis Nielsen 33e824a323 core: remove unnecessary env vars from pod specs
The fsid, username, and user secrets were from legacy and no
longer needed on the osd daemon or mon pods. This removes the
unnecessary env vars from the pod specs.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-11-18 16:39:22 -07:00
Divyansh Kamboj 9008409f87 core: add context parameter to functions
This commit adds context parameter to various functions, and remove the
usage of context.TODO.

Closes: https://github.com/rook/rook/issues/8701
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
2022-03-22 08:19:07 +05:30
Travis Nielsen 7a6a62f43b Merge pull request #9386 from leseb/mgr-active-side-car-permissions
mgr: run the watch side car config on /etc/ceph
2021-12-15 12:20:28 -07:00
Sébastien Han 892c9a211c mgr: add missing context to the active-watch sidecar
The context was not initialized and thus the configmap fetch will fail
with a nil pointer.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-13 14:13:30 +01:00
Yuichiro Ueno 0d8c581aa9 core: add context parameter to k8sutil kvstore
This commit adds context parameter to k8sutil kvstore functions. By
this, we can handle cancellation during API call of kvstore resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-12-11 11:17:12 +09:00
Sébastien Han 7402c2cce6 osd: check if osd is ok-to-stop before removal
If multiple removal jobs are fired in parallel, there is a risk of
losing data since we will forcefully remove the OSD. It's also simply
true if a single OSD is not safe to destroy, there is also a risk of
data loss.

So now, we check if the OSD is safe-to-destroy first and then proceed.
The code waits forever and retries every minute unless the
--force-osd-removal flag is passed.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-11-25 10:21:32 +01:00
Travis Nielsen 1afd322650 core: ensure cluster name is available on cluster info
The cluster info is important context for the cluster controller to
create the cluster, and all the fields must be properly set.
A test cluster name was being set temporarily, resulting in
mons incorrectly getting the wrong cluster CR name. There is no
known issue from the temporary value, it was just exposed by
https://github.com/rook/rook/pull/8678 setting the value to a label.

Now the functions are more clearly named so only unit and
integration tests should be using the test value for the cluster
name where it is not important.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-11-18 15:30:26 -07:00
Sébastien Han c5783a77cf Merge pull request #9163 from y1r/add-context-k8sutil-node
core: add context parameter to k8sutil node
2021-11-15 16:25:13 +01:00
Satoru Takeuchi 07e1ca678a Merge pull request #9168 from cybozu-go/core-fix-unnecessary-option
core: remove unnecessary option
2021-11-15 22:50:08 +09:00
Yuichiro Ueno 4cc716a7ca core: add context parameter to k8sutil node
This commit adds context parameter to k8sutil node functions. By this,
we can handle cancellation during API call of node resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-15 22:45:59 +09:00
Satoru Takeuchi 77e7c249cc core: remove unnecessary option
rook command doesn't interpret `logtostderr` option. It's OK to just
remove this option because `capnslog` outputs all logs to stdout
by default. It's better to keep `AddGoFlagSet()` call because
some libraries might define their own flags with Go's `flag` package.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-11-15 11:53:45 +00:00
Yuichiro Ueno 0559977b8a core: add context parameter to k8sutil pod
This commit adds context parameter to k8sutil pod functions. By this, we
can handle cancellation during API call of pod resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-13 15:39:41 +09:00
Blaine Gardner d43327de2a docs: move purge osd to cluster namespace
The rook-ceph-purge-osd job should be run for a particular CephCluster
and should therefore be namespaced to the cluster.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-19 13:23:42 -06:00
Travis Nielsen 0a0b9c98bd build: remove obsolete flex driver
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:20 -06:00
Sébastien Han b89730d895 ceph: refactor operator initialization sequence
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:

* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited

As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.

A second new controller for the operator's general config has been
created, it manages:

* the logging level
* the ceph CLI command timeout
* the discovery daemon

The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-17 16:57:12 +02:00
Hiroya Onoe 2ff5413b75 ceph: fix error message in UpdateNodeStatus
The error message in UpdateNodeStatus regards the second argument
as node name. However, it is a PVC name in OSD on PVC.

Signed-off-by: Hiroya Onoe <onoehiroya@gmail.com>
2021-09-02 02:43:16 +00:00
Sébastien Han 41e915d411 Merge pull request #8493 from leseb/admission-controller
ceph: move the admission webhook to the operator
2021-08-25 09:30:54 +02:00
Sébastien Han 656dd0f334 ceph: move the admission webhook to the operator
Our admission webhooks will now run as part of the Operator container
and not an additional deployment. This has the advantage of consuming
fewer resources in the cluster and not having to manage affinities and
tolerations. This only drawback is that the Secret containing the
certificates is not mounted anymore and the content needs to be written
inside the Operator. This is not practical since we also need to watch
for the Secret content to change. Meaning that the certificates have
been renewed and the webhook server needs to use them.
A new approach is on its way to hopefully simplify this last issue and
implement a watcher for the Secret.
In the meantime, users need to use the cert-manager or renew
certificates manually. Additionally, they must update the
ValidatingWebhookConfiguration object with the new CA bundle.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 19:07:04 +02:00
Sébastien Han 579fc27783 ceph: embed ceph-csi templates in rook binary
Thanks to Golang 1.16, we can now embed files in the Go binary. This
means we don't need to add the CSI templates files to the container
image. They are added in the Go binary at build time.

The existing location of the template must be in the package calling it.
So they moved to pkg/operator/ceph/csi/template. All the files have been
symlinked back to cluster/examples/kubernetes/ceph/csi/template.

Closes: https://github.com/rook/rook/issues/7609
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 16:22:20 +02:00
Sébastien Han 07d14a93a2 ceph: remove cli unused flags
I don't know why these flags are there but it's not like we run the
operator with them and with a different value.
So removing for clarity.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-06 11:05:20 +02:00
Sébastien Han 8fdb9fe2b9 ceph: remove old network types
Those types are legacy, not really used anywhere and not needed anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-04 10:56:01 +02:00
Satoru Takeuchi 89b6a6028c ceph: add an option to preserve pvc in osd purge job
Sometimes we want to investigate a PVC for removed OSD.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-07-29 14:20:06 +00:00
Travis Nielsen 107f89260d ceph: reconcile mgr services with every reconcile
When there are multiple mgr daemons, the mgr sidecar owns reconciling
the services for the active mgr. However, for efficiency the sidecar
only reconciles the services if the active mgr changes. Now the operator
will also reconcile the mgr services to ensure that services are re-created
after being deleted, or otherwise in an incorrect state.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 07:23:58 -06:00
subhamkraiandSébastien Han 4241ce7e66 ceph: operator pod killed due to resource limit
when operator pod resources are set(which are
default in case of helm), operator pod are
killed due to using concurrency while creating
object store.

this commit checks if operator resources are set
or not. if set then we'll *not* create object store
in concurrency or if not set then we'll create in
concurrency.

Closes: https://github.com/rook/rook/issues/8149
Co-authored-by: Sébastien Han <seb@redhat.com>
Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-02 20:18:58 +05:30
Denis Egorenko fa2c276cc1 ceph: do not fail prepareOSD job if devices are not passed
If devices are not specified for node return empty desired devices
list and do not fail prepareOSD job.

Closes: #8097
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
2021-06-10 15:55:10 +04:00
Travis Nielsen eba91a4bd6 ceph: remove obsolete references to filestore
Filestore support has been long gone with only bluestore
currently supported.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-04 15:50:20 -06:00
Shachar Sharon 9766e9b8fb ceph: allow passing 'osd-crush-initial-weight'
Ceph support the option '--osd-crush-initial-weight' upon OSD start,
which sets an explicit weight (in TiB units) to specific OSD. Allow
passing this option all the way from the user (similar to
'DeviceClass'), for the special case where end users wants it cluster
to have non-even balance over specific OSDs (e.g., one of the OSDs is
placed over a partition alongside OS-partition).

ROOK issue: https://github.com/rook/rook/issues/7448

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2021-04-20 20:50:54 +03:00
Travis Nielsen 64e28af741 ceph: allow flex driver and discovery to be enabled with configmap
For testing purposes, we need to configure the flex driver
and discovery daemon with the operator settings configmap.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:26 -06:00
Travis Nielsen 933113eac8 ceph: remove obsolete storeType setting
All clusters are configured with bluestore as the store type
therefore we can remove the settings where mentioned in the
docs and dead code.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 08:39:25 -06:00
Satoru Takeuchi 26c8fd9bd1 ceph: improve owner reference management
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-03-16 10:29:33 +00:00
Travis Nielsen 38efe8ceba ceph: get active mgr with new pacific command
In Pacific the simple command to retrieve the active mgr is with
ceph mgr stat instead of retrieving the full ceph mgr dump.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:18 -07:00
Travis Nielsen 388ff3e78b ceph: start sidecar to monitor active mgr
The mgr daemon may be failed over by ceph if the active mgr is not
responding and the standby mgr is available. If the active mgr changes
the services for the dashboard and metrics will be updated with a
label selector for the new active mgr. The services cannot direct
traffic to the standby mgr or else they will be incorrectly redirected
to the active mgr.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-10 08:37:17 -07:00
Blaine Gardner efa7d66d45 ceph: remove driveGroup support
Remove features and design that supports adding OSDs to Ceph clusters
via `spec:driveGroups`. Update the Ceph upgrade doc that informs users
who currently use Drive Groups (we believe there are none of these
users) how to migrate to using the `spec:storage` config.

Resolves https://github.com/rook/rook/issues/7275

Revert "ceph: fix drive group deployment failure"
This reverts commit 76f1d9944e.

Revert "ceph: osd: add drive groups spec to cluster CR"
This reverts commit 7117fc12b7.

Revert "design: ceph orchestrator module add/remove OSDs"
This reverts commit 178187d035.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-02-25 17:08:47 -07:00
Travis Nielsen 291bf9e061 ceph: enforce portable osds in same topology as osd prepare job
The osd prepare job detects the failure domain for the OSD by querying
the node topology labels. This topology is then assigned to the OSD
daemon in the CRUSH map. Previously, the affinity was required to be
set in the cluster CR, but it was very difficult to get right. Now the
operator will enforce the correct topology label on the OSD daemon
nodeAffinity by setting the label of the lowest topology in the hierarcy.

For example, if there are region, zone, and rack labels, the rack label
would be used to set the node affinity for the OSD daemon. If an
OSD prepare job is run in rack1, the corresponding OSD daemon will
have node affinity to rack1 to ensure the same topology. Previously,
the OSD could have ended up in rack2 unless the storageClassDeviceSet
placement was very carefully crafted.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-18 12:49:37 -07:00