Commit Graph
297 Commits
Author SHA1 Message Date
subhamkrai 24802c559e core: fix golangci linter
fix golangci linter

Signed-off-by: subhamkrai <srai@redhat.com>
2022-04-04 20:59:31 +05:30
Sébastien Han 05506e7a68 core: reload go routine after CR is edited
Previously, the struct maintaining the list of cluster was still
initialized with a cluster item. Then the monitoring check will see that
the cluster is part of the struct already and thus won't run the
monitoring go routine again.
Now each time we cancel the context, we also remove the cluster item
from the map so that when the controller runs again, the monitoring
struct is re-populated and the go routine runs and statuses are updated.

Closes: https://github.com/rook/rook/issues/9911
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-03-31 08:49:42 +02:00
Divyansh Kamboj 9008409f87 core: add context parameter to functions
This commit adds context parameter to various functions, and remove the
usage of context.TODO.

Closes: https://github.com/rook/rook/issues/8701
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
2022-03-22 08:19:07 +05:30
Travis Nielsen 86eebc7486 mon: set the msgr1 port by default for csi driver
Older kernels still fail to connect for rbd/cephfs with
the msgr2 port due to a difference in http protocol expected.
For now, we keep the default port as msgr1 for the csi driver
and other client connections, unless msgr2 is strictly required
for encryption or compression.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-17 16:34:13 -06:00
Travis Nielsen c580e8f09c Merge pull request #9118 from travisn/network-encryption
mon: Options for enabling msgr2 encryption and compression
2022-03-11 09:49:55 -07:00
Travis Nielsen 22dea0a1ef mon: disable startup probe on canary pods
The canary pods are very short-lived pods and are not
running ceph in any form. The startup probe was mistakenly
enabled for the canary pod, which could cause the pod
to fail unexpectedly if the startup probe was triggered
after the delay timeout.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 12:19:55 -07:00
Travis Nielsen 03c7164b15 mon: options for enabling msgr2 encryption and compression
msgr2 allows for encryption and/or compression across the wire.
Settings for enabling the encryption and compression are now
available in the cluster CR to that ceph will be automatically
configured with these settings when desired.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 11:50:42 -07:00
Travis Nielsen 9f6804515c mon: enable msgr2 by default for new mons
New mons all support msgr2, so there is no need to keep the default
msgr1 port for new mons.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 11:48:47 -07:00
parth-gr 413662d475 core: change livenessProbe file name to probes
Added unit test for probes changes and
did re-naming of liveness probes file
as it will contain multiple types of probes

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-23 17:20:07 +05:30
parth-gr 1679af4a02 mds: override mds startup liveness probes from CephFilesystem CR
Rook code allows overriding the probes for mds from the CephCluster,
but this should really come from the CephFilesystem CR.

Closes: https://github.com/rook/rook/issues/9519

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-22 20:36:40 +05:30
Sébastien Han 0143a8f8ee core: remove unnecessary secret validation
We don't need to validate the name of Ceph-CSI secret, simply because
the names can be different. The default example have default names but
since the secret can be created by the administrator we should not
assume anything. This check will fail if the secret names are different
for whatever reason. This is especially true for external mode. In
converged mode Rook creates the secrets on its own if needed to ease the
integration with the default StorageClass examples.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-17 15:33:54 +01:00
Sébastien Han fdcad15b87 core: check for canceled context in external mode
This is fixing a potential race where the context is canceled and the
external cluster is trying to fetch the cluster info. If the context is
canceled then the loop will be stuck forever. Now we check for the
canceled context and return appropriately.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-17 15:30:25 +01:00
Sébastien Han 32998cbda4 core: do not dereference a non-pointer
In this case `oldMappingToGo` and `newMappingToGo` were not pointers so
there is nothing to dereference.
This happens when the monitors endpoints configmap is updated, but only
on the pool and file controllers. They are watching for changes on that
configmap so that they can reconcile and upate the bootstrap peers token
when replication is enabled.

Closes: https://github.com/rook/rook/issues/9714
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-10 17:11:38 +01:00
Sébastien Han 5404ec13a2 core: dereference pointer before trying to compare with deepequal
Prior to this, we were comparing a pointer (the memory address) with a
struct. This was obviously always failing and returned false. We must
dereference the pointer to access the data contained at that memory
location.

Closes: https://github.com/rook/rook/issues/9544
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-27 17:41:42 +01:00
Eng Zer Jun ed63571530 test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-24 23:52:54 +08:00
Sébastien Han 739abf6662 csi: change rook-ceph-csi-config to expose clusterID for subvolume
The rook-ceph-csi-config config map now includes CephFS subvolumegroups
when a subvolumegroup CR is created. It is useful for ceph-csi to
understand which subvolumegroup to use for a given cluster.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-21 10:12:56 +01:00
subhamkrai 02d43e82e0 mon: add annotation to mon secret and cm
add annotations to secrets and configmaps to use
different tools to sync secrets and configmap.
Currently, these annotation is applied only on
the `rook-ceph-mon-endpoints` configmap and
the secrets `rook-ceph-mon` and `rook-ceph-admin-keyring`.

Closes: https://github.com/rook/rook/issues/8999
Signed-off-by: subhamkrai <srai@redhat.com>
2022-01-19 10:15:17 +05:30
Blaine Gardner c07d89d9ea core: rgw: allow specifying daemon startup probes
Allow specifying daemon startup probes where we also allow configuring
liveness probes. Startup probes allow Rook to tolerate when Ceph daemons
occasionally take a long time to start up while not also making
Kubernetes liveness probes slower to detect runtime failures of daemons.

Startup probes are beta in Kubernetes 1.18, so we should not enable
probes by default for earlier Kubernetes versions.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-12-21 15:12:41 -07:00
Sébastien Han 97b41eacdc Merge pull request #9391 from y1r/add-context-k8sutil-pvc
core: add context parameter to k8sutil pvc
2021-12-13 10:07:11 +01:00
Yuichiro Ueno 03e976b52c core: add context parameter to k8sutil service
This commit adds context parameter to k8sutil service functions. By
this, we can handle cancellation during API call of service resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-12-11 11:40:28 +09:00
Yuichiro Ueno 5352cf58a7 core: add context parameter to k8sutil pvc
This commit adds context parameter to k8sutil pvc functions. By this, we
can handle cancellation during API call of pvc resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-12-11 11:32:46 +09:00
Sébastien Han 4f1a2d47a4 core: fix error message
Let's have a more accurate and correct error message that reflects what
the failing function tried to do.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-08 17:30:49 +01:00
Sébastien Han fdd243d3de core: always return nil on error
We should always return a nil pointer of clusterInfo if
CreateOrLoadClusterInfo() returns an error.

Closes: https://github.com/rook/rook/issues/9314
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-08 17:30:07 +01:00
parth-gr 0a86d26b2e core: create rook resources with k8s recommended labels
Adding Recommended Labels on the resources created by rook
    and using Recommended Labels in the helm chart,
    for better visuals and management of k8s object

Closes: https://github.com/rook/rook/issues/8400
Signed-off-by: parth-gr <paarora@redhat.com>
2021-12-07 18:16:44 +05:30
Travis Nielsen d55669293a mon: set stretch tiebreaker reliably during failover
The failover of the arbiter mon in a stretch cluster was sometimes
failing due to the new tiebreaker not being set in ceph.
Rook would repeatedly try to remove the old tiebreaker mon
and keep failing because the new tiebreaker had not been set.
Now we make setting the tiebreaker idempotent in case the operator
restarts in the middle of the operation or some other corner
case causes the expected tiebreaker to be set. In that case,
the next reconcile will also ensure the tiebreaker mon is
set as expected.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-12-02 12:50:07 -07:00
Travis Nielsen 1afd322650 core: ensure cluster name is available on cluster info
The cluster info is important context for the cluster controller to
create the cluster, and all the fields must be properly set.
A test cluster name was being set temporarily, resulting in
mons incorrectly getting the wrong cluster CR name. There is no
known issue from the temporary value, it was just exposed by
https://github.com/rook/rook/pull/8678 setting the value to a label.

Now the functions are more clearly named so only unit and
integration tests should be using the test value for the cluster
name where it is not important.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-11-18 15:30:26 -07:00
Travis Nielsen 54a2b56b0c Merge pull request #9167 from parth-gr/mons-CRname
mon: update mons Cluster ClusterInfo with CR name
2021-11-15 11:11:09 -07:00
parth-gr 242f98e94b mon: update mons Cluster ClusterInfo with CR name
the mons re-initialize its ClusterInfo which results in missing
CR name on the ClusterInfo
Set the CR name to the mons cluster ClusterInfo

Closes: https://github.com/rook/rook/issues/9159
Signed-off-by: parth-gr <paarora@redhat.com>
2021-11-15 18:52:55 +05:30
Sébastien Han ecd7fa7880 Merge pull request #9164 from y1r/add-context-k8sutil-pod
core: add context parameter to k8sutil pod
2021-11-15 11:58:36 +01:00
Yuichiro Ueno 0559977b8a core: add context parameter to k8sutil pod
This commit adds context parameter to k8sutil pod functions. By this, we
can handle cancellation during API call of pod resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-13 15:39:41 +09:00
Yuichiro Ueno 0b575703c7 core: add context parameter to k8sutil deployment
This commit adds context parameter to k8sutil deployment functions. By
this, we can handle cancellation during API call of deployment resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-13 14:58:13 +09:00
subhamkrai 0150966024 ceph: remove ceph nautilus, ceph octopus to default
since rook 1.8, ceph nautilus no longer supported,
ceph octopus will be the minimum ceph version.

Closes: https://github.com/rook/rook/issues/7908
Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 14:45:12 +05:30
Travis Nielsen 12689bd119 ceph: enable mon failover for the arbiter in stretch mode
Prior to ceph v16.2.7 the failover of the arbiter mon was
not supported. Now the new tiebreaker mon can be set during
the failover event and provide more dynamic stability to
the mon quorum if another node is available in the arbiter
zone.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-10-18 08:58:25 -06:00
parth-gr 7c99858a77 ceph: add finalizers to rook-ceph-mon secrets and configmap
Adding finalizers to rook-ceph-mon secrets
and rook-ceph-mon-endpoints configmap
We don't want to delete this resources during disaster
because these details are needed during disaster recovery

Closes: https://github.com/rook/rook/issues/8369
Signed-off-by: parth-gr <paarora@redhat.com>
2021-10-07 19:44:17 +00:00
Sébastien Han 121c2987e3 ceph: stop using tini
We don't need to use tini.
We don't have anything in the rook operator that would
either create zombie processes (no threads) or use
exec (to fork). The Go binary has a really good
signal handling mechanism.

Closes: https://github.com/rook/rook/issues/8794
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-27 10:54:16 +02:00
Travis Nielsen 0a0b9c98bd build: remove obsolete flex driver
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:20 -06:00
Sébastien Han b89730d895 ceph: refactor operator initialization sequence
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:

* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited

As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.

A second new controller for the operator's general config has been
created, it manages:

* the logging level
* the ceph CLI command timeout
* the discovery daemon

The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-17 16:57:12 +02:00
Sébastien Han 031be4be19 Merge pull request #8675 from subhamkrai/ok-continue
ceph: modify the log info when ok to continue fails
2021-09-09 17:22:13 +02:00
subhamkraiandZeaone 0657804491 ceph: modify the log info when ok to continue fails
correct typo in logging, it was showing `ok-to-stop`
instead of `ok-to-continue` when 'continueUpgradeAfterChecksEvenIfNotHealthy' is true

Co-Authored-by: Zeaone <zeaone@ZeaonedeMacBook-Pro.local>
Signed-off-by: subhamkrai <srai@redhat.com>
2021-09-09 19:42:06 +05:30
JrCs c606f4c488 ceph: use node externalIP if no internalIP defined
In some cases node internalIP is not defined. Then use externalIP if it
exists.

Signed-off-by: JrCs <90z7oey02@sneakemail.com>
2021-09-08 16:26:05 +02:00
Travis Nielsen 1cb97574ea ceph: allow an even number of mons
While an even number of mons can cause lower availability of
mon quorum, it also can provide higher durability for the cluster.
Mon quorum can be restored from a single mon according to the
disaster recovery guide, so there may be scenarios where
an even number of mons may be preferable.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-03 07:43:32 -06:00
Travis Nielsen f9f322683a ceph: refuse to failover the arbiter mon on stretch clusters
Stretch clusters in Ceph do not yet support failing over the arbiter
mon, so we now disable the arbiter mon from failing over. Soon
Ceph will support the arbiter mon failover and we will reenable
this failover feature.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-10 16:26:47 -06:00
Sébastien Han 07d14a93a2 ceph: remove cli unused flags
I don't know why these flags are there but it's not like we run the
operator with them and with a different value.
So removing for clarity.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-06 11:05:20 +02:00
Sébastien Han 630c2f6a8b ceph: add an rbd-mirror bootstrap token on cluster creation
They are scenarios where the mirroring information want to be shared
between clusters prior to creating pool. Because the bootstrap peer
import command needs a pool name to operate this is not suitable. So
additionally now each time the cluster is reconciled and on any new
clusters a new secret will be created that contains a boostrap peer
token. It can be exchanged with another cluster.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-02 18:34:27 +02:00
parth-gr 6ebd109088 ceph: updated mon health check goroutine for reconfiguring patch values
When changing the settings for the Rook mon health check
(.spec.healthCheck.mon) in the CephCluster resource,
the mon health check isn't reconfigured with the new values

Updated checkHealth goroutine so it can
updates the new patched values

Closes: https://github.com/rook/rook/issues/8363
Signed-off-by: parth-gr <paarora@redhat.com>
2021-07-26 22:51:37 +05:30
Sébastien Han d55fb8d13b Merge pull request #8351 from leseb/refact-exec-helpers
ceph: remove unnecessary exec helpers
2021-07-23 18:12:57 +02:00
Sébastien Han 0811359d28 Merge pull request #8358 from leseb/move-to-quay
ceph: move all of our docker.io reference to quay.io
2021-07-23 18:03:53 +02:00
Sébastien Han 6d77a9976c ceph: remove unnecessary exec helpers
Both `ExecuteCommandWithOutputFileTimeout()` and
`ExecuteCommandWithOutputFile()` generate unnecessary system calls by
creating/reading/removing files where the stream output of the command
can simply be used. So sticking with `ExecuteCommandWithOutput()` and
`ExecuteCommandWithCombinedOutput()` for reading outputs is sufficient.

Closes: https://github.com/rook/rook/issues/8343
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-23 09:16:33 +02:00
Sébastien Han 6bce1ff3e9 ceph: move all of our docker.io reference to quay.io
Recently, the builds of `ceph/ceph` image moved to quay.io, see
https://github.com/ceph/ceph-build/pull/1883 for more details.
Current images will remain but new builds will happen on quay.io only.

This means that tags such as `v14.2`, `v15.2`,`v16.2` will need to
switch to quay.io to get updates.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-22 11:17:05 +02:00
parth-gr 77813f7093 ceph: update PodDisruptionBudget from v1beta1 to v1
This commit update the PodDisruptionBudget policy to use version v1
Updated to policy/v1 as policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+

Closes: https://github.com/rook/rook/issues/7917
Signed-off-by: parth-gr <paarora@redhat.com>
2021-07-22 14:09:33 +05:30