Commit Graph
3048 Commits
Author SHA1 Message Date
Sébastien Han 9744f0f2c6 Merge pull request #8583 from leseb/cancel-retry
ceph: do not check ok-to-stop when OSDs are in CLBO
2021-08-25 17:38:06 +02:00
Travis Nielsen b528310b79 Merge pull request #8590 from travisn/mon-store-config
ceph: Consolidate the calls to set mon config
2021-08-25 08:31:20 -06:00
Travis Nielsen edb8aa1834 Merge pull request #8584 from parth-gr/string-instance1
core: convert util.NewSet() to sets.NewString()
2021-08-25 08:30:24 -06:00
parth-gr b1cfeb9dbe core: removing util.set package files
We are no longer using package util.set
for creating instances,
and instead of that using sets.String package

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-25 13:15:14 +05:30
parth-gr 77ba1ef11b core: convert util.NewSet() to sets.NewString()
Converting util.NewSet() instance to use sets.NewString() instance

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-25 13:13:05 +05:30
Sébastien Han 41e915d411 Merge pull request #8493 from leseb/admission-controller
ceph: move the admission webhook to the operator
2021-08-25 09:30:54 +02:00
Sébastien Han 88048cc4e8 ceph: do not check ok-to-stop when OSDs are in CLBO
This handles the scenario where the OSDs have been created but not yet
started due to a wrong CR configuration.
For instance, when OSDs are encrypted and Vault is used to store
encryption keys, if the KV version is incorrect during the cluster
initialization the OSDs will fail to start and stay in CLBO until the
CR is updated again with the correct KV version so that it can start.

For this scenario, if the CRUSH map has no host registered yet it's fair
to assume the initialization broke and we need to fix it. So when don't
need to call ok-to-stop since it will always fail and eventually force
pass but let's not wait for nothing.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-25 09:30:38 +02:00
Travis Nielsen cdfe1982d1 ceph: consolidate the calls to set mon config
The mon config had two different implementations that have evolved
over the lifetime of the project. This is a simple refactor to remove
the SetConfig() option and stick with the MonStore as a single
implementation for updating the mon store.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-24 14:38:26 -06:00
Travis Nielsen 7af10af781 Merge pull request #8514 from thotz/obcupdateapi
ceph: add support for update() from lib-bucket-provisioner
2021-08-24 13:28:46 -06:00
Sébastien Han 656dd0f334 ceph: move the admission webhook to the operator
Our admission webhooks will now run as part of the Operator container
and not an additional deployment. This has the advantage of consuming
fewer resources in the cluster and not having to manage affinities and
tolerations. This only drawback is that the Secret containing the
certificates is not mounted anymore and the content needs to be written
inside the Operator. This is not practical since we also need to watch
for the Secret content to change. Meaning that the certificates have
been renewed and the webhook server needs to use them.
A new approach is on its way to hopefully simplify this last issue and
implement a watcher for the Secret.
In the meantime, users need to use the cert-manager or renew
certificates manually. Additionally, they must update the
ValidatingWebhookConfiguration object with the new CA bundle.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 19:07:04 +02:00
Travis Nielsen 9a96afab05 Merge pull request #8582 from Madhu-1/fix-csidriver-panic
ceph: fix panic when recreating the csidriver object
2021-08-24 09:05:31 -06:00
Sébastien Han 579fc27783 ceph: embed ceph-csi templates in rook binary
Thanks to Golang 1.16, we can now embed files in the Go binary. This
means we don't need to add the CSI templates files to the container
image. They are added in the Go binary at build time.

The existing location of the template must be in the package calling it.
So they moved to pkg/operator/ceph/csi/template. All the files have been
symlinked back to cluster/examples/kubernetes/ceph/csi/template.

Closes: https://github.com/rook/rook/issues/7609
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 16:22:20 +02:00
Madhu Rajanna d246dde443 ceph: remove variable declaration
remove csiDriver and csiClient variables
creation and reuse them from the method
receivers.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-24 19:43:18 +05:30
Madhu Rajanna e2dc418371 ceph: fix logging of csidriver
log the successful message about
starting CSIDriver after creating
the daemonset and deployment objects.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-24 19:41:04 +05:30
Madhu Rajanna d48f67a82c ceph: fix panic in reCreateCSIDriverInfo
Initialize the client and the csidriver object
before calling the reCreateCSIDriverInfo
function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-24 12:15:48 +05:30
Satoru Takeuchi 6582bac8e7 Merge pull request #8441 from leseb/signal
ceph: signal signals handling with context
2021-08-24 14:01:29 +09:00
parth-gr 72e4bb8099 ceph: remove legacy PDBs methods
Removing the deleteLegacyPDBForOSD method
because we don't support legacyPDBs

Closes: https://github.com/rook/rook/issues/8360
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-20 21:24:52 +05:30
parth-gr 1601ba1c1c core: convert util.NewSet() to sets.NewString()
Converting util.NewSet() instance to use sets.NewString() instance

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-20 18:56:25 +05:30
Jiffin Tony Thottan f4bb47e440 ceph: add support for update() from lib-bucket-provisioner
Recently lib-bucket-provisioner add support for update() API.
Include that on the obc implementation since it can be used to
update quota for OBC.

Fixes: #7146

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-08-19 23:57:11 +05:30
Blaine Gardner 36c53fe6f3 ceph: make storage device config nullable
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-17 12:21:32 -06:00
Ali Maredia 7a804ac27e ceph: tweak to types.go and additions after make crds
This commit will be squashed.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2021-08-11 13:25:44 -04:00
Ali Maredia 7b8d3fd27b ceph: change required pattern for pull realm endpoint
Ensure that the endpoint for the pull realm spec
starts with "http" or "https". Also update
object-multisite-pull-realm(-test).yaml with
updated comments.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2021-08-11 13:25:44 -04:00
Travis Nielsen 993b82674f Merge pull request #8520 from travisn/no-arbiter-failover
ceph: Refuse to failover the arbiter mon on stretch clusters
2021-08-11 07:16:25 -06:00
Travis Nielsen f9f322683a ceph: refuse to failover the arbiter mon on stretch clusters
Stretch clusters in Ceph do not yet support failing over the arbiter
mon, so we now disable the arbiter mon from failing over. Soon
Ceph will support the arbiter mon failover and we will reenable
this failover feature.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-08-10 16:26:47 -06:00
Denis Egorenko fb04908315 ceph: add ability to specify ca bundle for rgw
Specify ca Bundle for RGW spec and mount inside pods.

Related-Issue: https://github.com/rook/rook/issues/8490
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
2021-08-10 23:18:26 +04:00
Sébastien Han e4ac38e555 Merge pull request #8482 from leseb/restrict-manager-cache
ceph: move scheme initialization to the same place
2021-08-06 17:45:30 +02:00
Travis Nielsen 91b7741ebb Merge pull request #8489 from leseb/rm-unused-flags
ceph: Remove unused mon timeout cli flags
2021-08-06 09:01:16 -06:00
Sébastien Han 2d55e69416 ceph: move scheme initialization to the same place
Let's initialize the schemes in a single place instead of doing it
when each controller initializes.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-06 11:33:10 +02:00
Sébastien Han 07d14a93a2 ceph: remove cli unused flags
I don't know why these flags are there but it's not like we run the
operator with them and with a different value.
So removing for clarity.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-06 11:05:20 +02:00
Sébastien Han bb4e572d2f Merge pull request #8465 from parth-gr/import-rbd-mirror
ceph: use mktemp file in ImportRBDMirrorBootstrapPeer
2021-08-05 09:44:43 +02:00
Sébastien Han df432fd822 Merge pull request #8475 from leseb/rm-net-type
ceph: remove old network types
2021-08-04 18:45:45 +02:00
parth-gr 48918c7cf0 ceph: use mktemp file in ImportRBDMirrorBootstrapPeer
Do not point to a made-up file in /tmp
Use ioutil.TempFile() for creating a token file

Closes: https://github.com/rook/rook/issues/8446
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-04 19:22:34 +05:30
Travis Nielsen cc16b60f5f Merge pull request #8381 from subhamkrai/osd-placement-merge
ceph: add an option to enable/disable merge all placement
2021-08-04 07:23:33 -06:00
Sébastien Han 1ddf28144a Merge pull request #8459 from humblec/cephfs-fsgroup
ceph: set default FsGroupChangePolicy value to 'None'
2021-08-04 11:03:41 +02:00
Sébastien Han 8fdb9fe2b9 ceph: remove old network types
Those types are legacy, not really used anywhere and not needed anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-04 10:56:01 +02:00
subhamkrai 9838313398 ceph: add an option to enable/disable merge all placement
earlier, we had the issue for osd placement regarding
whether we should merge the placements or not.

Now, we have option `skipApplyAllPlacement` to
enable/disable osd placement. By default it is false
which means it will merge both placement.All() and
storageClassDeviceSets.Placement, when true
it will only apply storageClassDeviceSets.Placement.

Also, adding unit test.

Closes: https://github.com/rook/rook/issues/8135
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-03 19:47:40 +05:30
Sébastien Han 412b3eaf5e Merge pull request #8447 from leseb/fix-8438
ceph: ignore errors when mirroring is not enabled on the filesystem
2021-08-03 16:06:16 +02:00
Sébastien Han 9e63aed974 Merge pull request #8460 from leseb/add-fs-mirror-cleanup
ceph: add fs-mirror to cleanup list
2021-08-03 16:05:59 +02:00
Sébastien Han 7318423506 ceph: ignore errors when mirroring is not enabled on the filesystem
Trying to disable mirroring on a cluster where mirroring is not enabled
results in an error during upgrades. Let's catch this error and ignore
it.

Funny enough the exec error resembles to:

```
Error ENOTSUP: Module 'mirroring' is not enabled (required by command 'fs snapshot mirror disable'): use `ceph mgr module enable mirroring` to enable it: exit status 95
```

So we get ENOTSUP which is 45 but exit status 95...

Closes: https://github.com/rook/rook/issues/8438
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-03 15:33:04 +02:00
Sébastien Han e1cd82beb3 ceph: add fs-mirror to cleanup list
Before running the cleanup job we now also wait for the fs-mirror pods
to be gone.
Also, small refactor to print nicely the list of remaining daemons.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-03 15:22:36 +02:00
Humble Chirammal 8034a4ef98 ceph: set default FsGroupChangePolicy value to 'None'
The `None` value Indicates that volumes will be mounted
with no modifications, as the CSI volume driver does not
support these operations. While volumes are provisioned
by the CephFS CSI driver the global permissions are set
on the volume and we dont expect the Fsgroup policy or
check from CO side  to play a role here.

Ref #ceph/ceph-csi/../internal/cephfs/nodeserver.go#L190

```
!csicommon.MountOptionContains(fuseMountOptions, readOnly) {
		// #nosec - allow anyone to write inside the stagingtarget path
		err = os.Chmod(stagingTargetPath, 0o777)
```

The current default value ie `ReadWriteOnceWithFSType` cause
volumes to be examined to determine if volume ownership and permissions
should be modified to match the pod's security policy. Changes could occur
if the fsType is defined and the persistent volume's accessModes contains
ReadWriteOnce.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-03 16:54:10 +05:30
Sébastien Han 73b0ce08ff ceph: do not stack trace on error
We should not use %+v when printing errors and wrapping it in the
caller. Passing %+v to errors.Wrap() returns a strack trace.

See:

```
2021-08-03 09:42:40.460946 E | ceph-file-controller: failed to reconcile failed to create filesystem "myfs2": failed to start deployment for MDS "a" for filesystem "myfs2": failed to update mds deployment "rook-ceph-mds-myfs2-a": failed to check if deployment "rook-ceph-mds-myfs2-a" can continue: failed to check if we can continue the deployment rook-ceph-mds-myfs2-a: failed to check if rook-ceph-mds-myfs2-a was ok to continue: max retries exceeded, last err: mds myfs2-a is up:creating, bad state
github.com/rook/rook/pkg/daemon/ceph/client.MdsActiveOrStandbyReplay
        /home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/status.go:286
github.com/rook/rook/pkg/daemon/ceph/client.okToContinueMDSDaemon.func1
        /home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/upgrade.go:215
github.com/rook/rook/pkg/util.Retry
        /home/leseb/go/src/github.com/rook/rook/pkg/util/retry.go:31
github.com/rook/rook/pkg/daemon/ceph/client.okToContinueMDSDaemon
        /home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/upgrade.go:214
github.com/rook/rook/pkg/daemon/ceph/client.OkToContinue
        /home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/upgrade.go:181
github.com/rook/rook/pkg/operator/ceph/cluster/mon.UpdateCephDeploymentAndWait.func1
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/cluster/mon/spec.go:370
github.com/rook/rook/pkg/operator/k8sutil.UpdateDeploymentAndWait
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/k8sutil/deployment.go:113
github.com/rook/rook/pkg/operator/ceph/cluster/mon.UpdateCephDeploymentAndWait
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/cluster/mon/spec.go:383
github.com/rook/rook/pkg/operator/ceph/file/mds.(*Cluster).startDeployment
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/mds/mds.go:216
github.com/rook/rook/pkg/operator/ceph/file/mds.(*Cluster).Start
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/mds/mds.go:138
github.com/rook/rook/pkg/operator/ceph/file.createFilesystem
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/filesystem.go:81
github.com/rook/rook/pkg/operator/ceph/file.(*ReconcileCephFilesystem).reconcileCreateFilesystem
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/controller.go:373
github.com/rook/rook/pkg/operator/ceph/file.(*ReconcileCephFilesystem).reconcile
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/controller.go:288
github.com/rook/rook/pkg/operator/ceph/file.(*ReconcileCephFilesystem).Reconcile
        /home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/controller.go:167
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /home/leseb/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /home/leseb/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /home/leseb/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:214
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1371
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-03 11:48:54 +02:00
Sébastien Han 8556f3fe26 ceph: remove pool id from the peer
We don't need to put this information in the token.
It's not useful and not used anywhere.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-03 09:44:54 +02:00
Sébastien Han 1c8caa2270 Merge pull request #8379 from leseb/manual-peer-bootstrap
ceph: add an rbd-mirror bootstrap token on cluster creation
2021-08-02 19:03:29 +02:00
Sébastien Han 630c2f6a8b ceph: add an rbd-mirror bootstrap token on cluster creation
They are scenarios where the mirroring information want to be shared
between clusters prior to creating pool. Because the bootstrap peer
import command needs a pool name to operate this is not suitable. So
additionally now each time the cluster is reconciled and on any new
clusters a new secret will be created that contains a boostrap peer
token. It can be exchanged with another cluster.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-02 18:34:27 +02:00
Sébastien Han 4e2879a154 ceph: signal signals handling with context
As of Golang 1.16, we can use `signal.NotifyContext()` from the signal
package. Essentially it combines signal handling for termination as well
as canceling the context to stop any ongoing operations.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-02 14:19:50 +02:00
Madhu Rajanna b556dbf109 ceph: update cephcsi to v3.4.0
This PR updates the required RBAC, templates,
CSI image version and examples for new
cephcsi v3.4.0 release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-02 10:22:54 +05:30
Sébastien Han 082858bf53 ceph: use v16 for mirroring test in ci
The CI test for mirroring now runs on stable v16 tag. Internally our
code has a minimum version of 16.2.5 for cephfs mirroring since it's
available.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-30 16:17:08 +02:00
Sébastien Han 7ef127b816 ceph: append additional info in the rbd-mirror bootstrap peer token
If the title looks familiar this is normal, this piece of code was
removed during https://github.com/rook/rook/pull/7604. Probably due to a
rebase. So I'm re-adding the code.

We know append additional information to the rbd-mirror bootstrap peer
token. It is useful for disaster recovery scenario where the other
cluster is reading the peer token and needs to know the pool_id as well
as the namespace.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-30 15:40:35 +02:00
Travis Nielsen 30063359da Merge pull request #8410 from leseb/simplify-cluster-creation-internal-flow
ceph: small bootstrap sequence refactor
2021-07-30 07:37:12 -06:00