earlier, we had the issue for osd placement regarding
whether we should merge the placements or not.
Now, we have option `skipApplyAllPlacement` to
enable/disable osd placement. By default it is false
which means it will merge both placement.All() and
storageClassDeviceSets.Placement, when true
it will only apply storageClassDeviceSets.Placement.
Also, adding unit test.
Closes: https://github.com/rook/rook/issues/8135
Signed-off-by: subhamkrai <srai@redhat.com>
Trying to disable mirroring on a cluster where mirroring is not enabled
results in an error during upgrades. Let's catch this error and ignore
it.
Funny enough the exec error resembles to:
```
Error ENOTSUP: Module 'mirroring' is not enabled (required by command 'fs snapshot mirror disable'): use `ceph mgr module enable mirroring` to enable it: exit status 95
```
So we get ENOTSUP which is 45 but exit status 95...
Closes: https://github.com/rook/rook/issues/8438
Signed-off-by: Sébastien Han <seb@redhat.com>
Before running the cleanup job we now also wait for the fs-mirror pods
to be gone.
Also, small refactor to print nicely the list of remaining daemons.
Signed-off-by: Sébastien Han <seb@redhat.com>
The `None` value Indicates that volumes will be mounted
with no modifications, as the CSI volume driver does not
support these operations. While volumes are provisioned
by the CephFS CSI driver the global permissions are set
on the volume and we dont expect the Fsgroup policy or
check from CO side to play a role here.
Ref #ceph/ceph-csi/../internal/cephfs/nodeserver.go#L190
```
!csicommon.MountOptionContains(fuseMountOptions, readOnly) {
// #nosec - allow anyone to write inside the stagingtarget path
err = os.Chmod(stagingTargetPath, 0o777)
```
The current default value ie `ReadWriteOnceWithFSType` cause
volumes to be examined to determine if volume ownership and permissions
should be modified to match the pod's security policy. Changes could occur
if the fsType is defined and the persistent volume's accessModes contains
ReadWriteOnce.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
We should not use %+v when printing errors and wrapping it in the
caller. Passing %+v to errors.Wrap() returns a strack trace.
See:
```
2021-08-03 09:42:40.460946 E | ceph-file-controller: failed to reconcile failed to create filesystem "myfs2": failed to start deployment for MDS "a" for filesystem "myfs2": failed to update mds deployment "rook-ceph-mds-myfs2-a": failed to check if deployment "rook-ceph-mds-myfs2-a" can continue: failed to check if we can continue the deployment rook-ceph-mds-myfs2-a: failed to check if rook-ceph-mds-myfs2-a was ok to continue: max retries exceeded, last err: mds myfs2-a is up:creating, bad state
github.com/rook/rook/pkg/daemon/ceph/client.MdsActiveOrStandbyReplay
/home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/status.go:286
github.com/rook/rook/pkg/daemon/ceph/client.okToContinueMDSDaemon.func1
/home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/upgrade.go:215
github.com/rook/rook/pkg/util.Retry
/home/leseb/go/src/github.com/rook/rook/pkg/util/retry.go:31
github.com/rook/rook/pkg/daemon/ceph/client.okToContinueMDSDaemon
/home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/upgrade.go:214
github.com/rook/rook/pkg/daemon/ceph/client.OkToContinue
/home/leseb/go/src/github.com/rook/rook/pkg/daemon/ceph/client/upgrade.go:181
github.com/rook/rook/pkg/operator/ceph/cluster/mon.UpdateCephDeploymentAndWait.func1
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/cluster/mon/spec.go:370
github.com/rook/rook/pkg/operator/k8sutil.UpdateDeploymentAndWait
/home/leseb/go/src/github.com/rook/rook/pkg/operator/k8sutil/deployment.go:113
github.com/rook/rook/pkg/operator/ceph/cluster/mon.UpdateCephDeploymentAndWait
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/cluster/mon/spec.go:383
github.com/rook/rook/pkg/operator/ceph/file/mds.(*Cluster).startDeployment
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/mds/mds.go:216
github.com/rook/rook/pkg/operator/ceph/file/mds.(*Cluster).Start
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/mds/mds.go:138
github.com/rook/rook/pkg/operator/ceph/file.createFilesystem
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/filesystem.go:81
github.com/rook/rook/pkg/operator/ceph/file.(*ReconcileCephFilesystem).reconcileCreateFilesystem
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/controller.go:373
github.com/rook/rook/pkg/operator/ceph/file.(*ReconcileCephFilesystem).reconcile
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/controller.go:288
github.com/rook/rook/pkg/operator/ceph/file.(*ReconcileCephFilesystem).Reconcile
/home/leseb/go/src/github.com/rook/rook/pkg/operator/ceph/file/controller.go:167
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/home/leseb/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:298
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/home/leseb/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:253
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/home/leseb/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.9.0/pkg/internal/controller/controller.go:214
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1371
```
Signed-off-by: Sébastien Han <seb@redhat.com>
They are scenarios where the mirroring information want to be shared
between clusters prior to creating pool. Because the bootstrap peer
import command needs a pool name to operate this is not suitable. So
additionally now each time the cluster is reconciled and on any new
clusters a new secret will be created that contains a boostrap peer
token. It can be exchanged with another cluster.
Signed-off-by: Sébastien Han <seb@redhat.com>
Recently, images moved from docker.io to quay.io so people using main
major tag like `v16` needs to switch to use the quay.io URL.
Signed-off-by: Sébastien Han <seb@redhat.com>
The section header was updated to Ceph Pacific since the tag used is
`v16.2.5-20210708`, which means Pacific and not Octopus.
Signed-off-by: Sébastien Han <seb@redhat.com>
This PR updates the required RBAC, templates,
CSI image version and examples for new
cephcsi v3.4.0 release.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Adds volumes and mounts for /var/lib/rook and /etc/ceph to the operator deployment in the Helm
chart. This is consistent with the example operator manifest and is sometimes even required on
clusters with stricter permissions on the local filesystem.
Signed-off-by: Dominik Münch <d.muench@celonis.com>
The CI test for mirroring now runs on stable v16 tag. Internally our
code has a minimum version of 16.2.5 for cephfs mirroring since it's
available.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the title looks familiar this is normal, this piece of code was
removed during https://github.com/rook/rook/pull/7604. Probably due to a
rebase. So I'm re-adding the code.
We know append additional information to the rbd-mirror bootstrap peer
token. It is useful for disaster recovery scenario where the other
cluster is reading the peer token and needs to know the pool_id as well
as the namespace.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit adds 'managedBy' label to the ceph metrics if the
cephcluster is managed by another higher Resource. The optional
label shall be added as part of cephcluster CRs monitoring labels.
Signed-off-by: Anmol Sachan <anmol13694@gmail.com>
Previous release upgrades included instructions for removing obsolete
drive groups and dealing with a preserveUnknownFields error. These
should no longer be an issue with the v1.7 upgrades.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The go-ceph library has removed the `Debug` field from the API type in
https://github.com/ceph/go-ceph/pull/543. Since the HTTP Client can be
mutated we now have our own client to dump requests and responses when
the operator log level is DEBUG.
Signed-off-by: Sébastien Han <seb@redhat.com>
Rook will now auto detect the kv version of the vault server. This
allows users not having to pass the VAULT_BACKEND configuration in the
CephCluster CR.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit adds an ingress resource to the rook-ceph-cluster helm chart, allowing
ingress to the ceph-dashboard service. It also adds the ability to define the
various storage types that you can run on ceph inside kubernetes.
Closes https://github.com/rook/rook/issues/8384
Signed-off-by: Tom Hellier <me@tomhellier.com>
References to luminous and mimic, and unnecessary references to nautilus
were found in the docs that have been cleaned up and clarified. At the same
time, the tests are updated to use more recent versions.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Travis Nielsen
Sébastien Han
rohan47
subhamkrai
Humble Chirammal
Madhu Rajanna
Dominik Münch
Blaine Gardner
Anmol Sachan
Carlos Eduardo Moreira dos Santos
Satoru Takeuchi
Blaine Gardner
Tom Hellier