If HostPID is enabled, this means the Ceph version is older than Octopus
and we are still running OSD with HostPID due to an internal
inter-process check.
If the log collector is enabled we need to enable ShareProcessNamespace
**only** if HostIPC is disabled.
Both HostPID and ShareProcessNamespace cannot be enabled.
Signed-off-by: Sébastien Han <seb@redhat.com>
If there is a single mon, we don't expect the PDBs to be created
and neither do we need to log that it is an error condition.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If HostIPC is enabled, this means the Ceph version is older than Octopus
and we are still running OSD with HostIPC due to an internal
inter-process check.
If the log collector is enabled we need to enable ShareProcessNamespace
**only** if HostIPC is disabled.
Signed-off-by: Sébastien Han <seb@redhat.com>
When creating object store, `radosgw-admin realm get ..` command is stuck forever when required number of OSDs are not available. Because of this the uninstall of object store is also stuck. User has to manually remove the finalizer to delete the object store.This PR uses `ExecuteCommandWithTimeout` for running `radosgw-admin` command. Timeout during installation will be reconciled. Cleanup will be treated as best effort. Any errors during uninstalling of single site object store will only be logged.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.
The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050
The same goes for integration tests, they will get added later once we
start testing on Pacific.
Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
when a node fails some pods remains in terminating while other manage to
be rescheduled on another node successfully.
this commit adds code to clean up all the Rook and CSI pods that are stuck in terminating
state on a failed node.
Signed-off-by: rohan47 <rohgupta@redhat.com>
The maxMonId should only track the mons that have been committed to starting.
Currently the maxMonId was tracking all mon scheduling. If the reconcile
was aborted or failed, or the operator restarted in a reconcile, the
maxMonId was being reset by analyzing the mon deployments that existed.
This does not work for scenarios where mon quroum is lost and being
reset to a single mon. Now the maxMonId will only be incremented
when a mon daemon is started, and will never be decremented after
that to ensure that new mons will be incremented to the next id.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the node is not found for mon assignment with host networking
enabled, the operator may crash. This change ensures the node is
not nil and returns an error instead of crashing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Previous patchset does not work before, because
`ConfigureLivenessProbe` does not directly modifies
input variable.
Signed-off-by: ushen <yshxxsjt715@gmail.com>
added waitTimeoutForHealthyOSD in the cluster crd that defines the time (in minutes) the operator would wait before an OSD can be stopped for
upgrade. This PR also removes the ok-to-continue logic for osds as its already handled by ok-to-stop. The default value is 10 minutes.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The settings for tuneDeviceClass and tuneFastDeviceClass were
stored as a map[string]string. When iterating over the map to
append the args, the args were being appended in an unpredictable
order. The order could change every reconcile, resulting in an
unnecessary OSD restart potentially every reconcile. Now we iterate
over a slice that will be ordered predictably and ensure OSDs
will not be restarted when not needed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
OSDs on PVCs previously were always replaced with PVCs of a given name.
For on-prem scenarios where the OSDs might need to be removed instead
of replaced, the operator now allows holes to exist in the PVC index
names as long as there are a sufficient number of PVCs to meet the
criteria for the deviceSet.count.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the CephCluster CR spec is updated to activate the logCollector, we
must reflect that change onto child CRDs, like the mds and rgw since
their configurationn would be impacted too.
Now we watch for the CephCluster object changes from the object/file
controllers and react upon the appropriate event.
Closes: https://github.com/rook/rook/issues/7022
Signed-off-by: Sébastien Han <seb@redhat.com>
in kubernetes 1.19 it added a support to set FSGroupPolicy
in the csidriver object, same supported is added to both
CephFS and RBD. where user will have an option to set
the FSGroupPolicy for CephFS and RBD separately.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The first patch to configure vault for ceph object store. If the `security.kms` configured in
`clusterSpec` CRD, RGW will be configured with vault kms settings to handle SSE request from s3 clients.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
In some cases the user dont want to run snapshotter
container either for CephFS or RBD. In that case the
user wont install the required snapshot CRD's due
to that the snapshotter sidecar container produces
lot of noisy logs.
Snapshotter will be enabled by default for both
CephFS and RBD, but with this PR we are providing
an option to disable snapshotter sidecar deployment
either for CephFS or RBD.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Multiple OSD pods for the same OSD might run simultaneously becasue OSD pod
is managed by Deployment resource. However, OSD locking mechanism
doens't work because the lock file (fsid file) exist for each OSD pod.
It resutls in OSD corruption.
Closes: https://github.com/rook/rook/issues/6530
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This was the last remaining CRD to not use the controller-runtime
library.
Small additions were added with the transition:
* the Kubernetes Secret that contains the CephX key has now an owner
reference to the CephClient object
* the secret name is present in the Status field of the CephClient:
```
status:
info:
secretName: rook-ceph-client-glance
phase: Ready
```
The controller will reconcile on CR updates and also if the Kubernetes
Secret is deleted.
Closes: https://github.com/rook/rook/issues/4938
Signed-off-by: Sébastien Han <seb@redhat.com>
By setting the working directory of a Ceph daemon to the log direcor
(which is bindmounted to the host), we can ensure that the coredumps
will be available.
On CentOS 7 **only**, the kernel is configured with:
```
cat /proc/sys/kernel/core_pattern
core
```
which means that the coredumps will end up in the process working
directory and will be named "core".
On CentOS 8, everything is different and this won't work but still
CentOS will be able to consume this and changing the working directory
is harmless.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the object store is not found during deletion of the object store CR,
proceed with the deletion instead of blocking and re-queueing the
deletion reconcile in an endless loop. This was causing instability
in the integration tests.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Sébastien Han
Travis Nielsen
Santosh Pillai
Blaine Gardner
rohan47
ushen
ron1
Jiffin Tony Thottan
Madhu Rajanna
Satoru Takeuchi