With 5f57b6fd3a, we introduced a
regression in the validation code.
If the OSD PV comes from another cluster, GetCephVolumeRawOSDs must
return an error.
The prepare job will crash loop but at least with a useful and
comprehensive message.
Signed-off-by: Sébastien Han <seb@redhat.com>
If Ceph pods were evicted onto other nodes, there is no reason for the
crash collector to remain on those nodes. So let's remove it.
Signed-off-by: Sébastien Han <seb@redhat.com>
nfs server requires valid pool for storing nfs-ganesha recovery objects. Only
when filesystem exports are created, filesystem should be configured first.
Signed-off-by: Varsha Rao <varao@redhat.com>
The pod IP address is no longer needed for port binding for the dashboard
and prometheus endpoints anymore in the mgr modules. The default binding
to localhost will be sufficient. This will allow the endpoints to be more
reliable in some environments where today the podIP is not working.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Binding the dashboard and prometheus to all interfaces instead of the pod ip
will ensure that they are listening properly. Binding to the pod ip was only
a short term solution that was intended to be removed with Nautilus, which
is already Rook's min supported version.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
completeOSDsForAllNodes() method waits for 20 minutes for all the OSDs to start on all the nodes. This PR checks if
cluster deletion request was sent while waiting for OSDs to start. If yes, then completeOSDsForAllNodes() method will returnwithout waiting for 20 minutes.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
This PR makes create and delete events on pdb a no-op. Update event only triggers reconcile if its the main OSD pdb and allowed disruptions is 0.
This prevents controller to reconcile on pdb events namespaces outside rook ceph.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
If the ceph cli outputs an error with "error calling conf_read_file"
this means that the operator has not written its ceph configuration
file. Thus ceph cli commands will fail, so we can just ignore that since
the operator will soon write this file in its initialization sequence.
Signed-off-by: Sébastien Han <seb@redhat.com>
The log-collector container name was exceeding the 63-char limit
if the parent CR name was too long. The container name just needs
to be unique to the pod spec, so we simplify it to remove the parent
name and avoid hitting the limit in the pod spec.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Before this patch, csi detection will run whether or
not rbd and cephfs are all disabled. Run detection only
if it is necessary.
Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
If HostPID is enabled, this means the Ceph version is older than Octopus
and we are still running OSD with HostPID due to an internal
inter-process check.
If the log collector is enabled we need to enable ShareProcessNamespace
**only** if HostIPC is disabled.
Both HostPID and ShareProcessNamespace cannot be enabled.
Signed-off-by: Sébastien Han <seb@redhat.com>
If there is a single mon, we don't expect the PDBs to be created
and neither do we need to log that it is an error condition.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If HostIPC is enabled, this means the Ceph version is older than Octopus
and we are still running OSD with HostIPC due to an internal
inter-process check.
If the log collector is enabled we need to enable ShareProcessNamespace
**only** if HostIPC is disabled.
Signed-off-by: Sébastien Han <seb@redhat.com>
When creating object store, `radosgw-admin realm get ..` command is stuck forever when required number of OSDs are not available. Because of this the uninstall of object store is also stuck. User has to manually remove the finalizer to delete the object store.This PR uses `ExecuteCommandWithTimeout` for running `radosgw-admin` command. Timeout during installation will be reconciled. Cleanup will be treated as best effort. Any errors during uninstalling of single site object store will only be logged.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.
The configuration of peers will come in a later PR since the mgr module
is still pending upstream: https://github.com/ceph/ceph/pull/39050
The same goes for integration tests, they will get added later once we
start testing on Pacific.
Closes: https://github.com/rook/rook/issues/7002
Signed-off-by: Sébastien Han <seb@redhat.com>
when a node fails some pods remains in terminating while other manage to
be rescheduled on another node successfully.
this commit adds code to clean up all the Rook and CSI pods that are stuck in terminating
state on a failed node.
Signed-off-by: rohan47 <rohgupta@redhat.com>
The maxMonId should only track the mons that have been committed to starting.
Currently the maxMonId was tracking all mon scheduling. If the reconcile
was aborted or failed, or the operator restarted in a reconcile, the
maxMonId was being reset by analyzing the mon deployments that existed.
This does not work for scenarios where mon quroum is lost and being
reset to a single mon. Now the maxMonId will only be incremented
when a mon daemon is started, and will never be decremented after
that to ensure that new mons will be incremented to the next id.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the node is not found for mon assignment with host networking
enabled, the operator may crash. This change ensures the node is
not nil and returns an error instead of crashing.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Previous patchset does not work before, because
`ConfigureLivenessProbe` does not directly modifies
input variable.
Signed-off-by: ushen <yshxxsjt715@gmail.com>
added waitTimeoutForHealthyOSD in the cluster crd that defines the time (in minutes) the operator would wait before an OSD can be stopped for
upgrade. This PR also removes the ok-to-continue logic for osds as its already handled by ok-to-stop. The default value is 10 minutes.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The settings for tuneDeviceClass and tuneFastDeviceClass were
stored as a map[string]string. When iterating over the map to
append the args, the args were being appended in an unpredictable
order. The order could change every reconcile, resulting in an
unnecessary OSD restart potentially every reconcile. Now we iterate
over a slice that will be ordered predictably and ensure OSDs
will not be restarted when not needed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
OSDs on PVCs previously were always replaced with PVCs of a given name.
For on-prem scenarios where the OSDs might need to be removed instead
of replaced, the operator now allows holes to exist in the PVC index
names as long as there are a sufficient number of PVCs to meet the
criteria for the deviceSet.count.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the CephCluster CR spec is updated to activate the logCollector, we
must reflect that change onto child CRDs, like the mds and rgw since
their configurationn would be impacted too.
Now we watch for the CephCluster object changes from the object/file
controllers and react upon the appropriate event.
Closes: https://github.com/rook/rook/issues/7022
Signed-off-by: Sébastien Han <seb@redhat.com>
in kubernetes 1.19 it added a support to set FSGroupPolicy
in the csidriver object, same supported is added to both
CephFS and RBD. where user will have an option to set
the FSGroupPolicy for CephFS and RBD separately.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The first patch to configure vault for ceph object store. If the `security.kms` configured in
`clusterSpec` CRD, RGW will be configured with vault kms settings to handle SSE request from s3 clients.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
In some cases the user dont want to run snapshotter
container either for CephFS or RBD. In that case the
user wont install the required snapshot CRD's due
to that the snapshotter sidecar container produces
lot of noisy logs.
Snapshotter will be enabled by default for both
CephFS and RBD, but with this PR we are providing
an option to disable snapshotter sidecar deployment
either for CephFS or RBD.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Multiple OSD pods for the same OSD might run simultaneously becasue OSD pod
is managed by Deployment resource. However, OSD locking mechanism
doens't work because the lock file (fsid file) exist for each OSD pod.
It resutls in OSD corruption.
Closes: https://github.com/rook/rook/issues/6530
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Sébastien Han
Varsha Rao
Travis Nielsen
Santosh Pillai
shenjiatong
Blaine Gardner
rohan47
ron1
Jiffin Tony Thottan
Madhu Rajanna
Satoru Takeuchi