The exporter requires the mon config and keyring.
Use the CEPH_ARGS env var to specify the config
instead of expecting a ceph.conf to be generated
by another component on the host path.
Co-Authored-By: Travis Nielsen <tnielsen@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
if a cluster won't use the deafult realm and try to
execute the python script it will fail to vaildate rgw
as the rgw-admin-user would be created on default realm
So creating user in specific realm if the realm name is passed
Signed-off-by: parth-gr <paarora@redhat.com>
ceph crash pod gets rook version label added unintentionally, which is only for deployment label.
changes include:
1, make a copy of the labels for pod spec from deployment labels just before adding rook version label.
2, add unit test to make sure it works as designed.
This fix along with pr 11674 is to stop ceph daemons pod restarting unnecessarily after upgrading rook operator.
Signed-off-by: Ben Gao <bengao168@msn.com>
The osd needs to bind to an ip that will allow both internal
and external access for rbd mirroring scenarios.
THe operator will create a service for each osd and
instruction the osd to bind to the pod ip while
publishing the clusterIP of the service as the
public IP.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 14e7225e0a18aff85ede1d476fcaddbbfbcbb732)
If a kernel device name change happens and a block device file
in the OSD directory becomes dangling link, this OSD fails
to start continuously. This problem can be resolved by confirming
the validity of the device file and recreating it if necessary.
The original problem happened in host-based clusters. A similar
problem exists in PVC-based clusters. In this case, if a PV,
corresponds to an existing OSD, and points to a missing block
device file, the OSD pod fails to consume this PV. Although this
behavior is undesirable, I don't think this problem should be
handled in Rook. If doing so, we must re-create the existing PV.
Rook shouldn't do such work.
Closes: https://github.com/rook/rook/issues/10860
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
With the new mgr HA implementation (based on readiness probe) we
don't need the mgr sidecar (live-watch) anymore. This commit is
intended to remove all the related code.
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Removing user caps will be skipped as the UserCaps is empty, which will cause updating user caps to not take effect.
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
After an update, the mgr pod will be restarted. Since the
readiness probe will indicate the standby mgr pod is never
ready, we need to check simply for the running mgr pod
and not whether or not it is ready.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
few functions got change as they were deprecated
for ex: ioutil.Readfile change to os.Readfile
ioutil.TempFile change to os.CreateTemp
And fixed golang-ci-lint-issues
Signed-off-by: parth-gr <paarora@redhat.com>
This fixes the the MultiClusterDeploySuite CI failure. The operator
is timing out waiting for the mgr deployments to be
ready, according to the WaitForDeploymentToStart() method. After
the reconcile times out after about five minutes, the next
reconcile succeeds since the wait is only done for new mgr
deployments.
Closes: https://github.com/rook/rook/issues/11685
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
This commit adds code to supply variables required
to enable read affinity for rbd volumes.
It leverages `read_from_replica=localize,crush_locations=key:value`
krbd map option from ceph to serve reads from the most local OSD.
refer:
https://docs.ceph.com/en/latest/man/8/rbd/#kernel-rbd-krbd-options
Signed-off-by: Rakshith R <rar@redhat.com>
Rook's implementation of OBC's GenerateUserID could generate colliding
user IDs for multisite clusters where each site is configured with the
same namespace/name of CephObjectStore. Add the Ceph cluster FSID to the
generated name to ensure the names won't conflict for multiple sites.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Set the rbd map default option to ms_mode=prefer_crc
when v2 ports are used and encryption is disabled.
If we dont pass/set the ms_mode in the map options
the kernel client will assume its using v1 ports
and map operation will fail.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The idea behind this change is to use the readiness probe to implement
the mgr HA mechanism. In the current ceph mgr implementation only the
active instance offers the command 'mgr_status' through the admin
socket. We use this command combined with a Readiness Exec Probe to
detect which manager is active. Kubernetes will automatically mark
it as ready and redirect any service traffic to the active instance.
Closes: https://github.com/rook/rook/issues/11640
Closes: https://github.com/rook/rook/issues/11638
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
The removal of the rgw_rados_pool_pg_num_min property
was completed in v16.2.11 for Pacific. The previous
fix in PR 11024 only assumed the fix was in Quincy v17.2.2
or newer. Now we account for v16.2.11 as well.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The rook-version tag must not be set on the pod spec labels
since it will result in the ceph daemons being restarted every
time there is a rook version update, even if the ceph version
or pod spec was not otherwise updated.
The rook-version tag was being added to the OSD pod spec
due to a shared pointer to the labels. The code intended
to only add the version label to the deployment labels,
but since the pod labels shared the map variable, the
pod unintentionally also had the version label added.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
increasing the rotation from default 7 to 28 as
in case of rbdmirror logs seems not enough in some cases
with maxLogSize 500 so it's better to increase the rotation
for rbdmirror specific.
Signed-off-by: subhamkrai <srai@redhat.com>
Mons that are configured on msgr1 will be using the v1 port
in the csi driver since only one port is allowed in the configmap.
To allow mons to be started on v2 while keeping quorum with the
v1 mons, the v1 mons must continue running on the v1 port.
So even if msgr2 is required, allow existing v1 mons to continue
running on that port.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The default configmap does not have any labels or annotations
applied. If the helm chart is later updated to configure the
ceph config overrides, the configmap will fail to be created
by the chart since it doesn't have the needed label and helm
annotations so helm will know it can take over the resource
management.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
avanthakkarandTravis Nielsen
Redouane Kachach
Blaine Gardner
parth-gr
sp98
Ben Gao
Satoru Takeuchi
Madhu Rajanna
Liang Zheng
zhucan
Rakshith R
Avan Thakkar
subhamkrai