When OSDs flap, ceph stops the OSD daemon if its marked down greater than
5 times in 600 seconds. But OSD pod restarts and marks the OSD `up` again.
This causes the PGs mapped to these OSDs to peer. While the PGs are peering,
IO to these PGs are blocked.
So we need to ensure that if ceph is marking OSD `down` due to flapping, OSD pod
should not restart to mark the OSDs `up` again.
This PR adds a sleep to the OSD pod if the container returned with a 0 exit code
Default behavior is to sleep for 6 hrs. But user can configure it from the
ceph cluster spec.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 4eb9f62205)
While two mgrs is considered sufficient, more mgr daemons are
now allowed in case the admin wants even more fault tolerance
to the active mgr going down, in case multiple mgrs go down.
Up to five mgr pods will be allowed. All mgrs will be in
standby mode except one active mgr. All mgr pods have a sidecar
that will update their respective pod specs with the active
or passive label.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 66547d27be)
Previously ceph-exporter would only bind to IPv4 interfaces. Now if the
CephCluster is configured with `dualStack: true` and/or `ipFamily: IPv6`
an additional flag (`--addrs ::`) will be added to the ceph-exporter
container to make it listen both IPv6 and IPv4 interfaces.
Signed-off-by: Matthew Penner <me@matthewp.io>
(cherry picked from commit d09a43df37)
For an OBC's name we cannot simply use the OBC's namespace and name,
because they can be reused, while the preceding bucket might be
retained by reclaimPolicy. (Commit 2733375ca4 also didn't solve
this issue, as it is an intra-cluster issue.) Therefore we instead
use the OBC's UID which should be unique within the cluster and
across clusters.
Signed-off-by: Sebastian Hasler <sebastian.hasler@stuvus.uni-stuttgart.de>
(cherry picked from commit 041fc1a71c)
Change how Rook detects network CIDRs for Multus networks. The IPAM
configuration is only defined as an arbitrary string JSON blob with a
"type" field and nothing more. Rook's detection of CIDRs for whereabouts
had already grown out of date since the initial implementation.
Additionally, Rook did not support DHCP IPAM, which is a reasonable
choice for users. And more, Rook did not support CNI plugin chaining,
which further complicates NADs. Based on the CNI spec, network chaning
can result in any changes to network CIDRs from the first-given plugin.
All these problems make it more and more difficult for Rook to support
Multus by inspecting the NAD itself to predict network CIDRs. Instead,
it is better for Rook to treat the CNI process as a black box. To
preserve legacy functionality of auto-detecting networks and to make
that as robust as possible, change to a canary-style architecture like
that used for Ceph mons, from which Rook will detect the network CIDRs
if possible.
Also allow users to specify overrides for CIDR ranges. This allows Rook
to still support esoteric and unexpected NAD or network configurations
where a CIDR range is not detectable or where the range detected would
be incomplete. Because it may be impossible for Rook to understand the
network CIDRs wholistically while residing only on a portion of the
network, this feature should have been present from Multus's inception.
Improving CIDR auto-detection and allowing users to specify overrides
for auto-detected CIDRs rounds out Rook's Multus support for CephCluster
(core/RADOS) installations. No further architectural changes should be
needed for CephClusters as regards application of public/cluster network
CIDRs for Multus networks.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 3c43268d0a)
Correctly set discoveryInterval default value by passing defaultDiscoverInterval to k8sutil.GetValue
Allow to set ROOK_DISCOVER_DEVICES_INTERVAL env var through discoveryDaemonInterval helm value
Signed-off-by: Louis Bailleul <louis.bailleul@pgs.com>
(cherry picked from commit e56122cbce)
It's better to move most of discover daemon setting
from env to configmap rook-ceph-operator-config.
Although, we are moving to configmap, we keep reading settings
from env but the priority will be configmap settings.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit fb39580067)
This follow up the #12507
- fixes replacing of encrypted OSDs.
- Updates OSD status at the end of reconcile
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 11b8d10a5a)
ceph-exporter's ServiceMonitor and Service CRD contain different port
name, which results in no metrics being collected by prometheus.
this commit makes GetCephMonitor configurable, which we use to set
consistent port names in ServiceMonitor and Service for ceph-exporter.
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
(cherry picked from commit 708dd29a97)
We need to restart all the ceph daemons whenever
cephCluster network settings are modified like
requiremsgr2, encryption and compression. This
required for Ceph to consider the new settings
it require new ceph daemons all over.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 29d2b6a071)
when deploying csi, we'll check if csi-addons crds
are deployed we'll enable csi-addons sidecar by default.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit a755ec156a)
If bucket exists for Provision(), then check whether user in the OBC and
owner of bucket are same.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
(cherry picked from commit b39e813290)
The json tag in struct MonMap had a spelling error where "json" was incorrectly splled as "josn". This commit corrects the spelling to "json".
Signed-off-by: go-bai <599500688@qq.com>
(cherry picked from commit f4cfa95768)
we need to return a valid CIDR IP
to get it fenced properly, the previously
returned one was like `10.63.0.5:0/32` which
is not a valid CIDR, removing `:0` to look like
proper IP CIDR.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit eb304a5073)
The object user was previously required to be created in the
same namespace as the object store and the cluster. Now,
the object user can be reconciled even in a different namespace
from the cluster and object store. The namespace would be specified
in the object user CR.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 2b1cc4c7b8)
The go package dependencies (go.mod and go.sum) were created
under the pkg/apis directory to simplify the dependencies for
other projects referencing the rook repo. The downside is that
the dependabot can no longer open a working and valid PR since
the bot is not aware of the go.mod and go.sum in the apis directory.
Since the reduction of dependencies for vault in #12455,
the list of extra dependencies is not quite so extensive.
We are putting effort into shrinking the dependency list instead
of using the modules files in the apis subdirectory.
Note that #12419 updated the dependencies for the latest controller
runtime which also increased the size of the modules
list in the pkgs subdirectory, which means the difference
in dependencies in the pkgs subdirectory is no longer
signficant anyway.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 1a4917de78)
# Conflicts:
# .github/workflows/update_dependabot_bundler_pr.yml
Setting ceph cluster as the ControllerRef
for the holder daemonset set so that when
a ceph cluster is deleted the holder daemoset
will also gets deleted.
fixes: #12645
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d7a4f0038c)
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 886bb357ef)
* fixes mon zonal deployment of 2 per zone to 1 if zones field is
specified.
* adds zones field doc to CRDs Documentation.
* add code for picking up failureDomainLabel if specified for zones
field in mon deployment
Signed-off-by: Deepika Upadhyay <deepika@koor.tech>
(cherry picked from commit b67bf2d6f0)
golangci linter was throughing error `k8s.io/utils/pointer`
package is deprecated. So, I have removed that.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit a25071ac29)
update vault and other indirect imports. Also,
updating k8s package to `v0.28.0-rc.0` since with
v0.27.4 it was through build error.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit ab3763767f)
earlier, we're always logging when `GetValue` method is
called to get operator config values which somettime
spamming the logs for example `ROOK_WATCH_FOR_NODE_FAILURE`
keep logging every time 5 min.
Now, we'll store the settings in the map and log only when
map key value is changed.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 733be02577)
Ignore the exit code 22 when adding the
datapool to a filesystem.
updates: #12673
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit bd9b42d221)
When the dbus-daemon in the sidecar is started as "root" user, it fails
with the following log entry:
Failed to start message bus: Failed to drop capabilities: Operation not permitted
By starting the sidecar as "dbus" user (uid=81), the executable does not
try to drop capabilities, and starts successfully.
Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit 320b112cc6)
Adding maxUnavailable parameter to the cephfs csi daemonset. The
default value of maxUnavailable = 1 can make the upgrade process
slower in environments with large number of nodes.
Fixes: #12636
Signed-off-by: karthik-us <ksubrahm@redhat.com>
(cherry picked from commit 2675c976d5)
Add nfs pods label to the list so that Rook
can get all the pods on the failed node and
force delete the pods stuck in terminating state.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 29faca9032)
Add holder pods label to the list so that Rook
can get all the pods on the failed node and
force delete the pods stuck in terminating state.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 22bfc9de3d)
Some logging instructions use formatting syntax `%q`, but the logging method is
`Info` or `Debug` instead of `Debugf` and `Infof`. As a result, the arguments of
the method were concatenated instead of formatted properly.
Signed-off-by: Lucas Henry <polyedre@disroot.org>
(cherry picked from commit c948183c0e)
The function ParseMonEndpoints existed twice, once in the mon package
and once in the controller package. It has now been deleted from the
mon package. Usages in the mon package now reference the function
in the controller package.
Signed-off-by: Henry Davies <henrydavies@hotmail.co.uk>
(cherry picked from commit 2c8949e195)
Currently if we use the msgr2 port,
cs-cluster-config-json looks like
"monitors":["192.59.109.11:3300","192.59.109.13:6789","192.59.109.12:6789"]
The first monitor is passed by python script and rest are
Discovered by rook operator, so if the first mon is using the v2 port
lets update the other mons also by setting up the RequireMsgr2 flag
Signed-off-by: parth-gr <paarora@redhat.com>
(cherry picked from commit 2c4897f88c)
0a151e913 results in a bug which causes null pointer dereference when
Kerberos is not setup. This causes a failure in Unit test
TestReconcileCephNFS_addSecurityConfigsToPod
Signed-off-by: Sachin Prabhu <sprabhu@redhat.com>
(cherry picked from commit 6cf4627ccd)
This will enable us to use GSSAPI in sssd for authentication when
connecting to the remote LDAP server.
Signed-off-by: Sachin Prabhu <sprabhu@redhat.com>
(cherry picked from commit 0a151e913d)
when node fencing is triggered and a negative case where
one deployment pod consuming pv not provisioned by csi
and another deployment pod with rbd rwo are on the same
node and node fencing is triggered.
In this case rook operator panics since it is trigger to get
field `pv.Spec.CSI.Driver` which will be nil.
Also, skip static pv as there is no `imageName` in the spec
rbd status command will fail.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit 72386eb1d0)
Satoru Takeuchi
sp98
travisn
Matthew Penner
Sebastian Hasler
Blaine Gardner
Louis Bailleul
subhamkrai
Divyansh Kamboj
Jiffin Tony Thottan
go-bai
Madhu Rajanna
Deepika Upadhyay
Niels de Vos
karthik-us
Lucas Henry
Henry Davies
parth-gr
Sheetal Pamecha
Sachin Prabhu