To make it easier for the user to switch to a namespace other than the
default 'rook-ceph', the tag '# namespace:cluster' must be inserted in
all the places where this namespace is used. This way, a simple 'sed'
command can update all the Yaml files to the new namespace.
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit ff081cc4ae)
Sometimes the pool is not enabled and the ceph cluster
shows health warning, so automatically initalize the pool
from the script
Signed-off-by: parth-gr <paarora@redhat.com>
(cherry picked from commit 40008bf18d)
If `FlappingRestartIntervalHours` for OSD is > 0 in the cephCluster CR,
then start OSD inside a bash script.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit ffd69291f1)
if the client.health checker already exited it was
returning extra information, It was making the structure and
return JSON non idempotent output, So fixed
that problem by returning a single value
Signed-off-by: parth-gr <paarora@redhat.com>
(cherry picked from commit bf5252e139)
Add the ability to specify node profiles in the multus validation test.
This addresses a few points of early feedback on the validation tool.
Statements below critique the tool's behavior before this patch.
1. The tool assumes all daemons are on public and cluster network, which
means users who have a significantly smaller cluster net (a
design choice) cannot run a single test to determine if Rook is
likely to install correctly.
2. The tool does not have placement options to select only a subset of
Kubernetes nodes to run validation on.
3. Users of multus seem to have a dedicated pool of storage nodes more
often than the average Rook install. This makes sense for security-
and perforance-minded users. The tool cannot run a single test to
verify storage-only and general-workload nodes at one time.
These points are addressed by allowing users to specify configurations
for different "NodeTypes."
Each NodeType config has options for selecting the number of OSDs as
well as the number of other (non-OSD) Ceph daemons. This limits the
unnecessary exhaustion of cluster network addresses from critique 1.
Each NodeType config has its own placement (critique 2).
Users can define as many NodeTypes as needed to test the network for
their planned CephCluster. Specifically, this allows the tool to test
storage-only nodes and generalized-workload nodes at the same time. An
arbitrary number of NodeTypes are allowed to support even more highly
specialized cluster setups, such as multiple tiers of storage nodes
where some storage-only nodes may run more OSDs than others.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 0c721e05d5)
Create each cosi user for each object store and secret which holds
credentials.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
(cherry picked from commit a941b3c33f)
when creating networkFence CR, it requires IP's to block
which we get from running `rbd status ...` command. But,
the client.healthchecker user didn't had the right caps
to run hence it was giving error. Now, adding the required
caps `profile rbd-read-only` to osd so that rbd command can
be executed.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit c78bd1cbc6)
Using the new configuration parameters prometheusEndpoint and
prometheusEndpointSSLVerify users can configure dashboard to
point to their prometheus instance.
closes: #12876
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
(cherry picked from commit c763935436)
add view & edit ClusterRole for obc.
the roles are added to the default user facing roles
view / edit / admin respectively using k8s ClusterRole aggregation.
feature disabled for backward compatibility
and controlled with rbacAggregate.enableOBCs.
Signed-off-by: Cyril Jouve <jv.cyril@gmail.com>
(cherry picked from commit 45d897b8b3)
When OSDs flap, ceph stops the OSD daemon if its marked down greater than
5 times in 600 seconds. But OSD pod restarts and marks the OSD `up` again.
This causes the PGs mapped to these OSDs to peer. While the PGs are peering,
IO to these PGs are blocked.
So we need to ensure that if ceph is marking OSD `down` due to flapping, OSD pod
should not restart to mark the OSDs `up` again.
This PR adds a sleep to the OSD pod if the container returned with a 0 exit code
Default behavior is to sleep for 6 hrs. But user can configure it from the
ceph cluster spec.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 4eb9f62205)
While two mgrs is considered sufficient, more mgr daemons are
now allowed in case the admin wants even more fault tolerance
to the active mgr going down, in case multiple mgrs go down.
Up to five mgr pods will be allowed. All mgrs will be in
standby mode except one active mgr. All mgr pods have a sidecar
that will update their respective pod specs with the active
or passive label.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 66547d27be)
some helm comments incorrectly indicate ceph v18 is not supported
Signed-off-by: Micah Nagel <micah.nagel@defenseunicorns.com>
(cherry picked from commit 0ff224d453)
Change how Rook detects network CIDRs for Multus networks. The IPAM
configuration is only defined as an arbitrary string JSON blob with a
"type" field and nothing more. Rook's detection of CIDRs for whereabouts
had already grown out of date since the initial implementation.
Additionally, Rook did not support DHCP IPAM, which is a reasonable
choice for users. And more, Rook did not support CNI plugin chaining,
which further complicates NADs. Based on the CNI spec, network chaning
can result in any changes to network CIDRs from the first-given plugin.
All these problems make it more and more difficult for Rook to support
Multus by inspecting the NAD itself to predict network CIDRs. Instead,
it is better for Rook to treat the CNI process as a black box. To
preserve legacy functionality of auto-detecting networks and to make
that as robust as possible, change to a canary-style architecture like
that used for Ceph mons, from which Rook will detect the network CIDRs
if possible.
Also allow users to specify overrides for CIDR ranges. This allows Rook
to still support esoteric and unexpected NAD or network configurations
where a CIDR range is not detectable or where the range detected would
be incomplete. Because it may be impossible for Rook to understand the
network CIDRs wholistically while residing only on a portion of the
network, this feature should have been present from Multus's inception.
Improving CIDR auto-detection and allowing users to specify overrides
for auto-detected CIDRs rounds out Rook's Multus support for CephCluster
(core/RADOS) installations. No further architectural changes should be
needed for CephClusters as regards application of public/cluster network
CIDRs for Multus networks.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 3c43268d0a)
Correctly set discoveryInterval default value by passing defaultDiscoverInterval to k8sutil.GetValue
Allow to set ROOK_DISCOVER_DEVICES_INTERVAL env var through discoveryDaemonInterval helm value
Signed-off-by: Louis Bailleul <louis.bailleul@pgs.com>
(cherry picked from commit e56122cbce)
It's better to move most of discover daemon setting
from env to configmap rook-ceph-operator-config.
Although, we are moving to configmap, we keep reading settings
from env but the priority will be configmap settings.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit fb39580067)
This allows the rook mgr module to check if the discovery daemon is enabled. The udev blacklist setting was also moved just to keep similar settings in the same place.
Signed-off-by: Stephen Muth <smuth4@gmail.com>
(cherry picked from commit 37d8de0993)
The toolbox based on the operator image was added after the initial
release of v1.12.0, where the image tags from master were replaced
by hte release version. When the feature was backported,
the master tag remained. Now the version tag is set
as expected.
Signed-off-by: travisn <tnielsen@redhat.com>
it was confusing if we are calling a ceph cluster name
or k8s cluster name, so re-named
Signed-off-by: parth-gr <paarora@redhat.com>
(cherry picked from commit 98c4567219)
The object user was previously required to be created in the
same namespace as the object store and the cluster. Now,
the object user can be reconciled even in a different namespace
from the cluster and object store. The namespace would be specified
in the object user CR.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 2b1cc4c7b8)
Signed-off-by: Michael Adam <obnox@samba.org>
this change introduces a new 'make target 'docs'that generates part of the documentation
It particular, it runs the `helm-docs` target.
Additionally, `make docs` updates the multus validation job manifest
by inserting the output of `rook multus validation run --help` between
two newly introduced markers.
The result of running `make docs` is intended to be committed to git after running
Fixes: #12706
Signed-off-by: Michael Adam <obnox@samba.org>
Signed-off-by: Blaine Gardner <b.blaine.gardner@gmail.com>
Co-authored-by: Blaine Gardner <b.blaine.gardner@gmail.com>
Co-authored-by: Michael Adam <obnox@samba.org>
Update build/deploy/generate-deploy-examples.sh
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit 22882c5358)
updating this to ease the local testing
using helm for master branch repo
Signed-off-by: parth-gr <paarora@redhat.com>
(cherry picked from commit 67f294b623)
Noticed that since the release of 1.9, the values.yaml file has some
old documentation references. Replaced them with the new locations.
Adjusted the link as requested by travisn
Placed extra line eof back as requested by path-gr
Signed-off-by: Rowan Ruseler <rowanruseler@gmail.com>
(cherry picked from commit 8f8f46dbd6)
The CRDs were not generated for the latest since #12384
was merged without rebasing, and other updates with a newer
dependency required the newer descriptions.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 438b61721f)
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 886bb357ef)
As part of a normal restart, an OSD can go down just as prometheus tries
to scrape it, causing this alert to fire. Alerts shouldn't fire as part
of normal operations. This change requires the OSD to be down for 5
consecutive minutes before the alert will fire.
Signed-off-by: Chris Jones <chris@cjones.org>
(cherry picked from commit 9f1e5ec4ee)
update vault and other indirect imports. Also,
updating k8s package to `v0.28.0-rc.0` since with
v0.27.4 it was through build error.
Signed-off-by: subhamkrai <srai@redhat.com>
(cherry picked from commit ab3763767f)
added a sample job manifest named `multus-validation` that
validates the multus configuration in the cluster.
Signed-off-by: Nikhil-Ladha <nikhilladha1999@gmail.com>
(cherry picked from commit 9fea9cd865)
Redouane Kachach
parth-gr
travisn
sp98
Blaine Gardner
Jiffin Tony Thottan
subhamkrai
Cyril Jouve
mergify[bot]
Micah Nagel
Louis Bailleul
Stephen Muth
Michael Adam
Rowan Ruseler
Deepika Upadhyay
Chris Jones
Nikhil-Ladha