In order to help users check that they have implemented the newly-added
Multus host configuration prerequisites, add a check to the validation
tool to verify connectivity.
Because users who are already running clusters with Multus enabled, add
a flag that allows users to only check for host configuration
prerequisites. This mode will not start the large number of clients that
would normally be started because those clients could disrupt a running
Rook cluster negatively.
Host checking pods require host network access. Many Kubernetes
distributions have pod security features enabled. In order to allow
non-Vanilla distros to run this tool, allow specifying a service account
that pods will run as, which can be configured by the admin to allow
test pods.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 33f5407dd4)
# Conflicts:
# Documentation/CRDs/Cluster/network-providers.md
# deploy/examples/multus-validation.yaml
For an advanced network configuration, the mon endpoints
expected for the ceph public network may be different from
the public IP address that is set on the K8s node. When
host networking is enabled in rook, the IP address of the
node was only allowed previously. Now, the IP address
used for a mon when scheduled on a node can be determined
by setting an annotation on each node where mons can run:
network.rook.io/mon-ip: <ip-address>
This annotation must be set before the mons are created.
Mon endpoints cannot be changed after they are already
running.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit e90ed3b912)
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit e479b051bc)
mkdocs uses a markdown renderer that is hardcoded to 4 spaces per tab
for detecting indentation levels, including ordered- and
unordered-lists. Since we cannot easily change the renderer, begin using
a markdown linter in CI that will fail if official docs do not adhere to
the spacing rules.
As a starting point, the markdownlint config does not begin with the
default set of checks, which might overwhelm attempts to fix them.
Instead, focus on list-tab-spacing rules and a few other highly useful
checks.
markdownlint also has some gaps in its abilities that allow common Rook
doc issues to pass acceptance. However, it allows creating custom
linting plugins. Create 2 such linting plugins to check 2 things:
- all doc lines (except code blocks) must be aligned to a 4-space
boundary, without exception. This ensures that markdown will render
correctly with mkdocs. This unfortunately makes it possible to create
lists that are internally aligned strangely.
- admonitions must all follow the same format of
```
!!! header
body
```
For the strange lists, this is allowed and renders correctly, but it
looks strange:
```md
- first bullet
- second bullet
still second bullet
- third bullet
has a paragraph
of text inside
- last bullet
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit c9d99e01a0)
This PR will add separate directory to handle the external manifest. Here external manifests will have same namespace (rook-ceph) for external cluster.
Signed-off-by: smoshiur1237 <moshiur.rahman@est.tech>
(cherry picked from commit dab17f72a1)
Unlike other components, the Ceph dashboard Service object could not be
configured with custom labels and annotations. This can be useful when
integrating Rook into a cluster with custom controllers similarly to how
the RGW service is customized.
Signed-off-by: Steven Fackler <sfackler@gmail.com>
(cherry picked from commit 0c38dd81c2)
For check if cluster PGs are healthy before osd updates
Which helps gainning more confidence in the osd upgrades
Signed-off-by: mmaoyu <wupiaoyu61@gmail.com>
(cherry picked from commit 6d4afe23e5)
Most bullets and numbering in the network providers doc (sub-doc of
CephCluster CR) wasn't rendering properly on the web. Fix by making sure
there is a blank line before each numbered/bulleted section. And make
sure nested numberings are indented 4 spaces.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit ad9e0976e2)
The network providers doc, including the holder pod migration sections,
were developed with the understanding that CSI could be placed into pod
networking mode if desired by users. In reality, this hasn't been
supported due to bugs.
This allows streamlining the holder pod migration sections to no longer
need to document for users what to do if they use CSI in pod network
mode.
Rook will continue to have the `CSI_ENABLE_HOST_NETWORK` config
available as an undocumented value. This can allow users who desperately
cannot use host networking to still use pod networking, though this will
mean any CSI node plugin updates could stall mounts for running apps.
Thus we do not support it.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 519c7e8911)
csi pr get merged ceph/ceph-csi#4459
so, This reverts commit 812a9c02b0.
The csi PR removes the dependency of the pool parameter from the
rbd storage class if the topology pools are passed,
so removing them in the examples and making it optional
Signed-off-by: parth-gr <partharora1010@gmail.com>
Add new CSI_DISABLE_HOLDER_PODS option for rook-ceph-operator.
This option will disable holder pods when set to "true".
In the long term, Rook plans to deprecate the holder pods entirely.
This new option will allow users to choose to migrate their clusters to
non-holder clusters when they are ready and able, giving them time to
gracefully migrate before the holders are permanently removed.
This option is set to "false" by default so that upgrading users don't
have their CSI pods modified unexpectedly.
Example manifests are modified to set this value to true so that new
clusters will not deploy holder pods.
Migrating users are provided with documentation to instruct them about
the new requirements they need to satisfy to successfully remove holder
pods, a procedure for migrating pods from holder to non-holder mounts,
and a way to delete holder pods once they are no longer in use.
When users set CSI_DISABLE_HOLDER_PODS="true", the CSI controller will
no longer deploy or update the holder pod Daemonsets, but it does not
delete any existing Daemonsets. This allows already-attached PVCs to
continue operating normally with their network connection continuing to
exist in the current holder pod. This is critical to avoid causing
ia cluster-wide storage outage.
More info: https://github.com/rook/rook/issues/13055
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
With the release of Reef v18.2.2 we update the default
recommended version of Ceph to fix the prometheus issues
seen with v18.2.1
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
with topologyconstrain pool we can enable any
replica pool usage and also can store data at any
topology
Signed-off-by: parth-gr <partharora1010@gmail.com>
We have prohibited changing all network-related configurations after cluster creation.
However, as a result of https://github.com/rook/rook/discussions/13584, we found that
the connections.encryption flag can be regarded as an exception.
Signed-off-by: ESASHIKA Kaoru <kaoru-esashika@cybozu.co.jp>
Some users are still confused by Ceph networking, especially involving
Multus. Continuing to add network provider-specific information -- a
pretty advanced concept -- to the already-heavy CephCluster CRD doc is
becoming cumbersome. Separate network provider settings into a separate
child doc that notes it is an advanced topic.
This separation will also help with work to remove the holder pod
design, which requires additional Multus prerequisites.
See: https://github.com/rook/rook/issues/13055
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
When CPU requests and limits are assigned to a pod,
the pod will be guaranteed the requests, up to the limits.
Even if there are spare CPU cycles, the pod cannot use
them. Thus, pods can be unnecessarily denied compute
when they need to burst if the limits are set.
Therefore, it is not recommended to set CPU limits
since the CPU requests are already guaranteeing that
no pod will be starved at least for its requests.
Signed-off-by: travisn <tnielsen@redhat.com>
Currently, when rook provisions OSDs(in the OSD prepare job), rook effectively run a
c-v command such as the following.
```console
ceph-volume lvm batch --prepare <deviceA> <deviceB> <deviceC> --db-devices <metadataDevice>
```
but c-v lvm batch only supports disk and lvm, instead of disk partitions.
We can resort to `ceph-volume lvm prepare` to implement it.
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
If the import script is run on a system that has many kubernetes configs
in the kubeconfig file, it is useful to be able to specify the context
to use.
Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
The pg_autoscaler mgr module is no longer possible to disable.
Ceph requires the module to always be enabled. Now the
autoscaling configuration is done with a per-pool setting.
Now the rook module is shown as an example to enable.
Signed-off-by: travisn <tnielsen@redhat.com>
Since v18.2.1 is the default recommendation for the ceph version,
update all the docs and examples to that version.
Signed-off-by: travisn <tnielsen@redhat.com>
Add the missing explanation of pgHealthCheckTimeout and pgHealthyRegex
in the CephCluster' CRD doc. In addition, remove a verbose sentence
from the explanation of `osdMaintenanceTimeout`.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
This implements the "Ceph Config via Ceph Cluster CRD" design document
as a `cephConfig:` structure on the CRD.
This also fixes the `yq` commands used to manipulate the
`cluster-test.yaml` that caused CI issues for this PR and potentially
unknowingly others.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
change the json output of storageclass to also inclusde rados namespace,
and also added the changes in the csi secret
Signed-off-by: parth-gr <paarora@redhat.com>
It's convenient to visualize the supported OSD configurations.
In addition, remove the reference to the common issue section
about OSD on LV-backed PVC because this issue doesn't happen anymore
for new OSDs.
Closes: https://github.com/rook/rook/issues/10859
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
when creating networkFence CR, it requires IP's to block
which we get from running `rbd status ...` command. But,
the client.healthchecker user didn't had the right caps
to run hence it was giving error. Now, adding the required
caps `profile rbd-read-only` to osd so that rbd command can
be executed.
Signed-off-by: subhamkrai <srai@redhat.com>
Blaine Gardner
parth-gr
Travis Nielsen
smoshiur1237
Steven Fackler
mmaoyu
subham rai
Mathieu Parent
ESASHIKA Kaoru
Satoru Takeuchi
Joshua Hoblitt
Liang Zheng
Raghavendra Talur
Yuma Ogami
Rakshith R
Alexander Trost
parth-gr
sp98
subhamkrai