In order to help users check that they have implemented the newly-added
Multus host configuration prerequisites, add a check to the validation
tool to verify connectivity.
Because users who are already running clusters with Multus enabled, add
a flag that allows users to only check for host configuration
prerequisites. This mode will not start the large number of clients that
would normally be started because those clients could disrupt a running
Rook cluster negatively.
Host checking pods require host network access. Many Kubernetes
distributions have pod security features enabled. In order to allow
non-Vanilla distros to run this tool, allow specifying a service account
that pods will run as, which can be configured by the admin to allow
test pods.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 33f5407dd4)
# Conflicts:
# Documentation/CRDs/Cluster/network-providers.md
# deploy/examples/multus-validation.yaml
For an advanced network configuration, the mon endpoints
expected for the ceph public network may be different from
the public IP address that is set on the K8s node. When
host networking is enabled in rook, the IP address of the
node was only allowed previously. Now, the IP address
used for a mon when scheduled on a node can be determined
by setting an annotation on each node where mons can run:
network.rook.io/mon-ip: <ip-address>
This annotation must be set before the mons are created.
Mon endpoints cannot be changed after they are already
running.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit e90ed3b912)
When the clusters reach full, nearfull, or backfill full thresholds
ceph will raise health warnings and stop allowing IO or backfill
depending on the threshold. These settings require special ceph
commands instead of being generic ceph config. Allow these settings
to be set from the CephCluster CR in the spec.storage
section.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit e479b051bc)
This PR adds a `settings` option in the `Module` struct
which can help users to futher customize the `balancer`
module with a differet mode.
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 1ad37ecce9)
LVM-based OSDs on PVCs are legacy and are not well tested.
Therefore, we add status to the cephcluster CR that at least
gives a clue that the OSDs should be replaced.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit 3628e83892)
mkdocs uses a markdown renderer that is hardcoded to 4 spaces per tab
for detecting indentation levels, including ordered- and
unordered-lists. Since we cannot easily change the renderer, begin using
a markdown linter in CI that will fail if official docs do not adhere to
the spacing rules.
As a starting point, the markdownlint config does not begin with the
default set of checks, which might overwhelm attempts to fix them.
Instead, focus on list-tab-spacing rules and a few other highly useful
checks.
markdownlint also has some gaps in its abilities that allow common Rook
doc issues to pass acceptance. However, it allows creating custom
linting plugins. Create 2 such linting plugins to check 2 things:
- all doc lines (except code blocks) must be aligned to a 4-space
boundary, without exception. This ensures that markdown will render
correctly with mkdocs. This unfortunately makes it possible to create
lists that are internally aligned strangely.
- admonitions must all follow the same format of
```
!!! header
body
```
For the strange lists, this is allowed and renders correctly, but it
looks strange:
```md
- first bullet
- second bullet
still second bullet
- third bullet
has a paragraph
of text inside
- last bullet
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit c9d99e01a0)
This PR will add separate directory to handle the external manifest. Here external manifests will have same namespace (rook-ceph) for external cluster.
Signed-off-by: smoshiur1237 <moshiur.rahman@est.tech>
(cherry picked from commit dab17f72a1)
Unlike other components, the Ceph dashboard Service object could not be
configured with custom labels and annotations. This can be useful when
integrating Rook into a cluster with custom controllers similarly to how
the RGW service is customized.
Signed-off-by: Steven Fackler <sfackler@gmail.com>
(cherry picked from commit 0c38dd81c2)
Example cephobjectstore uses wrong field name for "dnsNames".
Signed-off-by: Adam Prycki <adam.prycki@gmail.com>
(cherry picked from commit c1e727384a)
The following printcolumns were added
CephBlockPoolRadosNamespace: Phase,
BlockPoolName, Age
CephFilesystemSubVolumeGroup: Phase,
FilesystemName, PinningConfig, Age
Also improved the error messages in
SubVolumeGroupClient
Signed-off-by: NymanRobin <nyman.robin@gmail.com>
(cherry picked from commit 70a7e61da5)
For check if cluster PGs are healthy before osd updates
Which helps gainning more confidence in the osd upgrades
Signed-off-by: mmaoyu <wupiaoyu61@gmail.com>
(cherry picked from commit 6d4afe23e5)
cephfs subvolumegroup supports creating svg
with quota and the datapool, This PR adds the
support for the same.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit f57a8b6bbe)
Most bullets and numbering in the network providers doc (sub-doc of
CephCluster CR) wasn't rendering properly on the web. Fix by making sure
there is a blank line before each numbered/bulleted section. And make
sure nested numberings are indented 4 spaces.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit ad9e0976e2)
The network providers doc, including the holder pod migration sections,
were developed with the understanding that CSI could be placed into pod
networking mode if desired by users. In reality, this hasn't been
supported due to bugs.
This allows streamlining the holder pod migration sections to no longer
need to document for users what to do if they use CSI in pod network
mode.
Rook will continue to have the `CSI_ENABLE_HOST_NETWORK` config
available as an undocumented value. This can allow users who desperately
cannot use host networking to still use pod networking, though this will
mean any CSI node plugin updates could stall mounts for running apps.
Thus we do not support it.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 519c7e8911)
csi pr get merged ceph/ceph-csi#4459
so, This reverts commit 812a9c02b0.
The csi PR removes the dependency of the pool parameter from the
rbd storage class if the topology pools are passed,
so removing them in the examples and making it optional
Signed-off-by: parth-gr <partharora1010@gmail.com>
Add new CSI_DISABLE_HOLDER_PODS option for rook-ceph-operator.
This option will disable holder pods when set to "true".
In the long term, Rook plans to deprecate the holder pods entirely.
This new option will allow users to choose to migrate their clusters to
non-holder clusters when they are ready and able, giving them time to
gracefully migrate before the holders are permanently removed.
This option is set to "false" by default so that upgrading users don't
have their CSI pods modified unexpectedly.
Example manifests are modified to set this value to true so that new
clusters will not deploy holder pods.
Migrating users are provided with documentation to instruct them about
the new requirements they need to satisfy to successfully remove holder
pods, a procedure for migrating pods from holder to non-holder mounts,
and a way to delete holder pods once they are no longer in use.
When users set CSI_DISABLE_HOLDER_PODS="true", the CSI controller will
no longer deploy or update the holder pod Daemonsets, but it does not
delete any existing Daemonsets. This allows already-attached PVCs to
continue operating normally with their network connection continuing to
exist in the current holder pod. This is critical to avoid causing
ia cluster-wide storage outage.
More info: https://github.com/rook/rook/issues/13055
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Until now, an object store would create all the necessary
metadata pools and the data pool that were exclusively
for its own object store. When isolation between object
stores is necessary, this would cause many pools and
PGs to be created in the cluster, which was not
manageable.
Now one set of pools can be created to be shared
by any number of object stores. The metadata and data
between each object store is isolated by
RADOS namespaces, which by design will keep the
data safe for multi-tenancy.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With the release of Reef v18.2.2 we update the default
recommended version of Ceph to fix the prometheus issues
seen with v18.2.1
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
with topologyconstrain pool we can enable any
replica pool usage and also can store data at any
topology
Signed-off-by: parth-gr <partharora1010@gmail.com>
We have prohibited changing all network-related configurations after cluster creation.
However, as a result of https://github.com/rook/rook/discussions/13584, we found that
the connections.encryption flag can be regarded as an exception.
Signed-off-by: ESASHIKA Kaoru <kaoru-esashika@cybozu.co.jp>
the rgw service was removed, because users reported certificate
issues when Rook created Services for RGWs that had TLS (HTTPS) enabled.
It is now up to the user to create a Service if they want to use one.
part-of: https://github.com/rook/rook/discussions/13863
Signed-off-by: parth-gr <partharora1010@gmail.com>
Some users are still confused by Ceph networking, especially involving
Multus. Continuing to add network provider-specific information -- a
pretty advanced concept -- to the already-heavy CephCluster CRD doc is
becoming cumbersome. Separate network provider settings into a separate
child doc that notes it is an advanced topic.
This separation will also help with work to remove the holder pod
design, which requires additional Multus prerequisites.
See: https://github.com/rook/rook/issues/13055
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Reduce CRD size by removing some descriptions for things that
(a) are long, (b) are repeated, and (c) are commong kubernetes types.
Remove the `Status` field from VolumeClaimTemplates (a.k.a. PVCs) in
types.go to remove all unnecessary status-related fields.
Use `yq` to go through generated `crds.yaml` to strip out descriptions
for well-known k8s types that are long and repeated where we can't
remove them from types.go. This is mostly placement descriptions and
volumeSource descriptions.
The overall `crds.yaml` file is reduced to less than 60% of its original
size.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Co-authored-by: Blaine Gardner <blaine.gardner@ibm.com>
Rook has been setting the application automatically on all
pools to rbd for CephBlockPools, rook-ceph-rgw for
CephObjectStores, mgr on the built-in .mgr pool,
and nfs on the built-in .nfs pool.
The legacy pool device_health_metrics is long gone
from Pacific which is no longer supported, so we can
remove special handling for that pool in the upgrade
guide and in the code.
The application setting is now available on the pool spec
although it is not expected to commonly need to override
the default applications set by Rook.
The application for CephFilesystem pools is now being
set to cephfs, where it was previously blank.
Signed-off-by: travisn <tnielsen@redhat.com>
Blaine Gardner
parth-gr
Travis Nielsen
sp98
Michael Adam
smoshiur1237
mergify[bot]
Steven Fackler
ev
NymanRobin
mmaoyu
Madhu Rajanna
Liang Zheng
subham rai
Mathieu Parent
Jiffin Tony Thottan
ESASHIKA Kaoru
Redouane KachachandBlaine Gardner
Satoru Takeuchi