Reduce CRD size by removing some descriptions for things that
(a) are long, (b) are repeated, and (c) are commong kubernetes types.
Remove the `Status` field from VolumeClaimTemplates (a.k.a. PVCs) in
types.go to remove all unnecessary status-related fields.
Use `yq` to go through generated `crds.yaml` to strip out descriptions
for well-known k8s types that are long and repeated where we can't
remove them from types.go. This is mostly placement descriptions and
volumeSource descriptions.
The overall `crds.yaml` file is reduced to less than 60% of its original
size.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
Co-authored-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit bd7477ec61)
# Conflicts:
# deploy/charts/rook-ceph/templates/resources.yaml
# deploy/examples/crds.yaml
Fixes: #13692
Since the introduction of the "host" network provider, the legacy
"hostNetwork" setting is intended to be used only in combination with
the default network provider (""), but the code did not enforce this.
This change adds the required validation checks to throw errors
in invalid constellations.
These checks are added both in the operator's input validation code
and as kubernetes x-validation admission policies in the Cepcluster CRD.
Signed-off-by: Michael Adam <obnox@samba.org>
(cherry picked from commit 25aa15e6c1)
The mds max daemons had been set arbitrarily to 10.
Allow more mds daemons for clusters that have a
scenario for large numbers of mds to manage the
filesystem metadata.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 98816ad79f)
Long CRD descriptions were pushing the size of the CRDs manifest
over 1MB, which causes issues when creating the CRDs. To shorten
the manifest, the descriptions are now limited to 100 chars.
This still gives some ability for dev tools to show a brief
description if needed. The generated CRD docs still contain
the full descriptions for reference.
Signed-off-by: travisn <tnielsen@redhat.com>
(cherry picked from commit 3c1b0692e4)
Allow changing spec.network.hostNetwork settings on a running cluster
by failing over the mons
Signed-off-by: sp98 <sapillai@redhat.com>
(cherry picked from commit 42ea2897cf)
added the cel rules at sunvolumgroup and rados
namesapce spec
closes: #13315
Signed-off-by: parth-gr <partharora1010@gmail.com>
(cherry picked from commit 098f1fe5e8)
Remove some lingering notes stating that features require Ceph Quincy.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
(cherry picked from commit 685a60ed76)
This implements the "Ceph Config via Ceph Cluster CRD" design document
as a `cephConfig:` structure on the CRD.
This also fixes the `yq` commands used to manipulate the
`cluster-test.yaml` that caused CI issues for this PR and potentially
unknowingly others.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
(cherry picked from commit 4ed35d6bd6)
this commit adds validating admission policy
for cephcluster cr according to webhook rules.
Not all the webhook can be moved to validating
admission policy for example checking multus
selector validation.
Signed-off-by: subhamkrai <srai@redhat.com>
Originally we create it using this cmd
ceph fs subvolume create <vol_name> <subvol_name>
So we can have 2 variables filesystem and subvolume name,
Currently the CR doesn't allow us to make subvolume-name
as constant as needed to "csi" because of k8s limitations
Signed-off-by: parth-gr <paarora@redhat.com>
This commit adds new CSIDriverOptions section in
cephCluster CR. This section contains settings
for read affinity and kernel+fuse Mount options
These settings will be injected directly into
rook-ceph-csi-config cm to be applicable per
ceph cluster.
Signed-off-by: Rakshith R <rar@redhat.com>
This patch adds `pgHealthyRegex` field to DisruptionManagementSpec.
`pgHealthyRegex` is a regular expression that is used to determine which
PG states should be considered healthy. The default value of
`pgHealthyRegex` is:
^(active\+clean|active\+clean\+scrubbing|active\+clean\+scrubbing\+deep)$
which is effectively the same as before.
Signed-off-by: Ryotaro Banno <ryotaro.banno@gmail.com>
Allow end-user to have full control on NFS-Ganesha liveness probe:
- Disabled
- Enabled, using user-provided definition
- Enabled, using Rook's default liveness probe for Ganesha.
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Using the new configuration parameters prometheusEndpoint and
prometheusEndpointSSLVerify users can configure dashboard to
point to their prometheus instance.
closes: #12876
Signed-off-by: Redouane Kachach <rkachach@redhat.com>
When OSDs flap, ceph stops the OSD daemon if its marked down greater than
5 times in 600 seconds. But OSD pod restarts and marks the OSD `up` again.
This causes the PGs mapped to these OSDs to peer. While the PGs are peering,
IO to these PGs are blocked.
So we need to ensure that if ceph is marking OSD `down` due to flapping, OSD pod
should not restart to mark the OSDs `up` again.
This PR adds a sleep to the OSD pod if the container returned with a 0 exit code
Default behavior is to sleep for 6 hrs. But user can configure it from the
ceph cluster spec.
Signed-off-by: sp98 <sapillai@redhat.com>
While two mgrs is considered sufficient, more mgr daemons are
now allowed in case the admin wants even more fault tolerance
to the active mgr going down, in case multiple mgrs go down.
Up to five mgr pods will be allowed. All mgrs will be in
standby mode except one active mgr. All mgr pods have a sidecar
that will update their respective pod specs with the active
or passive label.
Signed-off-by: travisn <tnielsen@redhat.com>
Change how Rook detects network CIDRs for Multus networks. The IPAM
configuration is only defined as an arbitrary string JSON blob with a
"type" field and nothing more. Rook's detection of CIDRs for whereabouts
had already grown out of date since the initial implementation.
Additionally, Rook did not support DHCP IPAM, which is a reasonable
choice for users. And more, Rook did not support CNI plugin chaining,
which further complicates NADs. Based on the CNI spec, network chaning
can result in any changes to network CIDRs from the first-given plugin.
All these problems make it more and more difficult for Rook to support
Multus by inspecting the NAD itself to predict network CIDRs. Instead,
it is better for Rook to treat the CNI process as a black box. To
preserve legacy functionality of auto-detecting networks and to make
that as robust as possible, change to a canary-style architecture like
that used for Ceph mons, from which Rook will detect the network CIDRs
if possible.
Also allow users to specify overrides for CIDR ranges. This allows Rook
to still support esoteric and unexpected NAD or network configurations
where a CIDR range is not detectable or where the range detected would
be incomplete. Because it may be impossible for Rook to understand the
network CIDRs wholistically while residing only on a portion of the
network, this feature should have been present from Multus's inception.
Improving CIDR auto-detection and allowing users to specify overrides
for auto-detected CIDRs rounds out Rook's Multus support for CephCluster
(core/RADOS) installations. No further architectural changes should be
needed for CephClusters as regards application of public/cluster network
CIDRs for Multus networks.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
The object user was previously required to be created in the
same namespace as the object store and the cluster. Now,
the object user can be reconciled even in a different namespace
from the cluster and object store. The namespace would be specified
in the object user CR.
Signed-off-by: travisn <tnielsen@redhat.com>
The CRDs were not generated for the latest since #12384
was merged without rebasing, and other updates with a newer
dependency required the newer descriptions.
Signed-off-by: travisn <tnielsen@redhat.com>
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk
Signed-off-by: sp98 <sapillai@redhat.com>
update vault and other indirect imports. Also,
updating k8s package to `v0.28.0-rc.0` since with
v0.27.4 it was through build error.
Signed-off-by: subhamkrai <srai@redhat.com>
Adding CephCOSIDriver CRD and controller. The controller will bring up
the ceph cosi driver when first object store is created in the rook
operator namespace. Then admin can defined COSI CRDs like BucketClass
and BucketAccessClass for different object stores deployed via Rook.
Using the BucketClass and BucketAccessClass, user can define
BucketAccess for backend bucket in the RGW. The CephCOSIDriver CRD
defines configuration options for ceph cosi driver. In the first version
its usability is minimal. Even if it is not defined Rook will bring up
the ceph cosi driver with default values.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Some users want to deploy two CephObjectStores for a single Zone. The first
configures RGWs to process the synchronization of the data, while the second
CephObjectStore configures the client RGWs.
Currently, this can be implemented by setting the RGW option
'rgw_run_sync_thread' in the 'rook-config-override' ConfigMap, though it is not
really user friendly.
Ref: https://docs.ceph.com/en/latest/radosgw/config-ref/#confval-rgw_run_sync_thread
This commit adds a new option in the CephObjectStore CRD as defined in issue
https://github.com/rook/rook/issues/12272. The new option
'disableMultisiteSyncTraffic' determine whether the operator should disable the
multisite sync threads for the RGWs.
If the option is set to 'false', or if the option is not specified, the operator
does nothing. This ensures that the multisite sync threads will not be enabled
for users that disabled explicitely the multisite sync threads either manually
or with the 'rook-config-override' ConfigMap.
This commit also recommends to use two objectstore when scaling Ceph Objectstore
Multisite replication, with one objectstore configured with disabled replication
traffic.
Signed-off-by: Lucas Henry <polyedre@disroot.org>
users & buckets admin capabilities aren't spelled the same
way between rook crd (singular) and ceph (plural). It's a bit
misleading when comparing ceph admin cap and rook users.
Signed-off-by: Peter Goron <peter.goron@gmail.com>
We add a new field domainName to the Kerberos section. The field is used
to setup /etc/idmapd.conf with the domain name. This allows idmapper to
map to kerberos credential to the correct uid/gid.
We add Spec.Security.Kerberos.DomainName to the CRD
Signed-off-by: Sachin Prabhu <sprabhu@redhat.com>
1) donot change rgw fqdn to ip if provided,
As now the bucket class supports the
entry of fqdn
2) update crds with new description in EndpointAddress
Signed-off-by: parth-gr <paarora@redhat.com>
The prometheus mgr module and ceph exporter can now be optionally
disabled by the monitoring.metricsDisabled setting in the
CephCluster CR. These will not be disabled by default, rather
than the mgr module being disabled by default from v1.11.4.
Signed-off-by: travisn <tnielsen@redhat.com>
Pare down the available volume sources for NFS config files so that the
CRD isn't unnecessarily huge. This allows us to recommend
`kubectl apply` again in the upgrade doc.
Size of the NFS CRD is reduced by approximately 60%.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
ClusterID uniquely identifies a cluster. It is used as a prefix to
nslookup exported services.
For example: <clusterid>.<svc>.<ns>.svc.clusterset.local
Signed-off-by: sp98 <sapillai@redhat.com>
Redouane Kachach
Michael Adam
travisn
sp98
parth-gr
parth-gr
Blaine Gardner
Alexander Trost
subhamkrai
Rakshith R
Ryotaro Banno
Shachar Sharon
Deepika Upadhyay
Jiffin Tony Thottan
Lucas Henry
Peter Goron
Sachin Prabhu
Blaine Gardner