For all of the controllers besides the cluster controller,
the logging now includes the namespaced name of the resource
that is being reconciled. This will help with log troubleshooting
to help analyze logs consistently for the resource being
reconciled.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The activeStandby property was being ignored when deciding
how many mds daemons to start. 2x the active count of mds
daemons were always being created. If the standby is not
desired, the standby can be removed, and it was unexpected
that it was not removed.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This change introduces support for setting the ceph cache memory limit
ratios that were previously hardcoded.
Codegen was drivey-by updated as the current version errored.
Resolves#16551
Signed-off-by: Tim Buchwaldt <tim.buchwaldt@deepl.com>
Rotate cephx keys for all the MDS daemons based on the cephxConfig in
teh cephCluster CR. Update the cephxStatus in the cephCluster CR and
cephfileSystem CR.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
This change addresses a permission issue where mon pods crashloop
on some Kubernetes setups with SELinux enabled, even when
ROOK_HOSTPATH_REQUIRES_PRIVILEGED is set.
To resolve this, a new function makeMonSecurityContext() was introduced
to explicitly set runAsUser: 0 at the pod level for mon pods
when the environment variable ROOK_CEPH_MON_RUN_AS_ROOT is set to true.
Additionally, the function previously named PodSecurityContext() was renamed
to DefaultContainerSecurityContext() to avoid confusion between
container-level and pod-level security context configuration. All container
SecurityContext usages across Ceph daemons were updated to reflect this change.
This ensures the root user configuration is applied
automatically and consistently in environments where it is required
for mon pod startup, while preserving clear separation of container
and pod-level security logic.
Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com>
This is the first implementation of CephX key rotation in Rook.
Adds key rotation API from design PR 15915.
Also add helper methods for rotating keys, determining when keys
need to be rotated, and for generating CephX key statuses. These will be
reusable for other Rook reconciles beyond object/RGW.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
The ci was using a pretty old version og golangci-lint.
This updates to the latest version.
Additionally, it silences some
gosec integer conversion overflow false positves
and fixes some real errors of this category
and string format errors found by golangci-lint, while at it.
Co-authored-by: Blaine Gardner <b.blaine.gardner@gmail.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Signed-off-by: Michael Adam <obnox@samba.org>
The upgrade of the mds daemons pauses to wait for the
stopping of standby daemons before the filesystem upgrade
is completed. If there are multiple filesystems, there
could be standbys for other filesystems that will not be
stopped at the time of the upgrade of another filesystem.
Thus, the standbys may not upgrade and be stuck on the
previous version. Now, the standby upgrade will only wait
for standbys for the same filesystem.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Resolves CIS benchmark rule 5.7.3, Pods part. SecurityContext
should be explicitly defined in pod level of Pod spec section.
It is sufficient to specify empty securityContext to satisfy
CIS 5.7.3 rule.
5.7.3 Apply Security Context to Your Pods and Containers
When designing your containers and pods, make sure
that you configure the security context for your pods,
containers, and volumes.
Signed-off-by: Peter Razumovsky <prazumovsky@mirantis.com>
When the MDS liveness probe times out, it should not fail the probe. If
the cluster has a network partition or other issue that causes the Ceph
mon cluster to become unstable, `ceph ...` commands can hang and cause
a timeout. In this case, the MDS should not be restarted so as to not
cause cascading cluster disruption.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
Given that Ceph Quincy (v17) is past end of life,
remove Quincy from the supported Ceph versions,
examples, and documentation.
Supported versions now include only Reef and Squid.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This adds an operator config setting ROOK_REVISION_HISTORY_LIMIT
defaulting to kubernetes'value for RevisionHistoryLimit.
If configured, the provided value will be used as RevisionHistoryLimit
for all Deployments rook creates.
Fixes: #12722
Signed-off-by: Michael Adam <obnox@samba.org>
The mds standby test was timing out after six seconds on
three different attempts, causing a single unit test to
take almost 20 seconds. Now we reduce the wait time to
a total of three seconds.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
During certain maintenance tasks the admin will own running
operations on the ceph mgr, rgw, mds and rbd-mirror daemons
and the operator should not interfere with those operations.
Co-authored-by: gauravsitlani <gaurav.sitlani@live.com>
Signed-off-by: subhamkrai <srai@redhat.com>
Change how Rook detects network CIDRs for Multus networks. The IPAM
configuration is only defined as an arbitrary string JSON blob with a
"type" field and nothing more. Rook's detection of CIDRs for whereabouts
had already grown out of date since the initial implementation.
Additionally, Rook did not support DHCP IPAM, which is a reasonable
choice for users. And more, Rook did not support CNI plugin chaining,
which further complicates NADs. Based on the CNI spec, network chaning
can result in any changes to network CIDRs from the first-given plugin.
All these problems make it more and more difficult for Rook to support
Multus by inspecting the NAD itself to predict network CIDRs. Instead,
it is better for Rook to treat the CNI process as a black box. To
preserve legacy functionality of auto-detecting networks and to make
that as robust as possible, change to a canary-style architecture like
that used for Ceph mons, from which Rook will detect the network CIDRs
if possible.
Also allow users to specify overrides for CIDR ranges. This allows Rook
to still support esoteric and unexpected NAD or network configurations
where a CIDR range is not detectable or where the range detected would
be incomplete. Because it may be impossible for Rook to understand the
network CIDRs wholistically while residing only on a portion of the
network, this feature should have been present from Multus's inception.
Improving CIDR auto-detection and allowing users to specify overrides
for auto-detected CIDRs rounds out Rook's Multus support for CephCluster
(core/RADOS) installations. No further architectural changes should be
needed for CephClusters as regards application of public/cluster network
CIDRs for Multus networks.
Signed-off-by: Blaine Gardner <blaine.gardner@ibm.com>
We need to restart all the ceph daemons whenever
cephCluster network settings are modified like
requiremsgr2, encryption and compression. This
required for Ceph to consider the new settings
it require new ceph daemons all over.
Signed-off-by: subhamkrai <srai@redhat.com>
to support mds with multus, we don't need
to add `--public-adr`, if multus is selected as
the network in cephCluster CR.
Signed-off-by: subhamkrai <srai@redhat.com>
Introduce a new env variable ROOK_CSI_IMAGE_PULL_POLICY in rook operator configmap which should be used to
customize the imagePullPolicy for the csi driver and imagePullPolicy property in cephVersionSpec for ceph pods.
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
With octopus coming to end of life, we remove support from
Rook for deploying Ceph Octopus and assume a min version of
Pacific v16. Any checks for octopus or earlier are removed
from the reconciles since they are obsolete.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This removes double package imports. Example:
```
"github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
```
Only one is now being used as shown in go-staticcheck ST1019
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Added unit test for probes changes and
did re-naming of liveness probes file
as it will contain multiple types of probes
Signed-off-by: parth-gr <paarora@redhat.com>
Allow specifying daemon startup probes where we also allow configuring
liveness probes. Startup probes allow Rook to tolerate when Ceph daemons
occasionally take a long time to start up while not also making
Kubernetes liveness probes slower to detect runtime failures of daemons.
Startup probes are beta in Kubernetes 1.18, so we should not enable
probes by default for earlier Kubernetes versions.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Adding Recommended Labels on the resources created by rook
and using Recommended Labels in the helm chart,
for better visuals and management of k8s object
Closes: https://github.com/rook/rook/issues/8400
Signed-off-by: parth-gr <paarora@redhat.com>
This commit adds context parameter to k8sutil deployment functions. By
this, we can handle cancellation during API call of deployment resource.
Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
The MDS core team suggested with deploy the MDS daemon first and then do
the filesystem creation and configuration. Reversing the sequence lets
us avoid spurious FS_DOWN warnings when creating the filesystem.
Closes: #8745
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:
* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited
As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.
A second new controller for the operator's general config has been
created, it manages:
* the logging level
* the ceph CLI command timeout
* the discovery daemon
The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.
Signed-off-by: Sébastien Han <seb@redhat.com>
Recently, the builds of `ceph/ceph` image moved to quay.io, see
https://github.com/ceph/ceph-build/pull/1883 for more details.
Current images will remain but new builds will happen on quay.io only.
This means that tags such as `v14.2`, `v15.2`,`v16.2` will need to
switch to quay.io to get updates.
Signed-off-by: Sébastien Han <seb@redhat.com>
earlier, for mds, when resource limit is not defined it has
default limit of `"mds_cache_memory_limit": "4294967296"`
even if resouce.request is defined.
Now, resource request will be given preference when both
request and limit is defined and if only limit is defined
it will be applied or otherwise.
Closes: https://github.com/rook/rook/issues/8143
Signed-off-by: subhamkrai <srai@redhat.com>
The rook.io/v1 package was only an internal implementation detail and
does not have any CRDs that rely on it. The CRD deserialization should
handle the change in internal types without any issue. This separation
gives more flexibility for the storage providers to implement exactly
what is needed for their storage provider instead of forcing to use the
same types and risk affecting another storage provider.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
set fsPreparedForUpgrade to true for upgrading mds, to
make sure that max_mds is set back to active count
after all mds agent gets upgraded
Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
Travis Nielsen
Tim Buchwaldt
José Martínez
Santosh Pillai
Patryk Rostkowski
Blaine Gardner
Carlos Barria
Artem Torubarov
Joshua Hoblitt
parth-gr
Peter Razumovsky
subham rai
gauravsitlani
parth-gr
subhamkrai
Avan Thakkar
Alexander Trost
Blaine Gardner
Yuichiro Ueno
Sébastien Han
shenjiatong