Rook's crashcollector pod posts entries to the ceph cluster when a crash occurs.
Over time the number cluster may hold crash entries needlessly.
To clean up old crash entries, this PR adds a field to the ceph cluster CR for the user to specify the number of days a crash entry should be kept for.
Providing a value for the field keepXDays creates a cronjob that runs every day at midnight, calling "ceph crash prune <keepXDays>".
Closes: https://github.com/rook/rook/issues/6332
Signed-off-by: Renan Campos <rcampos@redhat.com>
-creates a single PDB (max-unavailable=1) for all OSDs. This PDB allows one OSD to go down at a given time.
-When a drain is detected, blocking PDBs (max-unavailable=0) will be created for each failure domain that is not being drained and the main PDB (max-unavilable=1) will be deleted. This will allow all the OSDs in the currently drained failure domain to be removed while blocking the deletion of OSDs in other failure domains.
-Once the PGs are healthy again, the blocking PDBs will be deleted and the main PDB will be restored.
-Add PG healthcheck timeout
-Delete any legacy node drain pods and blocking OSD PDBs
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Now, we can schedule snapshots on pools from the CephBlockPool CR when
the pool is mirrored.
It can be enabled like this:
```
mirroring:
enabled: true
mode: pool
snapshotSchedules:
- interval: 24h # daily snapshots
startTime: 14:00:00-05:00
```
Multiple schedules are supported since snapshotSchedules is a list.
Signed-off-by: Sébastien Han <seb@redhat.com>
Added the support of NAD from diffrent namespaces.
they can be referrenced as <namespace>/<name-of-nad> e.g.,
default/public-nw.
Updated the multus doc to explain the same.
Signed-off-by: rohan47 <rohgupta@redhat.com>
The CRs must be created in a separate kubectl create command
from the creation of the CRDs, otherwise the create command
will fail.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Use common.yaml to apply changes to resources instead of keeping an
upgrade-*-apply.yaml file. Update the upgrade docs to reflect this also.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This adds the functionality to add custom pod labels to the CSI
components through the operator configuration way of env vars or config
map.
Resolves#6593
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The CRDs need to be installed and updated separately from the
rest of the manifests. Helm charts also do not have a way to update
CRDs, therefore, any updates to the CRD schema must be done separately
from the helm chart. Now the CRDs are created in a new crd.yaml that must
be created along with common.yaml and before creating the operator.
The helm chart still contains the CRDs for initial creation, though
the helm chart will never update or remove them.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The description of cleanupPolicy is a bit ambiguous. First, we should clarify
that both skipping orchestration and data cleanup will happen iff
"yes-really-destroy-data" is set to `confirmation`. Second, We should also
describe wiping devices will happen even if user don't specify `sanitizeDisks`
explicitly.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Since Rook no longer supports legacy filestore devices, there is
no need to keep testing upgrades from v1.2 all the way to master.
We can now just test v1.4 to master.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The discovery daemon is not needed in most scenarios, therefore we disable it
by default. More and more clusters are moving to the cluster-on-pvc scenario
which certainly does not need the local discovery. Even where clusters are not
running on PVCs, the discovery is not needed since the device discovery is again
performed in the osd prepare job.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If users want to restrict the nodes where Ceph daemons should exist,
it's better to make the placement of admission controller configurable
as other daemons.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The nullable attribute was causing the integration tests to fail
on 1.13 and earlier where it is not supported in the schema.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit cc8f406ce9)
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
cephfs csi driver still uses the ceph CLI for all the
ceph fs operations, when cephcsi gets a request from
the kubernetes it will internally starts threads for
each call. in cephcsi we are setting the pids
limit to max, to set this value the container has
to be running as the privileged one.
if we go with default pid limit we will face core dump
due to resource crunch.
without privileged access we will see Failed to set new
PID limit to -1: open /sys/fs/cgroup/pids/kubepods.slice/
kubepods-besteffort.slice/kubepods-besteffort-pod1029ed37_bebc_466d_9e53_79e7323a67ed.slice/crio-9ef6768d027916f414bf30b54ef13eebad8efd0aa4741d97618affa838d7b4c0.scope/pids.max: permission denied
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In clusters where only two datacenters (or similar failure domains)
are available, a different mon and osd approach is needed to deal
with the network partitions or some other reason for one of the failure
domains going down. The Ceph stretched cluster makes the mons aware
of the failure domains by configuring one as the arbiter in a third
zone, while keeping two replicas of the data in each of the data
zoens.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently, Rook does not support a prometheus exporter port different
than 9283 so let's fail and force the user to set it to the default.
Signed-off-by: Sébastien Han <seb@redhat.com>
as we are approaching the rook 1.5 release, we
are making a required RBAC changes for cephcsi ahead
of cephcsi release to provide smooth upgrade experience
for the users without any RBAC changes in rook minor release.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Fixed an issue where monitoring port, even when set by user, being
overridden by the default value.
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
Changes related to Monitoring endpoint
a. for any monitoring endpoint and port (either user given through arguments
or internally found), added code to make sure we convert the endpoint
host_name to a proper IP address.
b. checks added to make sure we have a valid and reachable endpoint
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
Due to #6492, preservePoolsOnDelete is not useful at all for CephFS;
after the filesystem is deleted, the leftover pools cannot be
reassocaited with a newly created filesystem without wiping all
metadata. The only way we can actually preserve data is keeping around
the entire filesystem.
This commit implements a `preserveFilesystemOnDelete` option which work
similar to the existing pool preservation option but instead keeps the
whole CephFS while taking it down and removing all MDSes.
This commit also changes all documentation to refer to this new option
with the intent of essentially deprecating `preservePoolsOnDelete`. IMO,
keeping around `preservePoolsOnDelete` is actively harmful because it
lulls users into thinking their data will be safe but, in reality,
recovering from this situation is highly complex and has large potential
for data loss.
Signed-off-by: Lalit Maganti <lalitm@google.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
The CRUSH map does not allow duplicate values (nor keys) for the
labels. That means that if the currently hardcoded label value
`default` occurs anywhere else in the tree (for example, because
some of the topology labels provided by the cloud provider use it),
the cluster cannot sucessfully spawn.
By giving the user control over the label value used for the root
CRUSH map label, it is possible to adapt the cluster to this
situation.
To be able to actually use the `default` value, the root=default
node which is created by default by ceph itself has to be removed;
since that node is accompanied by a replicated crush rule, we have
to remove that rule, too.
The integration tests are modified to sometimes use the custom
crushRoot (based on an arbitrary criterium) to get coverage across
the various suites. Separate tests could be added, but were not
deemed necessary at this point.
Fixes#4993.
Signed-off-by: Jonas Schäfer <jonas.schaefer@cloudandheat.com>
Renan Campos
Madhu Rajanna
Travis Nielsen
Santosh Pillai
Sébastien Han
rohan47
Blaine Gardner
Alexander Trost
subhamkrai
Satoru Takeuchi
Mateusz Gozdek
Arun Kumar Mohan
Jared Watts
Lalit Maganti
Jonas Schäfer