The description of cleanupPolicy is a bit ambiguous. First, we should clarify
that both skipping orchestration and data cleanup will happen iff
"yes-really-destroy-data" is set to `confirmation`. Second, We should also
describe wiping devices will happen even if user don't specify `sanitizeDisks`
explicitly.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Since Rook no longer supports legacy filestore devices, there is
no need to keep testing upgrades from v1.2 all the way to master.
We can now just test v1.4 to master.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The discovery daemon is not needed in most scenarios, therefore we disable it
by default. More and more clusters are moving to the cluster-on-pvc scenario
which certainly does not need the local discovery. Even where clusters are not
running on PVCs, the discovery is not needed since the device discovery is again
performed in the osd prepare job.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If users want to restrict the nodes where Ceph daemons should exist,
it's better to make the placement of admission controller configurable
as other daemons.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The nullable attribute was causing the integration tests to fail
on 1.13 and earlier where it is not supported in the schema.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
(cherry picked from commit cc8f406ce9)
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
cephfs csi driver still uses the ceph CLI for all the
ceph fs operations, when cephcsi gets a request from
the kubernetes it will internally starts threads for
each call. in cephcsi we are setting the pids
limit to max, to set this value the container has
to be running as the privileged one.
if we go with default pid limit we will face core dump
due to resource crunch.
without privileged access we will see Failed to set new
PID limit to -1: open /sys/fs/cgroup/pids/kubepods.slice/
kubepods-besteffort.slice/kubepods-besteffort-pod1029ed37_bebc_466d_9e53_79e7323a67ed.slice/crio-9ef6768d027916f414bf30b54ef13eebad8efd0aa4741d97618affa838d7b4c0.scope/pids.max: permission denied
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In clusters where only two datacenters (or similar failure domains)
are available, a different mon and osd approach is needed to deal
with the network partitions or some other reason for one of the failure
domains going down. The Ceph stretched cluster makes the mons aware
of the failure domains by configuring one as the arbiter in a third
zone, while keeping two replicas of the data in each of the data
zoens.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently, Rook does not support a prometheus exporter port different
than 9283 so let's fail and force the user to set it to the default.
Signed-off-by: Sébastien Han <seb@redhat.com>
as we are approaching the rook 1.5 release, we
are making a required RBAC changes for cephcsi ahead
of cephcsi release to provide smooth upgrade experience
for the users without any RBAC changes in rook minor release.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Fixed an issue where monitoring port, even when set by user, being
overridden by the default value.
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
Changes related to Monitoring endpoint
a. for any monitoring endpoint and port (either user given through arguments
or internally found), added code to make sure we convert the endpoint
host_name to a proper IP address.
b. checks added to make sure we have a valid and reachable endpoint
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
Due to #6492, preservePoolsOnDelete is not useful at all for CephFS;
after the filesystem is deleted, the leftover pools cannot be
reassocaited with a newly created filesystem without wiping all
metadata. The only way we can actually preserve data is keeping around
the entire filesystem.
This commit implements a `preserveFilesystemOnDelete` option which work
similar to the existing pool preservation option but instead keeps the
whole CephFS while taking it down and removing all MDSes.
This commit also changes all documentation to refer to this new option
with the intent of essentially deprecating `preservePoolsOnDelete`. IMO,
keeping around `preservePoolsOnDelete` is actively harmful because it
lulls users into thinking their data will be safe but, in reality,
recovering from this situation is highly complex and has large potential
for data loss.
Signed-off-by: Lalit Maganti <lalitm@google.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
The CRUSH map does not allow duplicate values (nor keys) for the
labels. That means that if the currently hardcoded label value
`default` occurs anywhere else in the tree (for example, because
some of the topology labels provided by the cloud provider use it),
the cluster cannot sucessfully spawn.
By giving the user control over the label value used for the root
CRUSH map label, it is possible to adapt the cluster to this
situation.
To be able to actually use the `default` value, the root=default
node which is created by default by ceph itself has to be removed;
since that node is accompanied by a replicated crush rule, we have
to remove that rule, too.
The integration tests are modified to sometimes use the custom
crushRoot (based on an arbitrary criterium) to get coverage across
the various suites. Separate tests could be added, but were not
deemed necessary at this point.
Fixes#4993.
Signed-off-by: Jonas Schäfer <jonas.schaefer@cloudandheat.com>
the apiextensions.k8s.io/v1beta1 version of CustomResourceDefinition
is deprecated in Kubernetes v1.16 and will no longer be supported from
v1.19. For now, we changing only for ceph and it's related documented.
Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: travisn <tnielsen@redhat.com>
Signed-off-by: subhamkrai <srai@redhat.com>
The default number of provisioner pods are 2. When there is only one node,
for example the CI environment, one of these becomes Pending state.
Although it's harmless, it's better to reduce this value to 1.
Closes: https://github.com/rook/rook/issues/6294
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The pool spec has now a new property called "replicasPerFailureDomain"
which essentially represents the number of replicas to store in each
failure domain.
Assuming the failure domain is a datacenter (if the cluster is
stretched) then you will have 2 replicas per datacenter where each
replica ends up on a different host. This gives you a total of 4
replicas and for this, the "size" must be set to 4.
Closes: https://github.com/rook/rook/issues/5591
Signed-off-by: Sébastien Han <seb@redhat.com>
The CI runs a github action that builds and deploys a Rook cluster and
then execute various tests on the create-external-cluster-resources.py
script.
Signed-off-by: Sébastien Han <seb@redhat.com>
The ceph mons should never be started with an even number in quorum.
The desired state should always be an odd number of mons to ensure
a healthy majority quorum. The operator now rejects a request for
an even number of mons.
If there are not a sufficient number of nodes for mons to be on a
unique host, the operator also returns an early error instead of
getting stuck waiting for a pending mon. This is not foolproof since
there might be fewer available nodes for the mons, but at least it
is a quick check for the common case.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
An '--upgrade' option is added to the option list, which upgrades the user
permissions (caps) to match newer version. Will try to detect
'rgw-pool-prefix' or user can provide it through the argument.
Segregated the command line arguments into three groups, common, output and
upgrade
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
on Azure, all SSDs are reporting as a rotational disk.
This commit will add `tuneFastDeviceClass` flag which will
tune for an SSD even if the cloud provider is reporting
rotational. We have a flag `tuneDeviceClass`
which works vice-versa.
Signed-off-by: subhamkrai <srai@redhat.com>