The NFS reconciler was not updating NFS daemons that already existed.
The controller would fail to update daemons if the number of active
daemons did not increase or decrease.
Make sure existing NFS daemons are updated to new versions with all
scale up and scale down events.
Resolves#6611
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This adds the functionality to add custom pod labels to the CSI
components through the operator configuration way of env vars or config
map.
Resolves#6593
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
When new pools are added to an already existing filesystem, we would
create them but not actually associate them to the filesystem. Ensure
that we do this.
Also, while we're here, we refactor the CreateFilesystem test: before,
we were not properly testing creating the filesystem from scratch
because we would return valid JSON for *every* ceph command. Change this
by making the first run return an error so that we get some better
coverage.
After this, we also add a check for the correctness of adding new pools
to the CreateFilesystem unittest.
Fixes#5876
Signed-off-by: Lalit Maganti <lalitm@google.com>
The RGW deployment's ceph-version label should now display the same
version as the image it is using.
It previously displayed an empty version: 0.0.0-0.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
If users want to restrict the nodes where Ceph daemons should exist,
it's better to make the placement of admission controller configurable
as other daemons.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
These are a collection of unstable ceph release that are not recommended
for production. It is recommended to rollback to the previous or
upgrading to the next one if possible.
Signed-off-by: Sébastien Han <seb@redhat.com>
Instead of directly defining our own constants, we should be using the k8s
constants for the well-known topology labels.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Pools in stretch clusters will all use the same crush rule that
is generated by the operator when configuring the cluster for
stretch mode, with replica 4 and two replicas per failure domain.
Pools cannot create new crush rules, so we require that replica be
4 when the pools is created, EC is not allowed, and no new
crush rule will be created.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In clusters where only two datacenters (or similar failure domains)
are available, a different mon and osd approach is needed to deal
with the network partitions or some other reason for one of the failure
domains going down. The Ceph stretched cluster makes the mons aware
of the failure domains by configuring one as the arbiter in a third
zone, while keeping two replicas of the data in each of the data
zoens.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The validation code for the admission controller is found in the
ceph.rook.io/v1 package, corresponding to each of the types.
To be better organized, the validators are now found in a source
file of the corresponding type name.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The pool subdomains were being validated based on the names of the
crush buckets instead of the crush failure domains, or types.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The mon canaries are now being added and removed at the same time
instead of starting or stopping them one-by-one. This will simplify
scheduling execution time to much more quickly detect where the mons
can be scheduled.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd purge job was not removing the osd prepare job or the
osd pvc due to an invalid label query for the pvc. Now the
prepare job and pvc will be removed as expected from the job.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We now support VAULT_TLS_SERVER_NAME which represents the name to use as
the SNI host when connecting via TLS.
This is typically needed when a single host has multiple certificates,
same as virtual hosts but for SSL certificates.
Signed-off-by: Sébastien Han <seb@redhat.com>
Golang maps order is not always guaranteed to be identical, which makes
the table driven test to fail. Because of the nature of the maps, it is
hard to predict the exact position of an element in the map. This
makes the table driven test flappy.
Reworked the unit test using a contains statement which achieves the
same validation.
Closes: https://github.com/rook/rook/issues/6522
Signed-off-by: Sébastien Han <seb@redhat.com>
Due to #6492, preservePoolsOnDelete is not useful at all for CephFS;
after the filesystem is deleted, the leftover pools cannot be
reassocaited with a newly created filesystem without wiping all
metadata. The only way we can actually preserve data is keeping around
the entire filesystem.
This commit implements a `preserveFilesystemOnDelete` option which work
similar to the existing pool preservation option but instead keeps the
whole CephFS while taking it down and removing all MDSes.
This commit also changes all documentation to refer to this new option
with the intent of essentially deprecating `preservePoolsOnDelete`. IMO,
keeping around `preservePoolsOnDelete` is actively harmful because it
lulls users into thinking their data will be safe but, in reality,
recovering from this situation is highly complex and has large potential
for data loss.
Signed-off-by: Lalit Maganti <lalitm@google.com>
Just like we do for the monitors, we set an ownerref to the OSD PVC so
when the cluster is deleted we remove the PVC too.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
To clarify the spec of ApplyToObjectMeta This also fixes the misleading
comments on the method.
Signed-off-by: Yusei Nishiyama <yusei.nishiyama@soundcloud.com>
The CRUSH map does not allow duplicate values (nor keys) for the
labels. That means that if the currently hardcoded label value
`default` occurs anywhere else in the tree (for example, because
some of the topology labels provided by the cloud provider use it),
the cluster cannot sucessfully spawn.
By giving the user control over the label value used for the root
CRUSH map label, it is possible to adapt the cluster to this
situation.
To be able to actually use the `default` value, the root=default
node which is created by default by ceph itself has to be removed;
since that node is accompanied by a replicated crush rule, we have
to remove that rule, too.
The integration tests are modified to sometimes use the custom
crushRoot (based on an arbitrary criterium) to get coverage across
the various suites. Separate tests could be added, but were not
deemed necessary at this point.
Fixes#4993.
Signed-off-by: Jonas Schäfer <jonas.schaefer@cloudandheat.com>
This allows the catch-22 situation where the filesystem cannot be
reconciled because there is no MDS but there is no MDS because the
operator has not reconciled the filesystem and brought up the MDS pods.
Closes#5967, #5846
Signed-off-by: Lalit Maganti <lalitm@google.com>
We don't always get an error of the type "*exec.ExitError" so we must
validate the type before printing it otherwise the interface conversion
will fail.
Signed-off-by: Sébastien Han <seb@redhat.com>
adding fields `TotalBytes`, `UsedBytes`, `AvailableBytes`
`LastUpdated` to the ceph cluster CR's status section to
report the storage details of ceph cluster.
`TotalBytes`: Total storage capacity in bytes
`UsedBytes`: Total storage used in bytes
`AvailableBytes`: Total storage used in bytes(Total - Used)
`LastUpdated`: Last update time
for example(I have added 10GB disk):
`kubectl get cephclusters.ceph.rook.io -o yaml`
```
status:
ceph:
capacity:
bytesAvailable: 9657188352
bytesTotal: 10733223936
bytesUsed: 1076035584
lastUpdated: "2020-10-23T12:57:48Z"
```
Signed-off-by: subhamkrai <srai@redhat.com>
The mon resources may be orphaned if there is a mon failover
that was cancelled, for example a mon pvc may remain if the
original mon comes up and the operator was restarted before
the mon failover completed. The cleaning up of mon resources
was only happening was each successful mon failover, now we
perform the check with every reconcile since the resources
were not being cleaned up predictably enough.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add unit tests for when the "Zone" is configured
in an object store so that the object store joins
the CephObjectZone and the zone's corresponding
multisite configuration.
Signed-off-by: Ali Maredia <amaredia@redhat.com>
updated mon pdb reconcile to delete mon pdb and create a new one when mon count changes in the cluster spec
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
The number of admission controller pods is 2. When there is only one
node as the CI environment, one of these becomes Pending state.
Although it's harmless, it's better to reduce this value to 1.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Blaine Gardner
Alexander Trost
Travis Nielsen
Varsha Rao
Blaine Gardner
Lalit Maganti
subhamkrai
Satoru Takeuchi
Sébastien Han
Mateusz Gozdek
Jared Watts
Yusei Nishiyama
Jonas Schäfer
Ali Maredia
Santosh Pillai