we should not use --force when creating filesystem(fs)
as it will recreate fs which will leads to data loss
when `preserveFilesystemOnDelete` is set to 'true'.
this commit removes --force argument when create fs.
Signed-off-by: subhamkrai <srai@redhat.com>
In the latest Ceph releases starting with v16.2.1, all clients are recommended
to be updated so they will have a security fix to connect with a secure
global ID. A health warning will be raised if any insecure clients are connected
and another health warning is raised if insecure clients are still allowed.
Rook will now disable allowing the insecure clients if the health warning
is not being raised to indicate that there are insecure clients still connected.
This means that upgraded clusters will not have this disabled until all the
daemons are updated.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently it is not possible to set resource limits based on device classes
for different OSDs. Now adding an ability to use predefined keys for main
cluster spec Resource section to reflect resource limits for different OSDs
with different device classes.
Closes: https://github.com/rook/rook/issues/8007
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
We don't need to pass the Volume with projection for TLS when TLS is not enabled
Somehow when this happens and we try to update a deployment spec it fails with:
```
ValidationError(Pod.spec.volumes[7].projected): missing required field "sources"
```
Signed-off-by: Sébastien Han <seb@redhat.com>
When configuring encrypted on-pvc clusters we now set the cluster fsid
in the LUKS header. If needed we can then determine if the encrypted
block is part of our cluster or not. We also attach the pvc_name in case
it might become useful.
Closes: https://github.com/rook/rook/issues/7991
Signed-off-by: Sébastien Han <seb@redhat.com>
`rbd mirror pool peer bootstrap create` will use cluster fsid by default, so specifying --site-name shouldn't be needed at all.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the monitors addresses change, we must update the peer secret with
the new addresses. For this, each time the configmap
"rook-ceph-mon-endpoints" is updated, we trigger a reconcile on the
CephBlockPool CRs, which will update the token secret.
Signed-off-by: Sébastien Han <seb@redhat.com>
The rook.io/v1 package was only an internal implementation detail and
does not have any CRDs that rely on it. The CRD deserialization should
handle the change in internal types without any issue. This separation
gives more flexibility for the storage providers to implement exactly
what is needed for their storage provider instead of forcing to use the
same types and risk affecting another storage provider.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Removing the site's name helps us reducing the size of the rbd-mirror
socket path which under certain circumstances could exceed the system
limit.
Also, the namespace information is not that relevant, the cluster FSID
is.
Signed-off-by: Sébastien Han <seb@redhat.com>
A user with unbalanced system (e.g., some OSDs use a partition which
resides on a device along-side OS partition) may want to have a
fine-grained control on the overall load of specific OSD. While osd
weight affects write-load, primary-affinity affects read-load.
This patch allow user to define osd's primary-affinity (similar to
'DeviceClass' and 'InitialWeight'), which is sent to the osd (via mon)
as part of deployment sequence, using 'ceph osd primary-affinity' command.
Valid values are within the range [0.0, 1.0). If not set, default value
is 1.
ROOK issue: https://github.com/rook/rook/issues/7773
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
The unit test TestConfigureCVDevices was failing intermittently
due to comparing the first element of the slice of two elements.
The slice ordering is not guaranteed, which caused a failure
intermittently. The test only compares the critical values in the test
and no need to check elements of the slice.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The deviceClass was only being set for LVM mode OSDs and was
missed for the raw mode for non-PVC. Now the deviceClass will be set
for all expected scenarios.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
currently, recreating erasure codedpPool with
different Settings show error and creation
failed.
Adding `--force` to the erasure coded profile
command works file with above condition.
Signed-off-by: subhamkrai <srai@redhat.com>
The ValidateConnectionDetails() contains `clusterSpec` assuming `SecuritySpec` only
part to `cephCluster` CRD. But we may need to validate same for `SecuritySpec` in
`cephObjectStore` CRD. Hence modifying the signature of the api.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
OSDs not automatically started when adding nodes to existing cluster.
Call the helper NormalizeCrushName(name) to make sure the node name can match the name in the crush map before querying for the node in the crush map, and reconcile when new nodes get added.
Closes: https://github.com/rook/rook/issues/7462
Signed-off-by: parth-gr <partharora1010@gmail.com>
This fixes cases where raw provisioning mode still cannot be used for
Rook v1.6 and Ceph Pacific clusters where json.Unmarshal fails to read
the ceph-volume output after OSD provisioning.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Ceph support the option '--osd-crush-initial-weight' upon OSD start,
which sets an explicit weight (in TiB units) to specific OSD. Allow
passing this option all the way from the user (similar to
'DeviceClass'), for the special case where end users wants it cluster
to have non-even balance over specific OSDs (e.g., one of the OSDs is
placed over a partition alongside OS-partition).
ROOK issue: https://github.com/rook/rook/issues/7448
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
This mops up the last of the multiple-imports within ceph-related
packages, in pkg/daemon/ceph/agent/flexvolume/manager/ceph and
pkg/daemon/ceph/osd.
Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
`ceph osd purge` should remove the OSD auth, but many users have issues
where they are unable to create new OSDs after removing old ones due to
the old OSD auth still being present. Therefore, run `ceph auth del` for
the OSD after purging to fix this.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Update OSDs in parallel per the design in
design/ceph/update-osds-in-parallel.md
The max number of OSDs updated in parallel is currently fixed at 20.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When the cluster is growing, the operator always processes all OSDs and
re-generate their Deployment spec. The generated environment variables
for encryption are coming from a map, which by design is unordered. So
once generated the deployment will see their spec change since the
environment variable position will change.
See from the following diff:
```
env: env:
- name: KMS_SERVICE_NAME <
value: vault <
- name: VAULT_NAMESPACE <
value: ocsns <
- name: VAULT_TLS_SERVER_NAME <
- name: VAULT_BACKEND_PATH <
value: n_ocs <
- name: VAULT_ADDR - name: VAULT_ADDR
value: https://vault.qe.rh-ocs.com:8200 value: https://vault.qe.rh-ocs.com:8200
> - name: VAULT_BACKEND_PATH
> value: n_ocs
> - name: VAULT_TLS_SERVER_NAME
- name: KMS_PROVIDER - name: KMS_PROVIDER
value: vault value: vault
> - name: KMS_SERVICE_NAME
> value: vault
> - name: VAULT_NAMESPACE
> value: ocsns
- name: VAULT_TOKEN - name: VAULT_TOKEN
valueFrom: valueFrom:
secretKeyRef: secretKeyRef:
key: token key: token
name: ocs-kms-token name: ocs-kms-token
```
Signed-off-by: Sébastien Han <seb@redhat.com>
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:
* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`
will be autogenerated for us.
We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.
It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:
```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```
The corresponding API type markers are:
```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```
For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html
Signed-off-by: Sébastien Han <seb@redhat.com>
This is a follow up to:
https://github.com/rook/rook/pull/7264
which added pool quotas. It was stated that it would be nice to
be able to set pool max_bytes quotas using standard k8s quantity
formats rather than an integer representing the number of bytes.
I was too busy at the time to make the changes. This PR adds the
functionality.
Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
subhamkrai
Travis Nielsen
Denis Egorenko
Sébastien Han
Shachar Sharon
shenjiatong
Jiffin Tony Thottan
Satoru Takeuchi
parth-gr
Blaine Gardner
Lars Lehtonen
Blaine Gardner
Frank Ritchie