Since the recent auto crds generation, I missed a few fields, re-adding
now:
* additionalPrinterColumns
* min and max
Signed-off-by: Sébastien Han <seb@redhat.com>
With simultaneous node drains, a mon can go down while another mon is failing over. This results in two mons down and ceph become inaccessible. The commit makes maxUnavailable=0 while a mon is failing over and updates it back to maxUnavailable=1 after the failover. This update action is best effort. Any errors while updating the maxUnavailable mon pdb is only logged
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
- Update mon PDB to use maxUnavailable=1 instead of minAvailable. The maxUnavailable will always be 1 irrespective of the number of mons in the cluster
- Move mon PDB reconcile logic from Disruption Controller to Cluster Controller
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Update OSDs in parallel per the design in
design/ceph/update-osds-in-parallel.md
The max number of OSDs updated in parallel is currently fixed at 20.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When a private docker registry is used and an image pull secret is specified in the chart, the pods with default Service Account fail to pull the image due to authentication issues.
Added rook-ceph-default service account and modify the pods specifications by adding the serviceAccountName.
Closes: https://github.com/rook/rook/issues/6673
Co-authored-by: Tareq Sharafy <tareq.sha@gmail.com>
Signed-off-by: parth-gr <partharora1010@gmail.com>
Document codespell flags that are necessary. Remove codespell flags with
minor code changes if possible.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When the cluster is growing, the operator always processes all OSDs and
re-generate their Deployment spec. The generated environment variables
for encryption are coming from a map, which by design is unordered. So
once generated the deployment will see their spec change since the
environment variable position will change.
See from the following diff:
```
env: env:
- name: KMS_SERVICE_NAME <
value: vault <
- name: VAULT_NAMESPACE <
value: ocsns <
- name: VAULT_TLS_SERVER_NAME <
- name: VAULT_BACKEND_PATH <
value: n_ocs <
- name: VAULT_ADDR - name: VAULT_ADDR
value: https://vault.qe.rh-ocs.com:8200 value: https://vault.qe.rh-ocs.com:8200
> - name: VAULT_BACKEND_PATH
> value: n_ocs
> - name: VAULT_TLS_SERVER_NAME
- name: KMS_PROVIDER - name: KMS_PROVIDER
value: vault value: vault
> - name: KMS_SERVICE_NAME
> value: vault
> - name: VAULT_NAMESPACE
> value: ocsns
- name: VAULT_TOKEN - name: VAULT_TOKEN
valueFrom: valueFrom:
secretKeyRef: secretKeyRef:
key: token key: token
name: ocs-kms-token name: ocs-kms-token
```
Signed-off-by: Sébastien Han <seb@redhat.com>
When no VAULT_BACKEND_PATH is specified we default to the API one which
is set to "secret/", we could use a different variable but we don't want
to break the library support. Bonus point this will also fix issue with
users passing a slash at the end of the path.
Signed-off-by: Sébastien Han <seb@redhat.com>
Previously, `interval` was just a string so no special validation was
done by the OpenAPI validator. Now that it is advertised as metav1.Duration
the server will introspect it correctly. Underneath the type is still a
string so no update issue to be worried about.
Signed-off-by: Sébastien Han <seb@redhat.com>
When running encrypted OSDs on multipath the expand container was
failing to find the underlying device of the encrypted block. This was
because the underlying block was located in /dev/mapper and this path
wasn't mounted in the init container "expand-encrypted-bluefs".
This will manifest like so:
```
Underlying device for crypt device ocs-deviceset-block-dev-from-san-1-data-0-5qkmm-block-dmcrypt disappeared.
```
Signed-off-by: Sébastien Han <seb@redhat.com>
When running encrypted OSDs on multipath the expand container was
failing to find the underlying device of the encrypted block. This was
because the underlying block was located in /dev/mapper and this path
wasn't mounted in the init container "expand-encrypted-bluefs".
This will manifest like so:
```
Underlying device for crypt device ocs-deviceset-block-dev-from-san-1-data-0-5qkmm-block-dmcrypt disappeared.
```
Signed-off-by: Sébastien Han <seb@redhat.com>
Once we are done opening the block device, we should remove the key even
though it is mounted on a tmpfs.
Signed-off-by: Sébastien Han <seb@redhat.com>
Other resources such as the service monitor need to match
the labels on the services to the active mgr, thus the active
mgr name is now set on the labels for the metrics service
and dashboard service.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
In the case of PVC,
We are giving lower priority to all placement.
We want deviceSet placement to applied and
override in case of overlapping settings and
we are merging nodeAffinity if applied in both
all placement and deviceSet.
In case of non-PVC,
we apply spec.placement
Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
The integration tests have long been painful to maintain with
settings in various places and copied to multiple types,
inconsistent variable names, and otherwise difficult to maintain
code. Now the settings for a test suite are all in one place and
they remain in the same settings type throughout the test.
The multi-cluster suite is also refactored to use the same install
and uninstall helpers as the other suites.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If the object store status is not yet initialized, the object store user should
fail its initialization and requeue the reconcile.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For testing purposes, we need to configure the flex driver
and discovery daemon with the operator settings configmap.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
All clusters are configured with bluestore as the store type
therefore we can remove the settings where mentioned in the
docs and dead code.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
deletion of cluster is stuck when cluster is not configured. The PR skips the cleanup and removes the finalizers if the cluster was not configured, that is, if clusterInfo could not be loaded.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
It's better to validate ownerReferences when setting them. In addition, we should use
controllerrutil.Set{Controller,Owner}Reference, that have such validation, as possible.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:
* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`
will be autogenerated for us.
We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.
It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:
```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```
The corresponding API type markers are:
```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```
For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html
Signed-off-by: Sébastien Han <seb@redhat.com>
This is a follow up to:
https://github.com/rook/rook/pull/7264
which added pool quotas. It was stated that it would be nice to
be able to set pool max_bytes quotas using standard k8s quantity
formats rather than an integer representing the number of bytes.
I was too busy at the time to make the changes. This PR adds the
functionality.
Signed-off-by: Frank Ritchie <12985912+fritchie@users.noreply.github.com>
As a recommendation from the core team we should turn on PG auto repair
when running on Bluestore OSDs. The Quincy release will enable this by
default but in the meantime let's enable it here.
Signed-off-by: Sébastien Han <seb@redhat.com>
Sébastien Han
Santosh Pillai
Blaine Gardner
Blaine Gardner
subhamkrai
parth-grandTareq Sharafy
Satoru Takeuchi
Travis Nielsen
Frank Ritchie