Merge pull request #13261 from subhamkrai/remove-controller-runtime

core: remove webhook & controller-runtime from apis
This commit is contained in:
Blaine Gardner authored and GitHub committed 2023-12-01 09:55:34 -07:00
commit f4d67fd4db
53 files changed
+202 -1698

No files matched your search

+1 -2
View File
@@ -39,8 +39,7 @@
"subvolumegroup",
"namespace",
"test",
"tests",
"webhook"
"tests"
]
],
"body-leading-blank": [
-5
View File
@@ -11461,11 +11461,6 @@ int64
</tr>
</tbody>
</table>
<h3 id="ceph.rook.io/v1.StatusConditionGetter">StatusConditionGetter
</h3>
<div>
<p>A StatusConditionGetter allows getting a pointer to an object&rsquo;s conditions.</p>
</div>
<h3 id="ceph.rook.io/v1.StorageClassDeviceSet">StorageClassDeviceSet
</h3>
<p>
@@ -36,16 +36,6 @@ vdb
If the `FSTYPE` field is not empty, there is a filesystem on top of the corresponding device. In this example, `vdb` is available to Rook, while `vda` and its partitions have a filesystem and are not available.
## Admission Controller
Enabling the Rook admission controller is recommended to provide an additional level of validation that Rook is configured correctly with the custom resource (CR) settings. An admission controller intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized.
To deploy the Rook admission controllers, install the cert manager before Rook is installed:
```console
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.11.1/cert-manager.yaml
```
## LVM package
Ceph OSDs have a dependency on LVM in the following scenarios:
@@ -47,7 +47,6 @@ The following table lists the configurable parameters of the rook-operator chart
| Parameter | Description | Default |
|-----------|-------------|---------|
| `admissionController` | Set tolerations and nodeAffinity [^1] for admission controller pod. The admission controller would be best to start on the same nodes as other ceph daemons. | `nil` |
| `allowLoopDevices` | If true, loop devices are allowed to be used for osds in test clusters | `false` |
| `annotations` | Pod annotations | `{}` |
| `cephCommandsTimeoutSeconds` | The timeout for ceph commands in seconds | `"15"` |
@@ -125,7 +124,6 @@ The following table lists the configurable parameters of the rook-operator chart
| `csi.topology.domainLabels` | domainLabels define which node labels to use as domains for CSI nodeplugins to advertise their domains | `nil` |
| `csi.topology.enabled` | Enable topology based provisioning | `false` |
| `currentNamespaceOnly` | Whether the operator should watch cluster CRD in its own namespace or not | `false` |
| `disableAdmissionController` | Whether to disable the admission controller | `true` |
| `disableDeviceHotplug` | Disable automatic orchestration when new devices are discovered. | `false` |
| `discover.nodeAffinity` | The node labels for affinity of `discover-agent` [^1] | `nil` |
| `discover.podLabels` | Labels to add to the discover pods | `nil` |
@@ -40,13 +40,13 @@ the CRs to their prior state without even necessarily suffering cluster downtime
In the following commands, the affected `CephCluster` resource is called `rook-ceph`. If yours is named differently, the
commands will need to be adjusted.
1. Scale down the operator.
1. Scale down the operator.
```console
kubectl -n rook-ceph scale --replicas=0 deploy/rook-ceph-operator
```
2. Backup all Rook CRs and critical metadata
2. Backup all Rook CRs and critical metadata
```console
# Store the `CephCluster` CR settings. Also, save other Rook CRs that are in terminating state.
@@ -57,18 +57,10 @@ the CRs to their prior state without even necessarily suffering cluster downtime
kubectl -n rook-ceph get configmap -o yaml > configmaps.yaml
```
3. (Optional, if webhook is enabled)
Delete the `ValidatingWebhookConfiguration`. This is the resource which connects Rook custom resources
to the operator pod's validating webhook. Because the operator is unavailable, we must temporarily disable
the valdiating webhook in order to make changes.
3. Remove the owner references from all critical Rook resources that were referencing the `CephCluster` CR.
```console
kubectl -n rook-ceph delete ValidatingWebhookConfiguration rook-ceph-webhook
```
1. Programmatically determine all such resources, using this command:
4. Remove the owner references from all critical Rook resources that were referencing the `CephCluster` CR.
1. Programmatically determine all such resources, using this command:
```console
# Determine the `CephCluster` UID
ROOK_UID=$(kubectl -n rook-ceph get cephcluster rook-ceph -o 'jsonpath={.metadata.uid}')
@@ -78,7 +70,7 @@ the CRs to their prior state without even necessarily suffering cluster downtime
kubectl -n rook-ceph get $RESOURCES
```
2. **Verify that all critical resources are shown in the output.** The critical resources are these:
2. **Verify that all critical resources are shown in the output.** The critical resources are these:
- Secrets: `rook-ceph-admin-keyring`, `rook-ceph-config`, `rook-ceph-mon`, `rook-ceph-mons-keyring`
- ConfigMap: `rook-ceph-mon-endpoints`
@@ -86,10 +78,11 @@ the CRs to their prior state without even necessarily suffering cluster downtime
- Deployments: `rook-ceph-mon-*`, `rook-ceph-osd-*`, `rook-ceph-mgr-*`
- PVCs (if applicable): `rook-ceph-mon-*` and the OSD PVCs (named `<deviceset>-*`, for example `set1-data-*`)
3. For each listed resource, remove the `ownerReferences` metadata field, in order to unlink it from the deleting `CephCluster`
3. For each listed resource, remove the `ownerReferences` metadata field, in order to unlink it from the deleting `CephCluster`
CR.
To do so programmatically, use the command:
```console
for resource in $(kubectl -n rook-ceph get $RESOURCES -o name); do
kubectl -n rook-ceph patch $resource -p '{"metadata": {"ownerReferences":null}}'
@@ -97,6 +90,7 @@ the CRs to their prior state without even necessarily suffering cluster downtime
```
For a manual alternative, issue `kubectl edit` on each resource, and remove the block matching:
```yaml
ownerReferences:
- apiVersion: ceph.rook.io/v1
@@ -107,11 +101,10 @@ the CRs to their prior state without even necessarily suffering cluster downtime
uid: <uid>
```
5. **Before completing this step, validate these things. Failing to do so could result in data loss.**
1. Confirm that `cluster.yaml` contains the `CephCluster` CR.
2. Confirm all critical resources listed above have had the `ownerReference` to the `CephCluster` CR removed.
4. **Before completing this step, validate these things. Failing to do so could result in data loss.**
1. Confirm that `cluster.yaml` contains the `CephCluster` CR.
2. Confirm all critical resources listed above have had the `ownerReference` to the `CephCluster` CR removed.
Remove the finalizer from the `CephCluster` resource. This will cause the resource to be immediately deleted by Kubernetes.
@@ -122,33 +115,32 @@ the CRs to their prior state without even necessarily suffering cluster downtime
After the finalizer is removed, the `CephCluster` will be immediately deleted. If all owner references were properly removed,
all ceph daemons will continue running and there will be no downtime.
6. Create the `CephCluster` CR with the same settings as previously
5. Create the `CephCluster` CR with the same settings as previously
```console
# Use the same cluster settings as exported in step 2.
kubectl create -f cluster.yaml
```
7. If there are other CRs in terminating state such as CephBlockPools, CephObjectStores, or CephFilesystems,
6. If there are other CRs in terminating state such as CephBlockPools, CephObjectStores, or CephFilesystems,
follow the above steps as well for those CRs:
1. Backup the CR
2. Remove the finalizer and confirm the CR is deleted (the underlying Ceph resources will be preserved)
3. Create the CR again
1. Backup the CR
2. Remove the finalizer and confirm the CR is deleted (the underlying Ceph resources will be preserved)
3. Create the CR again
8. Scale up the operator
7. Scale up the operator
```console
kubectl -n rook-ceph scale --replicas=1 deploy/rook-ceph-operator
```
9. Watch the operator log to confirm that the reconcile completes successfully.
8. Watch the operator log to confirm that the reconcile completes successfully.
```console
kubectl -n rook-ceph logs -f deployment/rook-ceph-operator
```
## Adopt an existing Rook Ceph cluster into a new Kubernetes cluster
Situations this section can help resolve:
@@ -304,14 +296,17 @@ type: kubernetes.io/rook
```
The values for the secret can be found in `$dataDirHostPath/rook-ceph/client.admin.keyring` and `$dataDirHostPath/rook-ceph/rook-ceph.config`.
- `ceph-secret` and `mon-secret` are to be filled with the `client.admin`'s keyring contents.
- `ceph-username`: set to the string `client.admin`
- `fsid`: set to the original ceph cluster id.
All the fields in data section need to be encoded in base64. Coding could be done like this:
```console
echo -n "string to code" | base64 -i -
```
Now save the secret as `rook-ceph-mon.yaml`, to be created later in the restore.
The second resource is the configmap named rook-ceph-mon-endpoints as seen in this example below:
+6
View File
@@ -46,6 +46,12 @@ those releases.
official releases. Builds from the master branch can have functionality changed or removed at any
time without compatibility support and without prior notice.
## Breaking changes in v1.13
* The minimum supported version of Kubernetes is v1.23.
* Support for the admission controller/webhooks has been removed. If admission controller/webhooks is enabled, disable by changing
`ROOK_DISABLE_ADMISSION_CONTROLLER: "true"` in operator.yaml before upgrading to rook v1.13. CRD validation is now enabled with [Common Expression Language](https://kubernetes.io/docs/reference/using-api/cel/). This requires Kubernetes version 1.25 or higher.
## Breaking changes in v1.12
* The minimum supported version of Kubernetes is v1.22.
+2
View File
@@ -3,5 +3,7 @@
## Breaking Changes
- Removed support for Ceph Pacific (v16)
- Support for the admission controller/webhooks has been removed. If admission controller/webhooks is enabled, disable by changing
`ROOK_DISABLE_ADMISSION_CONTROLLER: "true"` in operator.yaml before upgrading to rook v1.13. CRD validation is now enabled with [Common Expression Language](https://kubernetes.io/docs/reference/using-api/cel/). This requires Kubernetes version 1.25 or higher.
## Features
@@ -18,9 +18,6 @@ rules:
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
- apiGroups: ["admissionregistration.k8s.io"]
resources: ["validatingwebhookconfigurations"]
verbs: ["create", "get", "delete", "update"]
- apiGroups: ["csiaddons.openshift.io"]
resources: ["networkfences"]
verbs: ["create", "get", "update", "delete", "watch", "list"]
@@ -81,7 +78,7 @@ rules:
- nodes/proxy
- services
# Rook watches secrets which it uses to configure access to external resources.
# e.g., external Ceph cluster; TLS certificates for the admission controller or object store
# e.g., external Ceph cluster or object store
- secrets
# Rook watches for changes to the rook-operator-config configmap
- configmaps
@@ -10,7 +10,6 @@ data:
ROOK_CEPH_COMMANDS_TIMEOUT_SECONDS: {{ .Values.cephCommandsTimeoutSeconds | quote }}
ROOK_OBC_WATCH_OPERATOR_NAMESPACE: {{ .Values.enableOBCWatchOperatorNamespace | quote }}
ROOK_CEPH_ALLOW_LOOP_DEVICES: {{ .Values.allowLoopDevices | quote }}
ROOK_DISABLE_ADMISSION_CONTROLLER: {{ .Values.disableAdmissionController | quote }}
ROOK_ENABLE_DISCOVERY_DAEMON: {{ .Values.enableDiscoveryDaemon | quote }}
{{- if .Values.discoverDaemonUdev }}
DISCOVER_DAEMON_UDEV_BLACKLIST: {{ .Values.discoverDaemonUdev | quote }}
@@ -38,12 +38,6 @@ spec:
name: rook-config
- mountPath: /etc/ceph
name: default-config-dir
- mountPath: /etc/webhook
name: webhook-cert
ports:
- containerPort: 9443
name: https-webhook
protocol: TCP
env:
- name: ROOK_CURRENT_NAMESPACE_ONLY
value: {{ .Values.currentNamespaceOnly | quote }}
@@ -123,5 +117,3 @@ spec:
emptyDir: {}
- name: default-config-dir
emptyDir: {}
- name: webhook-cert
emptyDir: {}
-14
View File
@@ -619,9 +619,6 @@ discover:
# cpu: 100m
# memory: 128Mi
# -- Whether to disable the admission controller
disableAdmissionController: true
# -- Runs Ceph Pods as privileged to be able to write to `hostPaths` in OpenShift with SELinux restrictions.
hostpathRequiresPrivileged: false
@@ -638,17 +635,6 @@ imagePullSecrets:
# -- Whether the OBC provisioner should watch on the operator namespace or not, if not the namespace of the cluster will be used
enableOBCWatchOperatorNamespace: true
# -- Set tolerations and nodeAffinity [^1] for admission controller pod.
# The admission controller would be best to start on the same nodes as other ceph daemons.
admissionController:
# tolerations:
# - key: key
# operator: Exists
# effect: NoSchedule
# nodeAffinity: key1=value1,value2; key2=value3
# [^1]: `nodeAffinity` and `*NodeAffinity` options should have the format `"role=storage,rook; storage=ceph"` or `storage=;role=rook-example` or `storage=;` (_checks only for presence of key_)
monitoring:
# -- Enable monitoring. Requires Prometheus to be pre-installed.
# Enabling will also create RBAC rules to allow Operator to create ServiceMonitors
+1 -4
View File
@@ -233,7 +233,7 @@ rules:
- nodes/proxy
- services
# Rook watches secrets which it uses to configure access to external resources.
# e.g., external Ceph cluster; TLS certificates for the admission controller or object store
# e.g., external Ceph cluster or object store
- secrets
# Rook watches for changes to the rook-operator-config configmap
- configmaps
@@ -563,9 +563,6 @@ rules:
- apiGroups: [""]
resources: ["pods/exec"]
verbs: ["create"]
- apiGroups: ["admissionregistration.k8s.io"]
resources: ["validatingwebhookconfigurations"]
verbs: ["create", "get", "delete", "update"]
- apiGroups: ["csiaddons.openshift.io"]
resources: ["networkfences"]
verbs: ["create", "get", "update", "delete", "watch", "list"]
-10
View File
@@ -560,8 +560,6 @@ data:
# Set to "ms_mode=secure" when connections.encrypted is enabled in CephCluster CR
# CSI_CEPHFS_KERNEL_MOUNT_OPTIONS: "ms_mode=secure"
ROOK_DISABLE_ADMISSION_CONTROLLER: "true"
# Whether the OBC provisioner should watch on the operator namespace or not, if not the namespace of the cluster will be used
ROOK_OBC_WATCH_OPERATOR_NAMESPACE: "true"
@@ -680,12 +678,6 @@ spec:
name: rook-config
- mountPath: /etc/ceph
name: default-config-dir
- mountPath: /etc/webhook
name: webhook-cert
ports:
- containerPort: 9443
name: https-webhook
protocol: TCP
env:
- name: ROOK_CURRENT_NAMESPACE_ONLY
value: "false"
@@ -765,6 +757,4 @@ spec:
emptyDir: {}
- name: default-config-dir
emptyDir: {}
- name: webhook-cert
emptyDir: {}
# OLM: END OPERATOR DEPLOYMENT
-10
View File
@@ -503,8 +503,6 @@ data:
# The CSI GRPC timeout value (in seconds). It should be >= 120. If this variable is not set or is an invalid value, it's default to 150.
CSI_GRPC_TIMEOUT_SECONDS: "150"
ROOK_DISABLE_ADMISSION_CONTROLLER: "true"
# Enable topology based provisioning.
CSI_ENABLE_TOPOLOGY: "false"
# Domain labels define which node labels to use as domains
@@ -608,12 +606,6 @@ spec:
name: rook-config
- mountPath: /etc/ceph
name: default-config-dir
- mountPath: /etc/webhook
name: webhook-cert
ports:
- containerPort: 9443
name: https-webhook
protocol: TCP
env:
# If the operator should only watch for cluster CRDs in the same namespace, set this to "true".
# If this is not set to true, the operator will watch for cluster CRDs in all namespaces.
@@ -682,6 +674,4 @@ spec:
emptyDir: {}
- name: default-config-dir
emptyDir: {}
- name: webhook-cert
emptyDir: {}
# OLM: END OPERATOR DEPLOYMENT
+25 -19
View File
@@ -1,15 +1,17 @@
# Admission Controllers in Rook
**The admission controller has been removed in favor of the declarative validation admission policies**
# Background
This proposal is to add support for admission controllers in Rook. An admission controller is a piece of code that intercepts requests to the Kubernetes API server prior to persistence of the object, but after the request is authenticated and authorized
There are two special controllers: MutatingAdmissionWebhook and ValidatingAdmissionWebhook.
Mutating controllers may modify the objects they admit but validation controllers are only allowed to validate requests.
Currently, user can manipulate Custom Resource specs with any values which may result in Rook not functioning as expected. The present validation method in Kubernetes is the OpenAPI schema validation which can be used for basic validations like checking type of data, providing a range for the values etc but anything more complex (checking resource availability, network status, error handling) would not be possible under this scenario.
# Admission Controllers Concept
![AC Image](ac_design.png)
@@ -19,20 +21,21 @@ Currently, user can manipulate Custom Resource specs with any values which may r
1. Webhook server which will validate the requests
2. TLS certificates for the server,
3. ValidatingWebhookConfig/MutatingWebhookConfig which will intercept requests and send a HTTPS request to the webhook server.
4. RBAC Components
4. RBAC Components
## Implementation
## Implementation
As shown in the above diagram, the admission control process proceeds in two phases. In the first phase, mutating admission controllers are run. In the second phase, validating admission controllers are run. Note again that some of the controllers are both.
The admission controllers intercept requests based on the values given in the configuration. In this config, we have to provide the details on
The admission controllers intercept requests based on the values given in the configuration. In this config, we have to provide the details on
* What resources should it be looking for ? (Pods, Service)
* What api version and group does it belong to ? *Example : ApiVersion = (v1, v1beta) Group version = (rook.ceph.io, admissionregistration.k8s.io)*
* What kind of operation should it intercept ? (Create, Update, Delete)
* A valid base64 encoded CA bundle.
* What path do we want to send with HTTPs request (/validate, /mutate)
A webhook server should be in place (*with valid TLS certificates*) to intercept any HTTPs request that comes with the above path value. Once the request is intercepted by the server, an [AdmissionRequest](https://github.com/kubernetes/api/blob/master/admission/v1beta1/types.go#L40) object is sent through with the resource specifications.
A webhook server should be in place (*with valid TLS certificates*) to intercept any HTTPs request that comes with the above path value. Once the request is intercepted by the server, an [AdmissionRequest](https://github.com/kubernetes/api/blob/master/admission/v1beta1/types.go#L40) object is sent through with the resource specifications.
When the webhook server receives Admission Request, it will perform predefined validations on the provided resource values and send back an [AdmissionResponse](https://github.com/kubernetes/api/blob/master/admission/v1beta1/types.go#L116) with the indication whether request is accepted or rejected.
@@ -40,26 +43,28 @@ If any of the controllers in either phase reject the request, the entire request
## Certificate Management
### Development and Testing
### Development and Testing
![Dev_AC_Image](./ac_dev_design.png)
We can use self-signed certificates approved by the Kubernetes Certificate Authority for development purposes.This can be done by following the steps given below.
* Creating the private key and certs using openssl.
**Sample : Generating the public and private keys**
**Sample : Generating the public and private keys**
```
openssl genrsa -out ${PRIVATE_KEY_NAME}.pem 2048
openssl req -new -key ${PRIVATE_KEY_NAME}.pem -subj "/CN=${service}.${namespace}.svc"
#Created after CSR is approved
openssl base64 -d -A -out ${PUBLIC_KEY_NAME}.pem
openssl base64 -d -A -out ${PUBLIC_KEY_NAME}.pem
```
* Creating and sending a CSR to kubernetes for approval
**Sample : Certificate Signing Request (CSR) in Kubernetes**
```
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
@@ -72,8 +77,9 @@ spec:
- key encipherment
- server auth
```
Verify with
` kubectl get csr ${csrName}`
`kubectl get csr ${csrName}`
**Sample : Approval of Signed Certificate**
@@ -81,7 +87,7 @@ Verify with
kubectl certificate approve ${csrName}
```
If it is approved, then we can check the certificate with following command
If it is approved, then we can check the certificate with following command
```
$(kubectl get csr ${csrName} -o jsonpath='{.status.certificate}')
@@ -90,18 +96,19 @@ $(kubectl get csr ${csrName} -o jsonpath='{.status.certificate}')
* Once approved, a generic secret will be created with the given public and private key which will be later mounted onto the server pod for use.
**Sample : Creating a Secret in Kubernetes**
```
kubectl create secret generic ${secret} \
--from-file=key.pem=${PRIVATE_KEY_NAME}.pem \
--from-file=cert.pem=${PUBLIC_KEY_NAME}.pem
```
* Modifying the webhook config to inject CA bundle onto the ValidatingWebhookConfig
* All the above resources will be created in rook-ceph namespace
Using the above approach, the dev/admins will have the responsibility of rotating the certificates when they expire.
Below is an excerpt of what a ValidatingWebhookConfig looks like
Below is an excerpt of what a ValidatingWebhookConfig looks like
```
apiVersion: admissionregistration.k8s.io/v1
@@ -129,7 +136,7 @@ We can make changes to the above values according to intercept based on whether
### Starting the Webhook Server
Based on the whether the secrets are present, the rook operator will deploy the relevant configuration files onto the cluster and start the server.
Based on the whether the secrets are present, the rook operator will deploy the relevant configuration files onto the cluster and start the server.
The secrets will be volume mounted on the rook operator pod dynamically when they are detected. After the volumes are mounted, an HTTPs server would be started.
@@ -140,9 +147,8 @@ Once the server starts, it will look for the appropriate tls key and crt files i
If the server is unable to find valid certificates, It will not deploy any admission controller components onto the cluster and hence rook will continue to function normally as before.
# References
1. https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/
1. https://github.com/kubernetes/api/blob/master/admission/v1beta1/types.go
1. https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/
1. https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
1. <https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/>
1. <https://github.com/kubernetes/api/blob/master/admission/v1beta1/types.go>
1. <https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/>
1. <https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/>
+43 -8
View File
@@ -1,7 +1,9 @@
# Resource dependency relationships
**Targeted for v1.7**
## Goal
Custom Resources in Rook-Ceph can act as data/storage "providers" for other Resources in the
Kubernetes cluster which may be "dependent" on the data/storage provided to them. This dependency
relationship is not codified in Rook, and it is possible for Custom Resources which have dependents
@@ -40,9 +42,10 @@ in the Kubernetes cluster.
1. Block deletion of object "multisite" resources with their hierarchy:
CephObjectRealm -> CephObjectZoneGroup -> CephObjectZone -> CephObjectStore
## Inventory of resources that could have dependency relationships
All Rook-Ceph CRDs:
- CephCluster
- CephBlockPool
- CephFilesystem
@@ -57,34 +60,40 @@ All Rook-Ceph CRDs:
- CephObjectZone
Resources for which Rook-Ceph acts as a driver/provisioner:
- ObjectBucket (lib-bucket-provisioner)
- Bucket (COSI)
Kubernetes resources which can depend on Rook:
- StorageClass (Kubernetes)
- PersistentVolume (Kubernetes)
## Proposed dependency relationships
A graph of proposed dependency relationships is shown below with more detail to follow.
![Graph view of resource dependencies](resource-dependencies.png)
#### `CephCluster`
A CephCluster does not create pools itself, but the Ceph cluster it represents houses pools, and
users can manually create pools using Ceph tooling outside of Kubernetes manifests. It is useful but
not critical to understand which resources interact with pools and in what ways.
Dependents which can create/delete pools:
- CephBlockPools in the same namespace
- CephFilesystems in the same namespace
- CephObjectStores in the same namespace
- CephObjectZones in the same namespace
Dependents which can consume arbitrary pools including user-created pools:
- CephNFSes in the same namespace
- CephClients in the same namespace
Dependents that do not interact with pools:
- CephRBDMirrors in the same namespace
- CephFilesystemMirrors in the same namespace
- CephObjectRealms in the same namespace
@@ -95,11 +104,14 @@ dependents. If a dependent exists in the same namespace, block deletion. In this
is the most protected provider resource. It also acts as a root for preserving deletion ordering.
#### `CephBlockPool`
Dependents which can consume this provider's pools:
- CephNFSes in the same namespace that have `spec.pool == <a provider pool>`
- CephClients in the same namespace that have any `spec.caps` value with the string `pool=<a provider pool>`
Dependents via CSI:
- StorageClasses that have:
- `provisioner == <operator namespace>.rbd.csi.ceph.com` AND
- `parameters.clusterID == <ceph cluster namespace>` AND
@@ -113,11 +125,14 @@ Dependents via CSI:
- NOTE: dependents should continue to be ignored if `spec.cleanupPolicy.allowUninstallWithVolumes == true`
#### `CephFilesystem`
Dependents which can consume this provider's pools:
- CephNFSes in the same namespace that have `spec.pool == <a provider pool>`
- CephClients in the same namespace that have any `spec.caps` value with the string `pool=<a provider pool>`
Dependents via CSI:
- StorageClasses that have:
- `provisioner == <operator namespace>.cephfs.csi.ceph.com` AND
- `parameters.clusterID == <ceph cluster namespace>` AND
@@ -131,49 +146,63 @@ Dependents via CSI:
- NOTE: dependents should continue to be ignored if `spec.cleanupPolicy.allowUninstallWithVolumes == true`
#### `CephObjectStore`
Dependents which can consume this provider's pools:
- CephNFSes in the same namespace that have `spec.pool == <a provider pool>`
- CephClients in the same namespace that have any `spec.caps` value with the string `pool=<a provider pool>`
Dependents which reference this provider by name:
- CephObjectStoreUsers in the same namespace that have `spec.store == <CephObjectStore.metadata.name>`
Dependents via lib-bucket-provisioner:
- ObjectBucketClaims that have:
- `spec.endpoint.bucketHost == <provider's service name>.<ceph cluster namespace>.svc` AND
- `spec.endpoint.bucketName == <a provider bucket>`
Dependents via COSI:
- Buckets with a reference to a provider's service and bucket (specific paths TBD by COSI design)
#### `CephObjectZone`, `CephObjectZoneGroup`, and `CephObjectRealm`
These resources are all part of Rook-Ceph's multi-site object storage strategy.
`CephObjectZone` creates pools. A zone can be effectively thought of as the "object store" itself.
Dependents which can consume this provider's pools:
- CephObjectStores in the same namespace that have `spec.zone.name == CephObjectZone.metadata.name`
- CephNFS in the same namespace that have `spec.pool = <a provider pool>`
- CephClient in the same namespace that has an `spec.caps` value with the string `pool=<a provider pool>`
`CephObjectRealm` has dependents:
- CephObjectZoneGroups in the same namespace that have `spec.realm == CephObjectRealm.metadata.name`
`CephObjectZoneGroup` has dependents:
- CephObjectZones in the same namespace that have `spec.zoneGroup == CephObjectZoneGroup.metadata.name`
#### Mirroring resources
CephRBDMirror has dependents:
- CephBlockPools in the same namespace that have `spec.mirroring.enabled == true`
CephFilesystemMirror has dependents:
- CephFilesystems in the same namespace that have `spec.mirroring.enabled == true`
## Analysis and discussion
### Reusable patterns
We can identify some common metrics for determining whether a resource is a dependent of a given
"provider" resource. Not all metrics are always applicable, but each these metrics appear more than
once. It should be possible to design reusable patterns/methods for reusing logic.
- Is a dependent in the same namespace as the provider?
- Does a dependent reference the current Rook-Ceph operator?
- Does a dependent reference the Ceph cluster namespace of the provider?
@@ -181,9 +210,11 @@ once. It should be possible to design reusable patterns/methods for reusing logi
- Does the dependent reference the provider by name? (e.g., CephObjectZone references CephObjectZoneGroup)
### User feedback
It will be important for the user to understand why resources are not being deleted if Rook is
blocking the deletion. This design proposes that the Rook operator report to the user when it is
blocking deletion of a resource due to dependents in two ways:
- report it in a status condition of the resource object being deleted
- report a Kubernetes Event ([reference](https://kubernetes.io/blog/2018/01/reporting-errors-using-kubernetes-events/))
@@ -191,6 +222,7 @@ Using both of these methods will maximize user visibility.
**Status:**
Reported statuses will be modified as follows:
1. Object's status.phase should be changed to "Deleting" as soon as a deletion timestamp is detected
and never changed
1. A status.condition should be added if the operator is blocking deletion:
@@ -203,6 +235,7 @@ Reported statuses will be modified as follows:
**Event:**
Reported events will have the following content:
- Type: "Warning"
- Reason: "ReconcileFailed"
- Message: "object deletion is blocked because it has dependents:" followed by a full list of which
@@ -210,6 +243,7 @@ Reported events will have the following content:
of which Ceph kinds (e.g., pools or buckets).
### Detecting PersistentVolume dependents
Rook currently inspects Kubernetes PersistentVolume (PV) resources when deleting CephClusters. This
provides protection from deleting the backing Ceph cluster when user applications are using it.
@@ -229,6 +263,7 @@ checking when the CephCluster is deleted. Detecting the existence of PVs when th
deleted then becomes redundant and should be removed.
### Detecting StorageClass dependents
As a note, StorageClasses (SCs) are only used during initial creation of a
PersistentVolume based on the StorageClass.
@@ -245,6 +280,7 @@ compelling use-case for treating StorageClasses as dependencies, this functional
implemented at that time.
### Detecting object bucket dependents without querying lib-bucket-provisioner or COSI resources
Detecting lib-bucket-provisioner ObjectBuckets that are dependent on a given CephObjectStore
requires inspecting ObjectBuckets for a reference to a bucket found in the object store as well as a
reference to the address of the Kubernetes Service created for access to RGWs
@@ -286,10 +322,12 @@ developers should revisit the decision at a later date to discuss whether the st
to adequately meet users' needs and whether the drawbacks noted are causing any issues.
### Implementation in stages
This design will result in changes to every major Rook-Ceph controller. However, it should be quite
easy to tackle these changes in stages so that changes can be more easily implemented and reviewed.
Stages by priority:
1. Block deletion of a CephCluster when there are other Rook-Ceph resources in the same namespace
1. Block deletion of a CephObjectStore when there is a user bucket present in the store.
- This already has an implementation, but it blocks for unrelated OBs
@@ -306,8 +344,5 @@ Stages by priority:
- CephClients are seldom used, and few users will be affected if this is not in place.
### Future work
Immediate feedback is always more helpful to users when possible. It should be possible to implement
dependency checks in a validating admission controller quite easily, but we don't currently know if
the controller is able to validate delete requests. This design leaves this investigation for the
future, but implementation of the design can strive to make dependency checking code easily reusable
for the admission controller when work on that might begin.
Immediate feedback is always more helpful to users when possible.
-1
View File
@@ -16,7 +16,6 @@ require (
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.4.0
github.com/hashicorp/vault/api v1.10.0
github.com/jetstack/cert-manager v1.7.3
github.com/k8snetworkplumbingwg/network-attachment-definition-client v1.4.0
github.com/kube-object-storage/lib-bucket-provisioner v0.0.0-20221122204822-d1a8c34382f1
github.com/libopenstorage/secrets v0.0.0-20231011182615-5f4b25ceede1
+2 -4
View File
@@ -556,8 +556,8 @@ github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLi
github.com/evanphx/json-patch/v5 v5.0.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4=
github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww=
github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fatih/color v1.12.0 h1:mRhaKNwANqRgUBGKmnI5ZxEk7QXmjQeCcuYFMX2bfcc=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
@@ -857,8 +857,6 @@ github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANyt
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/jetstack/cert-manager v1.7.3 h1:GMbRmyEqKf/ve0TQIXIOjbokdm805rj3uWRlifJnd6U=
github.com/jetstack/cert-manager v1.7.3/go.mod h1:xj0TPp31HE0Jub5mNOnF3Fp3XvhIsiP+tsPZVOmU/Qs=
github.com/jmespath/go-jmespath v0.4.0 h1:BEgLn5cpjn8UN1mAw4NjwDrS35OdebyEtFe+9YPoQUg=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1 h1:shLQSRRSCCPj3f2gpwzGwWFoC7ycTf1rcQZHOlsJ6N8=
@@ -1058,7 +1056,7 @@ github.com/ryanuber/columnize v2.1.0+incompatible/go.mod h1:sm1tb6uqfes/u+d4ooFo
github.com/ryanuber/go-glob v1.0.0 h1:iQh3xXAumdQ+4Ufa5b25cRpC5TYKlno6hsv6Cb3pkBk=
github.com/ryanuber/go-glob v1.0.0/go.mod h1:807d1WSdnB0XRJzKNil9Om6lcp/3a0v4qIHxIXzX/Yc=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.2.0 h1:XU+rvMAioB0UC3q1MFrIQy4Vo5/4VsRDQQXHsEya6xQ=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
-64
View File
@@ -16,19 +16,6 @@ limitations under the License.
package v1
import (
"reflect"
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
// compile-time assertions ensures CephCluster implements webhook.Validator so a webhook builder
// will be registered for the validating webhook.
var _ webhook.Validator = &CephCluster{}
// RequireMsgr2 checks if the network settings require the msgr2 protocol
func (c *ClusterSpec) RequireMsgr2() bool {
if c.Network.Connections == nil {
@@ -54,57 +41,6 @@ func (c *ClusterSpec) ZonesRequired() bool {
return c.IsStretchCluster() || len(c.Mon.Zones) > 0
}
func (c *CephCluster) ValidateCreate() (admission.Warnings, error) {
if c.Spec.ZonesRequired() {
if len(c.Spec.Mon.Zones) < c.Spec.Mon.Count {
return nil, errors.New("Not enough zones available for mons, please specify more zones")
}
}
logger.Infof("validate create cephcluster %q", c.ObjectMeta.Name)
//If external mode enabled, then check if other fields are empty
if c.Spec.External.Enable {
if !reflect.DeepEqual(c.Spec.Mon, MonSpec{}) || c.Spec.Dashboard != (DashboardSpec{}) || !reflect.DeepEqual(c.Spec.Monitoring, (MonitoringSpec{})) || c.Spec.DisruptionManagement != (DisruptionManagementSpec{}) || len(c.Spec.Mgr.Modules) > 0 || len(c.Spec.Network.Provider) > 0 || len(c.Spec.Network.Selectors) > 0 {
return nil, errors.New("invalid create : external mode enabled cannot have mon,dashboard,monitoring,network,disruptionManagement,storage fields in CR")
}
}
if err := ValidateNetworkSpec(c.GetNamespace(), c.Spec.Network); err != nil {
return nil, errors.Errorf("invalid create: %s", err)
}
return nil, nil
}
func (c *CephCluster) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Infof("validate update cephcluster %q", c.ObjectMeta.Name)
occ := old.(*CephCluster)
return validateUpdatedCephCluster(c, occ)
}
func (c *CephCluster) ValidateDelete() (admission.Warnings, error) {
return nil, nil
}
func validateUpdatedCephCluster(updatedCephCluster *CephCluster, found *CephCluster) (admission.Warnings, error) {
if updatedCephCluster.Spec.DataDirHostPath != found.Spec.DataDirHostPath {
return nil, errors.Errorf("invalid update: DataDirHostPath change from %q to %q is not allowed", found.Spec.DataDirHostPath, updatedCephCluster.Spec.DataDirHostPath)
}
if err := ValidateNetworkSpecUpdate(found.GetNamespace(), found.Spec.Network, updatedCephCluster.Spec.Network); err != nil {
return nil, errors.Errorf("invalid update: %s", err)
}
if len(updatedCephCluster.Spec.Storage.StorageClassDeviceSets) == len(found.Spec.Storage.StorageClassDeviceSets) {
for i, storageClassDeviceSet := range found.Spec.Storage.StorageClassDeviceSets {
if storageClassDeviceSet.Encrypted != updatedCephCluster.Spec.Storage.StorageClassDeviceSets[i].Encrypted {
return nil, errors.Errorf("invalid update: StorageClassDeviceSet %q encryption change from %t to %t is not allowed", storageClassDeviceSet.Name, found.Spec.Storage.StorageClassDeviceSets[i].Encrypted, storageClassDeviceSet.Encrypted)
}
}
}
return nil, nil
}
func (c *CephCluster) GetStatusConditions() *[]Condition {
return &c.Status.Conditions
}
-113
View File
@@ -1,113 +0,0 @@
/*
Copyright 2020 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"testing"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func Test_validateUpdatedCephCluster(t *testing.T) {
type args struct {
updatedCephCluster *CephCluster
found *CephCluster
}
tests := []struct {
name string
args args
wantErr bool
}{
{"everything is ok", args{&CephCluster{}, &CephCluster{}}, false},
{"good mon count", args{&CephCluster{Spec: ClusterSpec{Mon: MonSpec{Count: 1}}}, &CephCluster{}}, false},
{"even mon count", args{&CephCluster{Spec: ClusterSpec{Mon: MonSpec{Count: 2}}}, &CephCluster{}}, false},
{"good mon count", args{&CephCluster{Spec: ClusterSpec{Mon: MonSpec{Count: 3}}}, &CephCluster{}}, false},
{"changed DataDirHostPath", args{&CephCluster{Spec: ClusterSpec{DataDirHostPath: "foo"}}, &CephCluster{Spec: ClusterSpec{DataDirHostPath: "bar"}}}, true},
{"changed network provider", args{&CephCluster{Spec: ClusterSpec{Network: NetworkSpec{Provider: "foo"}}}, &CephCluster{Spec: ClusterSpec{Network: NetworkSpec{Provider: "bar"}}}}, true},
{"changed storageClassDeviceSet encryption", args{&CephCluster{Spec: ClusterSpec{Storage: StorageScopeSpec{StorageClassDeviceSets: []StorageClassDeviceSet{{Name: "foo", Encrypted: false}}}}}, &CephCluster{Spec: ClusterSpec{Storage: StorageScopeSpec{StorageClassDeviceSets: []StorageClassDeviceSet{{Name: "foo", Encrypted: true}}}}}}, true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if _, err := validateUpdatedCephCluster(tt.args.updatedCephCluster, tt.args.found); (err != nil) != tt.wantErr {
t.Errorf("validateUpdatedCephCluster() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestCephClusterValidateCreate(t *testing.T) {
c := &CephCluster{
ObjectMeta: metav1.ObjectMeta{
Name: "rook-ceph",
},
Spec: ClusterSpec{
DataDirHostPath: "/var/lib/rook",
},
}
_, err := c.ValidateCreate()
assert.NoError(t, err)
c.Spec.External.Enable = true
c.Spec.Monitoring = MonitoringSpec{
Enabled: true,
}
_, err = c.ValidateCreate()
assert.Error(t, err)
}
func TestCephClusterValidateUpdate(t *testing.T) {
c := &CephCluster{
ObjectMeta: metav1.ObjectMeta{
Name: "rook-ceph",
},
Spec: ClusterSpec{
DataDirHostPath: "/var/lib/rook",
Storage: StorageScopeSpec{
StorageClassDeviceSets: []StorageClassDeviceSet{
{
Name: "sc1",
Encrypted: true,
},
},
},
},
}
_, err := c.ValidateCreate()
assert.NoError(t, err)
// Updating the CRD specs with invalid values
uc := c.DeepCopy()
uc.Spec.DataDirHostPath = "var/rook"
uc.Spec.Storage.StorageClassDeviceSets[0].Encrypted = false
_, err = uc.ValidateUpdate(c)
assert.Error(t, err)
// reverting the to older hostPath
uc.Spec.DataDirHostPath = "/var/lib/rook"
uc.Spec.Storage.StorageClassDeviceSets = []StorageClassDeviceSet{
{
Name: "sc1",
Encrypted: true,
},
{
Name: "sc2",
},
}
_, err = uc.ValidateUpdate(c)
assert.NoError(t, err)
}
+11 -25
View File
@@ -21,8 +21,6 @@ import (
"github.com/pkg/errors"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
// KerberosEnabled returns true if Kerberos is enabled from the spec.
@@ -51,50 +49,38 @@ func (n *CephNFS) IsHostNetwork(c *ClusterSpec) bool {
return c.Network.IsHost()
}
func (n *CephNFS) ValidateCreate() (admission.Warnings, error) {
return n.Spec.Security.Validate()
}
func (n *CephNFS) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
return n.ValidateCreate()
}
func (n *CephNFS) ValidateDelete() error {
return nil
}
func (sec *NFSSecuritySpec) Validate() (admission.Warnings, error) {
func (sec *NFSSecuritySpec) Validate() error {
if sec == nil {
return nil, nil
return nil
}
if sec.SSSD != nil {
sidecar := sec.SSSD.Sidecar
if sidecar == nil {
return nil, errors.New("System Security Services Daemon (SSSD) is enabled, but no runtime option is specified; supported: [runInSidecar]")
return errors.New("System Security Services Daemon (SSSD) is enabled, but no runtime option is specified; supported: [runInSidecar]")
}
if sidecar.Image == "" {
return nil, errors.New("System Security Services Daemon (SSSD) sidecar is enabled, but no image is specified")
return errors.New("System Security Services Daemon (SSSD) sidecar is enabled, but no image is specified")
}
if volSourceExistsAndIsEmpty(sidecar.SSSDConfigFile.VolumeSource.ToKubernetesVolumeSource()) {
return nil, errors.New("System Security Services Daemon (SSSD) sidecar is enabled with config from a VolumeSource, but no source is specified")
return errors.New("System Security Services Daemon (SSSD) sidecar is enabled with config from a VolumeSource, but no source is specified")
}
subDirs := map[string]bool{}
for _, additionalFile := range sidecar.AdditionalFiles {
subDir := additionalFile.SubPath
if subDir == "" {
return nil, errors.New("System Security Services Daemon (SSSD) sidecar is enabled with additional file having no subPath specified")
return errors.New("System Security Services Daemon (SSSD) sidecar is enabled with additional file having no subPath specified")
}
if volSourceExistsAndIsEmpty(additionalFile.VolumeSource.ToKubernetesVolumeSource()) {
return nil, errors.Errorf("System Security Services Daemon (SSSD) sidecar is enabled with additional file (subPath %q), but no source is specified", subDir)
return errors.Errorf("System Security Services Daemon (SSSD) sidecar is enabled with additional file (subPath %q), but no source is specified", subDir)
}
if _, ok := subDirs[subDir]; ok {
return nil, errors.Errorf("System Security Services Daemon (SSSD) sidecar is enabled with additional file containing duplicate subPath %q", subDir)
return errors.Errorf("System Security Services Daemon (SSSD) sidecar is enabled with additional file containing duplicate subPath %q", subDir)
}
subDirs[subDir] = true
}
@@ -103,15 +89,15 @@ func (sec *NFSSecuritySpec) Validate() (admission.Warnings, error) {
krb := sec.Kerberos
if krb != nil {
if volSourceExistsAndIsEmpty(krb.ConfigFiles.VolumeSource.ToKubernetesVolumeSource()) {
return nil, errors.New("Kerberos is enabled with config from a VolumeSource, but no source is specified")
return errors.New("Kerberos is enabled with config from a VolumeSource, but no source is specified")
}
if volSourceExistsAndIsEmpty(krb.KeytabFile.VolumeSource.ToKubernetesVolumeSource()) {
return nil, errors.New("Kerberos is enabled with keytab from a VolumeSource, but no source is specified")
return errors.New("Kerberos is enabled with keytab from a VolumeSource, but no source is specified")
}
}
return nil, nil
return nil
}
func volSourceExistsAndIsEmpty(v *v1.VolumeSource) bool {
+1 -1
View File
@@ -147,7 +147,7 @@ func TestNFSSecuritySpec_Validate(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if _, err := tt.security.Validate(); (err != nil) != tt.wantErr {
if err := tt.security.Validate(); (err != nil) != tt.wantErr {
t.Errorf("NFSSecuritySpec.Validate() error = %v, wantErr %v", err, tt.wantErr)
}
})
-54
View File
@@ -1,54 +0,0 @@
/*
Copyright 2021 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
// compile-time assertions ensures CephBucketNotification implements webhook.Validator so a webhook builder
// will be registered for the validating webhook.
var _ webhook.Validator = &CephBucketNotification{}
// ValidateNotificationSpec validate the bucket notification arguments
func ValidateNotificationSpec(n *CephBucketNotification) error {
// done in the generated code by kubebuilder
return nil
}
func (n *CephBucketNotification) ValidateCreate() (admission.Warnings, error) {
logger.Infof("validate create cephbucketnotification %v", n)
if err := ValidateNotificationSpec(n); err != nil {
return nil, err
}
return nil, nil
}
func (n *CephBucketNotification) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Infof("validate update cephbucketnotification %v", n)
if err := ValidateNotificationSpec(n); err != nil {
return nil, err
}
return nil, nil
}
func (n *CephBucketNotification) ValidateDelete() (admission.Warnings, error) {
logger.Infof("validate delete cephbucketnotification %v", n)
return nil, nil
}
@@ -1,85 +0,0 @@
/*
Copyright 2021 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"testing"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestValidateNotificationSpec(t *testing.T) {
notification := &CephBucketNotification{
ObjectMeta: metav1.ObjectMeta{
Name: "notification1",
},
Spec: BucketNotificationSpec{
Topic: "fish-topic",
Events: []BucketNotificationEvent{BucketNotificationEvent("s3:ObjectCreated:*")},
Filter: &NotificationFilterSpec{
KeyFilters: []NotificationKeyFilterRule{
{Name: "prefix", Value: "hello"},
{Name: "suffix", Value: ".txt"},
{Name: "regex", Value: "he[a-zA-Z0-9]o"},
},
MetadataFilters: []NotificationFilterRule{
{Name: "x-amz-m1", Value: "value1"},
{Name: "x-amz-v2", Value: "value2"},
{Name: "x-amz-v3", Value: "value3"},
},
TagFilters: []NotificationFilterRule{
{Name: "color", Value: "blue"},
{Name: "application", Value: "streaming"},
{Name: "city", Value: "chicago"},
},
},
},
}
t.Run("valid", func(t *testing.T) {
err := ValidateNotificationSpec(notification)
assert.NoError(t, err)
})
}
func TestPublicNotificationValidation(t *testing.T) {
notification := &CephBucketNotification{
ObjectMeta: metav1.ObjectMeta{
Name: "notification1",
},
Spec: BucketNotificationSpec{
Topic: "fish-topic",
Events: []BucketNotificationEvent{},
},
}
t.Run("create", func(t *testing.T) {
_, err := notification.ValidateCreate()
assert.NoError(t, err)
})
t.Run("update", func(t *testing.T) {
_, err := notification.ValidateUpdate(notification)
assert.NoError(t, err)
})
t.Run("delete", func(t *testing.T) {
_, err := notification.ValidateDelete()
assert.NoError(t, err)
})
}
-28
View File
@@ -18,15 +18,8 @@ package v1
import (
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
// compile-time assertions ensures CephObjectStore implements webhook.Validator so a webhook builder
// will be registered for the validating webhook.
var _ webhook.Validator = &CephObjectStore{}
const ServiceServingCertKey = "service.beta.openshift.io/serving-cert-secret-name"
// 38 is the max length of a ceph store name as total length of the resource name cannot be more than 63 characters limit
@@ -70,14 +63,6 @@ func (s *ObjectRealmSpec) IsPullRealm() bool {
return s.Pull.Endpoint != ""
}
func (o *CephObjectStore) ValidateCreate() (admission.Warnings, error) {
logger.Infof("validate create cephobjectstore %v", o)
if err := ValidateObjectSpec(o); err != nil {
return nil, err
}
return nil, nil
}
// ValidateObjectSpec validate the object store arguments
func ValidateObjectSpec(gs *CephObjectStore) error {
if gs.Name == "" {
@@ -106,19 +91,6 @@ func ValidateObjectSpec(gs *CephObjectStore) error {
return nil
}
func (o *CephObjectStore) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Info("validate update cephobjectstore")
err := ValidateObjectSpec(o)
if err != nil {
return nil, err
}
return nil, nil
}
func (o *CephObjectStore) ValidateDelete() (admission.Warnings, error) {
return nil, nil
}
func (s *ObjectStoreSpec) GetServiceServingCert() string {
if s.Gateway.Service != nil {
return s.Gateway.Service.Annotations[ServiceServingCertKey]
-49
View File
@@ -17,20 +17,9 @@ limitations under the License.
package v1
import (
"github.com/coreos/pkg/capnslog"
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
var (
webhookName = "rook-ceph-webhook"
logger = capnslog.NewPackageLogger("github.com/rook/rook", webhookName)
)
var _ webhook.Validator = &CephBlockPool{}
func (p *PoolSpec) IsReplicated() bool {
return p.Replicated.Size > 0
}
@@ -51,16 +40,6 @@ func (p *ReplicatedSpec) IsTargetRatioEnabled() bool {
return p.TargetSizeRatio != 0
}
func (p *CephBlockPool) ValidateCreate() (admission.Warnings, error) {
logger.Infof("validate create cephblockpool %v", p)
err := ValidateCephBlockPool(p)
if err != nil {
return nil, err
}
return nil, nil
}
// ValidateCephBlockPool validates specifically a CephBlockPool's spec (not just any NamedPoolSpec)
func ValidateCephBlockPool(p *CephBlockPool) error {
if p.Spec.Name == "device_health_metrics" || p.Spec.Name == ".mgr" || p.Spec.Name == ".nfs" {
@@ -112,34 +91,6 @@ func (p *CephBlockPool) ToNamedPoolSpec() NamedPoolSpec {
}
}
func (p *CephBlockPool) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Info("validate update cephblockpool")
ocbp := old.(*CephBlockPool)
err := ValidateCephBlockPool(p)
if err != nil {
return nil, err
}
if ocbp.Spec.Name != p.Spec.Name {
return nil, errors.New("invalid update: pool name cannot be changed")
}
if p.Spec.ErasureCoded.CodingChunks > 0 || p.Spec.ErasureCoded.DataChunks > 0 || p.Spec.ErasureCoded.Algorithm != "" {
if ocbp.Spec.Replicated.Size > 0 || ocbp.Spec.Replicated.TargetSizeRatio > 0 {
return nil, errors.New("invalid update: replicated field is set already in previous object. cannot be changed to use erasurecoded")
}
}
if p.Spec.Replicated.Size > 0 || p.Spec.Replicated.TargetSizeRatio > 0 {
if ocbp.Spec.ErasureCoded.CodingChunks > 0 || ocbp.Spec.ErasureCoded.DataChunks > 0 || ocbp.Spec.ErasureCoded.Algorithm != "" {
return nil, errors.New("invalid update: erasurecoded field is set already in previous object. cannot be changed to use replicated")
}
}
return nil, nil
}
func (p *CephBlockPool) ValidateDelete() (admission.Warnings, error) {
return nil, nil
}
func (p *CephBlockPool) GetStatusConditions() *[]Condition {
return &p.Status.Conditions
}
-24
View File
@@ -45,30 +45,6 @@ func TestValidatePoolSpec(t *testing.T) {
assert.Error(t, err)
}
func TestCephBlockPoolValidateUpdate(t *testing.T) {
p := &CephBlockPool{
ObjectMeta: metav1.ObjectMeta{
Name: "ec-pool",
},
Spec: NamedBlockPoolSpec{
PoolSpec: PoolSpec{
Replicated: ReplicatedSpec{RequireSafeReplicaSize: true, Size: 3},
},
},
}
up := p.DeepCopy()
up.Spec.ErasureCoded.DataChunks = 2
up.Spec.ErasureCoded.CodingChunks = 1
_, err := up.ValidateUpdate(p)
assert.Error(t, err)
// validate with different name in Spec.Name
ip := p.DeepCopy()
ip.Spec.Name = "new-ec-pool"
_, err = ip.ValidateUpdate(p)
assert.Error(t, err)
}
func TestMirroringSpec_SnapshotSchedulesEnabled(t *testing.T) {
type fields struct {
Enabled bool
@@ -1,43 +0,0 @@
/*
Copyright 2022 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
var _ webhook.Validator = &CephBlockPoolRadosNamespace{}
func (c *CephBlockPoolRadosNamespace) ValidateCreate() (admission.Warnings, error) {
return nil, nil
}
func (c *CephBlockPoolRadosNamespace) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Infof("validate update %s/%s CephBlockPoolRadosNamespace", c.Namespace, c.Name)
oprs := old.(*CephBlockPoolRadosNamespace)
if oprs.Spec.BlockPoolName != c.Spec.BlockPoolName {
return nil, errors.New("invalid update: blockpool name cannot be changed")
}
return nil, nil
}
func (c *CephBlockPoolRadosNamespace) ValidateDelete() (admission.Warnings, error) {
return nil, nil
}
@@ -1,43 +0,0 @@
/*
Copyright 2022 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"testing"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestCephBlockPoolRadosNamespaceValidateUpdate(t *testing.T) {
rn := &CephBlockPoolRadosNamespace{
ObjectMeta: metav1.ObjectMeta{
Name: "rados-name",
},
Spec: CephBlockPoolRadosNamespaceSpec{
BlockPoolName: "pool-name",
},
}
ur := rn.DeepCopy()
// validate with same pool name
_, err := ur.ValidateUpdate(rn)
assert.NoError(t, err)
// validate with different pool name
ur.Spec.BlockPoolName = "new-pool-name"
_, err = ur.ValidateUpdate(rn)
assert.Error(t, err)
}
-12
View File
@@ -20,20 +20,8 @@ import (
"time"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client"
)
// Based on code from https://github.com/kubernetes/apimachinery/blob/master/pkg/api/meta/conditions.go
// A StatusConditionGetter allows getting a pointer to an object's conditions.
type StatusConditionGetter interface {
client.Object
// GetStatusConditions returns a pointer to the object's conditions compatible with
// SetStatusCondition and FindStatusCondition.
GetStatusConditions() *[]Condition
}
// SetStatusCondition sets the corresponding condition in conditions to newCondition.
// conditions must be non-nil.
// 1. if the condition of the specified type already exists (all fields of the existing condition are updated to
@@ -1,43 +0,0 @@
/*
Copyright 2022 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
var _ webhook.Validator = &CephFilesystemSubVolumeGroup{}
func (c *CephFilesystemSubVolumeGroup) ValidateCreate() (admission.Warnings, error) {
return nil, nil
}
func (c *CephFilesystemSubVolumeGroup) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Infof("validate update %s/%s CephFilesystemSubVolumeGroup", c.Namespace, c.Name)
sg := old.(*CephFilesystemSubVolumeGroup)
if sg.Spec.FilesystemName != c.Spec.FilesystemName {
return nil, errors.New("invalid update: filesystem name cannot be changed")
}
return nil, nil
}
func (c *CephFilesystemSubVolumeGroup) ValidateDelete() (admission.Warnings, error) {
return nil, nil
}
@@ -1,43 +0,0 @@
/*
Copyright 2022 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package v1
import (
"testing"
"github.com/stretchr/testify/assert"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func TestCephFilesystemSubVolumeGroupValidateUpdate(t *testing.T) {
rn := &CephFilesystemSubVolumeGroup{
ObjectMeta: metav1.ObjectMeta{
Name: "subvol-name",
},
Spec: CephFilesystemSubVolumeGroupSpec{
FilesystemName: "filesystem-name",
},
}
ur := rn.DeepCopy()
// validate with same filesystem name
_, err := ur.ValidateUpdate(rn)
assert.NoError(t, err)
// validate with different filesystem name
ur.Spec.FilesystemName = "new-filesystem-name"
_, err = ur.ValidateUpdate(rn)
assert.Error(t, err)
}
+1 -29
View File
@@ -21,15 +21,8 @@ import (
"strings"
"github.com/pkg/errors"
"k8s.io/apimachinery/pkg/runtime"
"sigs.k8s.io/controller-runtime/pkg/webhook"
"sigs.k8s.io/controller-runtime/pkg/webhook/admission"
)
// compile-time assertions ensures CephBucketTopic implements webhook.Validator so a webhook builder
// will be registered for the validating webhook.
var _ webhook.Validator = &CephBucketTopic{}
func validateURI(uri string, expectedSchemas []string) error {
parsedURI, err := url.Parse(uri)
if err != nil {
@@ -57,7 +50,7 @@ func ValidateKafkaSpec(s *KafkaEndpointSpec) error {
}
// ValidateTopicSpec validate the bucket notification topic arguments
func ValidateTopicSpec(t *CephBucketTopic) error {
func (t *CephBucketTopic) ValidateTopicSpec() error {
hasEndpoint := false
if t.Spec.Endpoint.HTTP != nil {
hasEndpoint = true
@@ -89,24 +82,3 @@ func ValidateTopicSpec(t *CephBucketTopic) error {
}
return nil
}
func (t *CephBucketTopic) ValidateCreate() (admission.Warnings, error) {
logger.Infof("validate create cephbuckettopic %v", t)
if err := ValidateTopicSpec(t); err != nil {
return nil, err
}
return nil, nil
}
func (t *CephBucketTopic) ValidateUpdate(old runtime.Object) (admission.Warnings, error) {
logger.Info("validate update cephbuckettopic")
if err := ValidateTopicSpec(t); err != nil {
return nil, err
}
return nil, nil
}
func (t *CephBucketTopic) ValidateDelete() (admission.Warnings, error) {
logger.Info("validate delete cephbuckettopic")
return nil, nil
}
+12 -47
View File
@@ -42,22 +42,22 @@ func TestValidateHTTPTopicSpec(t *testing.T) {
}
t.Run("valid", func(t *testing.T) {
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.NoError(t, err)
})
t.Run("invalid endpoint host", func(t *testing.T) {
topic.Spec.Endpoint.HTTP.URI = "http://my server:9999"
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.Error(t, err)
})
t.Run("https host", func(t *testing.T) {
topic.Spec.Endpoint.HTTP.URI = "https://127.0.0.1:9999"
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.NoError(t, err)
})
t.Run("invalid endpoint schema", func(t *testing.T) {
topic.Spec.Endpoint.HTTP.URI = "kaboom://myserver:9999"
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.Error(t, err)
})
}
@@ -82,17 +82,17 @@ func TestValidateAMQPTopicSpec(t *testing.T) {
}
t.Run("valid", func(t *testing.T) {
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.NoError(t, err)
})
t.Run("amqps host", func(t *testing.T) {
topic.Spec.Endpoint.AMQP.URI = "amqps://myserver:9999"
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.NoError(t, err)
})
t.Run("endpoint schema mismatch", func(t *testing.T) {
topic.Spec.Endpoint.AMQP.URI = "http://myserver:9999"
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.Error(t, err)
})
}
@@ -117,12 +117,12 @@ func TestValidateKafkaTopicSpec(t *testing.T) {
}
t.Run("valid", func(t *testing.T) {
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.NoError(t, err)
})
t.Run("endpoint schema mismatch", func(t *testing.T) {
topic.Spec.Endpoint.Kafka.URI = "http://myserver:9999"
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.Error(t, err)
})
}
@@ -153,52 +153,17 @@ func TestInvalidTopicSpec(t *testing.T) {
}
t.Run("too many endpoint specs", func(t *testing.T) {
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.Error(t, err)
})
t.Run("valid", func(t *testing.T) {
topic.Spec.Endpoint.AMQP = nil
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.NoError(t, err)
})
t.Run("too few endpoint specs", func(t *testing.T) {
topic.Spec.Endpoint.Kafka = nil
err := ValidateTopicSpec(topic)
err := topic.ValidateTopicSpec()
assert.Error(t, err)
})
}
func TestPublicTopicValidation(t *testing.T) {
topic := &CephBucketTopic{
ObjectMeta: metav1.ObjectMeta{
Name: "fish-topic",
},
Spec: BucketTopicSpec{
OpaqueData: "me@email.com",
Persistent: true,
Endpoint: TopicEndpointSpec{
Kafka: &KafkaEndpointSpec{
URI: "kafka://myserver:9999",
UseSSL: true,
DisableVerifySSL: true,
AckLevel: "broker",
},
},
},
}
t.Run("create", func(t *testing.T) {
_, err := topic.ValidateCreate()
assert.NoError(t, err)
})
t.Run("update", func(t *testing.T) {
_, err := topic.ValidateUpdate(topic)
assert.NoError(t, err)
})
t.Run("delete", func(t *testing.T) {
_, err := topic.ValidateDelete()
assert.NoError(t, err)
})
}
+3 -17
View File
@@ -3,7 +3,6 @@ module github.com/rook/rook/pkg/apis
go 1.20
require (
github.com/coreos/pkg v0.0.0-20230601102743-20bbbf26f4d8
github.com/hashicorp/vault/api v1.10.0
github.com/k8snetworkplumbingwg/network-attachment-definition-client v1.4.0
github.com/kube-object-storage/lib-bucket-provisioner v0.0.0-20221122204822-d1a8c34382f1
@@ -13,19 +12,14 @@ require (
github.com/stretchr/testify v1.8.4
k8s.io/api v0.28.4
k8s.io/apimachinery v0.28.4
sigs.k8s.io/controller-runtime v0.16.3
)
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v3 v3.2.2 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/containernetworking/cni v1.1.2 // indirect
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/evanphx/json-patch/v5 v5.6.0 // indirect
github.com/fatih/color v1.12.0 // indirect
github.com/evanphx/json-patch v5.6.0+incompatible // indirect
github.com/fsnotify/fsnotify v1.6.0 // indirect
github.com/go-jose/go-jose/v3 v3.0.1 // indirect
github.com/go-logr/logr v1.2.4 // indirect
@@ -49,41 +43,33 @@ require (
github.com/hashicorp/hcl v1.0.1-vault-5 // indirect
github.com/hashicorp/vault/api/auth/approle v0.5.0 // indirect
github.com/hashicorp/vault/api/auth/kubernetes v0.5.0 // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/onsi/ginkgo/v2 v2.11.0 // indirect
github.com/onsi/gomega v1.27.10 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_golang v1.16.0 // indirect
github.com/prometheus/client_model v0.4.0 // indirect
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.11.1 // indirect
github.com/ryanuber/go-glob v1.0.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
go.uber.org/zap v1.26.0 // indirect
golang.org/x/crypto v0.14.0 // indirect
golang.org/x/exp v0.0.0-20230206171751-46f607a40771 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.12.0 // indirect
golang.org/x/sys v0.13.0 // indirect
golang.org/x/term v0.13.0 // indirect
golang.org/x/text v0.13.0 // indirect
golang.org/x/time v0.3.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/protobuf v1.31.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.28.4 // indirect
k8s.io/client-go v0.28.4 // indirect
k8s.io/klog/v2 v2.100.1 // indirect
k8s.io/kube-openapi v0.0.0-20230905202853-d090da108d2f // indirect
+4 -41
View File
@@ -76,8 +76,6 @@ github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
github.com/aws/aws-sdk-go v1.44.164/go.mod h1:aVsgQcEevwlmQ7qHE9I3h+dtQgpqhFB+i8Phjh7fkwI=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
github.com/cenkalti/backoff/v3 v3.0.0/go.mod h1:cIeZDE3IrqwwJl6VUwCN6trj1oXrTS4rc0ij+ULvLYs=
@@ -86,8 +84,6 @@ github.com/cenkalti/backoff/v3 v3.2.2/go.mod h1:cIeZDE3IrqwwJl6VUwCN6trj1oXrTS4r
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU=
@@ -103,10 +99,6 @@ github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe/go.mod h1:eXthEFrGJvWH
github.com/cncf/xds/go v0.0.0-20211011173535-cb28da3451f1/go.mod h1:eXthEFrGJvWHgFFCl3hGmgk+/aYT6PnTQLykKQRLhEs=
github.com/containernetworking/cni v1.1.2 h1:wtRGZVv7olUHMOqouPpn3cXJWpJgM6+EUl31EQbXALQ=
github.com/containernetworking/cni v1.1.2/go.mod h1:sDpYKmGVENF3s6uvMvGgldDWeG8dMxakj/u+i9ht9vw=
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf h1:iW4rZ826su+pqaw19uhpSCzhj44qo35pNgKFGqzDKkU=
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20230601102743-20bbbf26f4d8 h1:NrLmX9HDyGvQhyZdrDx89zCvPdxQ/EHCo+xGNrjNmHc=
github.com/coreos/pkg v0.0.0-20230601102743-20bbbf26f4d8/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -131,11 +123,9 @@ github.com/envoyproxy/go-control-plane v0.10.2-0.20220325020618-49ff273808a1/go.
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/evanphx/json-patch v4.12.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/evanphx/json-patch v5.6.0+incompatible h1:jBYDEEiFBPxA0v50tFdvOzQQTCvpL6mnFh5mB2/l16U=
github.com/evanphx/json-patch/v5 v5.6.0 h1:b91NhWfaz02IuVxO9faSllyAtNXHMPkC5J8sJCLunww=
github.com/evanphx/json-patch/v5 v5.6.0/go.mod h1:G79N1coSVB93tBe7j6PhzjmR3/2VvlbKOFpnXhI9Bw4=
github.com/evanphx/json-patch v5.6.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fatih/color v1.12.0 h1:mRhaKNwANqRgUBGKmnI5ZxEk7QXmjQeCcuYFMX2bfcc=
github.com/fatih/color v1.12.0/go.mod h1:ELkj/draVOlAH/xkhN6mQ50Qd0MPOk5AAr3maGEBuJM=
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/form3tech-oss/jwt-go v3.2.3+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
@@ -157,7 +147,6 @@ github.com/go-logr/logr v1.2.0/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbV
github.com/go-logr/logr v1.2.3/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ=
github.com/go-logr/logr v1.2.4/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/zapr v1.2.4 h1:QHVo+6stLbfJmYGkQ7uGHUCu5hnAFAj6mDe6Ea0SeOo=
github.com/go-openapi/jsonpointer v0.19.2/go.mod h1:3akKfEdA7DF1sugOqz1dVQHBcuDBPKZGEoHC/NkiQRg=
github.com/go-openapi/jsonpointer v0.19.3/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
github.com/go-openapi/jsonpointer v0.19.5/go.mod h1:Pl9vOtqEWErmShwVjC8pYs9cog34VGT37dQOVbmoatg=
@@ -190,7 +179,6 @@ github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfU
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20191227052852-215e87163ea7/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da h1:oI5xCqsCo564l8iNU+DwB5epxmsaqB+rhGL0m5jtYqE=
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
@@ -333,9 +321,6 @@ github.com/ianlancetaylor/demangle v0.0.0-20181102032728-5e5cf60278f6/go.mod h1:
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639/go.mod h1:aSSvb/t6k1mPoxDqO4vJh6VOCGPwU4O0C2/Eqndh1Sc=
github.com/imdario/mergo v0.3.5/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/imdario/mergo v0.3.13/go.mod h1:4lJ1jqUDcsbIECGy0RUJAXNIhg+6ocWgb1ALK2O4oXg=
github.com/imdario/mergo v0.3.16 h1:wwQJbIsHYGMUyLSPrEq1CT16AhnhNJQ51+4fdHUnCl4=
github.com/imdario/mergo v0.3.16/go.mod h1:WBLT9ZmE3lPoWsEzCh9LPo3TiwVN+ZKEjmz+hD27ysY=
github.com/jessevdk/go-flags v1.4.0/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
github.com/jmespath/go-jmespath v0.4.0/go.mod h1:T8mJZnbsbmF+m6zOOFylbeCJqk5+pHWvzYPziyZiYoo=
github.com/jmespath/go-jmespath/internal/testify v1.5.1/go.mod h1:L3OGu8Wl2/fWfCI6z80xFu9LTZmf1ZRjMHUOPmWr69U=
github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
@@ -373,7 +358,6 @@ github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJ
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
@@ -383,8 +367,6 @@ github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Ky
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/matttproud/golang_protobuf_extensions v1.0.4 h1:mmDVorXM7PCGKw94cs5zkfA9PSy5pEvNWRP0ET0TIVo=
github.com/matttproud/golang_protobuf_extensions v1.0.4/go.mod h1:BSXmuO+STAnVfrANrmjBb36TMTDstsz7MSK+HVaYKv4=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
@@ -425,6 +407,7 @@ github.com/onsi/ginkgo/v2 v2.4.0/go.mod h1:iHkDK1fKGcBoEHT5W7YBq4RFWaQulw+caOMkA
github.com/onsi/ginkgo/v2 v2.5.0/go.mod h1:Luc4sArBICYCS8THh8v3i3i5CuSZO+RaQRaJoeNwomw=
github.com/onsi/ginkgo/v2 v2.6.0/go.mod h1:63DOGlLAH8+REH8jUGdL3YpCpu7JODesutUjdENfUAc=
github.com/onsi/ginkgo/v2 v2.11.0 h1:WgqUCUt/lT6yXoQ8Wef0fsNn5cAuMK7+KT9UFRz2tcU=
github.com/onsi/ginkgo/v2 v2.11.0/go.mod h1:ZhrRA5XmEE3x3rhlzamx/JJvujdZoJ2uvgI7kR0iZvM=
github.com/onsi/gomega v0.0.0-20170829124025-dcabb60a477c/go.mod h1:C1qb7wdrVGGVU+Z6iS04AVkA3Q65CEZX59MT0QO5uiA=
github.com/onsi/gomega v1.7.1/go.mod h1:XdKZgCCFLUoM/7CFJVPcG8C1xQ1AJ0vpAezJrB7JYyY=
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
@@ -437,13 +420,13 @@ github.com/onsi/gomega v1.23.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2
github.com/onsi/gomega v1.24.0/go.mod h1:Z/NWtiqwBrwUt4/2loMmHL63EDLnYHmVbuBpDr2vQAg=
github.com/onsi/gomega v1.24.1/go.mod h1:3AOiACssS3/MajrniINInwbfOOtfZvplPzuRSmvt1jM=
github.com/onsi/gomega v1.27.10 h1:naR28SdDFlqrG6kScpT8VWpu1xWY5nJRCF3XaYyBjhI=
github.com/onsi/gomega v1.27.10/go.mod h1:RsS8tutOdbdgzbPtzzATp12yT7kM5I5aElG3evPbQ0M=
github.com/openshift/api v0.0.0-20230816181854-a7ca92db022a h1:1bylAza0mFIchRCRPVY9qy62CxJE18fpjEAUSpIA5O4=
github.com/openshift/api v0.0.0-20230816181854-a7ca92db022a/go.mod h1:yimSGmjsI+XF1mr+AKBs2//fSXIOhhetHGbMlBEfXbs=
github.com/pborman/uuid v1.2.0 h1:J7Q5mO4ysT1dv8hyrUGHb9+ooztCXu1D8MY8DZYsu3g=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
@@ -452,15 +435,7 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH
github.com/portworx/dcos-secrets v0.0.0-20180616013705-8e8ec3f66611/go.mod h1:4hklRW/4DQpLqkcXcjtNprbH2tz/sJaNtqinfPWl/LA=
github.com/portworx/kvdb v0.0.0-20200929023115-b312c7519467/go.mod h1:Q8YyrNDvPp3DVF96BDcQuaC7fAYUCuUX+l58S7OnD2M=
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/prometheus/client_golang v1.16.0 h1:yk/hx9hDbrGHovbci4BY+pRMfSuuat626eFsHb7tmT8=
github.com/prometheus/client_golang v1.16.0/go.mod h1:Zsulrv/L9oM40tJ7T815tM89lFEugiJ9HzIqaAx4LKc=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.4.0 h1:5lQXD3cAg1OXBf4Wq03gTrXHeaV0TQvGfUooCfx1yqY=
github.com/prometheus/client_model v0.4.0/go.mod h1:oMQmHW1/JoDwqLtg57MGgP/Fb1CJEYF2imWWhWtMkYU=
github.com/prometheus/common v0.44.0 h1:+5BrQJwiBB9xsMygAB3TNvpQKOwlkc25LbISbrdOOfY=
github.com/prometheus/common v0.44.0/go.mod h1:ofAIvZbQ1e/nugmZGz4/qCb9Ap1VoSTIO7x0VV9VvuY=
github.com/prometheus/procfs v0.11.1 h1:xRC8Iq1yyca5ypa9n1EZnWZkt7dwcoRPQwX/5gwaUuI=
github.com/prometheus/procfs v0.11.1/go.mod h1:eesXgaPo1q7lBpVMoMy0ZOFTth9hBn4W/y0/p/ScXhY=
github.com/rogpeppe/fastuuid v1.2.0/go.mod h1:jVj6XXZzXRy/MSR5jhDC/2q6DgLz+nrA6LYCDYWNEvQ=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.6.1/go.mod h1:xXDCJY+GAPziupqXw64V24skbSoqbTEfhy4qGm1nDQc=
@@ -508,9 +483,6 @@ go.opencensus.io v0.22.4/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.opencensus.io v0.22.5/go.mod h1:5pWMHQbX5EPX2/62yrJeAkowc+lfs/XD7Uxpq3pI6kk=
go.opencensus.io v0.23.0/go.mod h1:XItmlyltB5F7CS4xOC1DcqMoFqwtC6OG2xF7mCv7P7E=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
go.uber.org/zap v1.26.0/go.mod h1:dtElttAiwGvoJ/vj4IwHBS/gXsEu/pZ50mUIRWuG0so=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190605123033-f99c8df09eb5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
@@ -536,8 +508,6 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0
golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
golang.org/x/exp v0.0.0-20230206171751-46f607a40771 h1:xP7rWLUr1e1n2xkK5YB4LI0hPEy3LJC6Wk+D4pGlOJg=
golang.org/x/exp v0.0.0-20230206171751-46f607a40771/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc=
golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
@@ -839,8 +809,6 @@ golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8T
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20220411194840-2f41105eb62f/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20220517211312-f3a8303e98df/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=
gomodules.xyz/jsonpatch/v2 v2.4.0 h1:Ci3iUJyx9UeRx7CeFN8ARgGbkESwJK+KB9lLcWxY/Zw=
gomodules.xyz/jsonpatch/v2 v2.4.0/go.mod h1:AH3dM2RI6uoBZxn3LVrfvJ3E0/9dG4cSrbuBJT4moAY=
google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=
google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
@@ -1050,8 +1018,6 @@ k8s.io/api v0.23.5/go.mod h1:Na4XuKng8PXJ2JsploYYrivXrINeTaycCGcYgF91Xm8=
k8s.io/api v0.26.0/go.mod h1:k6HDTaIFC8yn1i6pSClSqIwLABIcLV9l5Q4EcngKnQg=
k8s.io/api v0.28.4 h1:8ZBrLjwosLl/NYgv1P7EQLqoO8MGQApnbgH8tu3BMzY=
k8s.io/api v0.28.4/go.mod h1:axWTGrY88s/5YE+JSt4uUi6NMM+gur1en2REMR7IRj0=
k8s.io/apiextensions-apiserver v0.28.4 h1:AZpKY/7wQ8n+ZYDtNHbAJBb+N4AXXJvyZx6ww6yAJvU=
k8s.io/apiextensions-apiserver v0.28.4/go.mod h1:pgQIZ1U8eJSMQcENew/0ShUTlePcSGFq6dxSxf2mwPM=
k8s.io/apimachinery v0.23.5/go.mod h1:BEuFMMBaIbcOqVIJqNZJXGFTP4W6AycEpb5+m/97hrM=
k8s.io/apimachinery v0.26.0/go.mod h1:tnPmbONNJ7ByJNz9+n9kMjNP8ON+1qoAIIC70lztu74=
k8s.io/apimachinery v0.28.4 h1:zOSJe1mc+GxuMnFzD4Z/U1wst50X28ZNsn5bhgIIao8=
@@ -1060,7 +1026,6 @@ k8s.io/client-go v0.23.5/go.mod h1:flkeinTO1CirYgzMPRWxUCnV0G4Fbu2vLhYCObnt/r4=
k8s.io/client-go v0.28.4 h1:Np5ocjlZcTrkyRJ3+T3PkXDpe4UpatQxj85+xjaD2wY=
k8s.io/client-go v0.28.4/go.mod h1:0VDZFpgoZfelyP5Wqu0/r/TRYcLYuJ2U1KEeoaPa1N4=
k8s.io/code-generator v0.20.1/go.mod h1:UsqdF+VX4PU2g46NC2JRs4gc+IfrctnwHb76RNbWHJg=
k8s.io/component-base v0.28.4 h1:c/iQLWPdUgI90O+T9TeECg8o7N3YJTiuz2sKxILYcYo=
k8s.io/gengo v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20201113003025-83324d819ded/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
k8s.io/gengo v0.0.0-20210813121822-485abfe95c7c/go.mod h1:FiNAH4ZV3gBg2Kwh89tzAEV2be7d5xI0vBa/VySYy3E=
@@ -1086,8 +1051,6 @@ k8s.io/utils v0.0.0-20230726121419-3b25d923346b/go.mod h1:OLgZIPagt7ERELqWJFomSt
rsc.io/binaryregexp v0.2.0/go.mod h1:qTv7/COck+e2FymRvadv62gMdZztPaShugOCi3I+8D8=
rsc.io/quote/v3 v3.1.0/go.mod h1:yEA65RcK8LyAZtP9Kv3t0HmxON59tX3rD+tICJqUlj0=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=
sigs.k8s.io/controller-runtime v0.16.3 h1:2TuvuokmfXvDUamSx1SuAOO3eTyye+47mJCigwG62c4=
sigs.k8s.io/controller-runtime v0.16.3/go.mod h1:j7bialYoSn142nv9sCOJmQgDXQXxnroFU4VnX/brVJ0=
sigs.k8s.io/json v0.0.0-20211020170558-c049b76a60c6/go.mod h1:p4QtZmO4uMYipTQNzagwnNoseA6OxSUutVw05NhYDRs=
sigs.k8s.io/json v0.0.0-20220713155537-f223a00ba0e2/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0=
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo=
-18
View File
@@ -82,21 +82,6 @@ func add(ctx context.Context, context *clusterd.Context, mgr manager.Manager, r
return err
}
value, err := k8sutil.GetOperatorSetting(ctx, context.Clientset, opcontroller.OperatorSettingConfigMapName, "ROOK_DISABLE_ADMISSION_CONTROLLER", "true")
if err != nil {
return err
}
if value == "false" {
// Watch for Secret (admission controller secret)
s := source.Kind(
mgr.GetCache(),
&v1.Secret{TypeMeta: metav1.TypeMeta{Kind: "Secret", APIVersion: v1.SchemeGroupVersion.String()}})
err = c.Watch(s, &handler.EnqueueRequestForObject{}, predicateController(ctx, mgr.GetClient()))
if err != nil {
return err
}
}
return nil
}
@@ -146,9 +131,6 @@ func (r *ReconcileConfig) reconcile(request reconcile.Request) (reconcile.Result
opcontroller.SetAllowLoopDevices(r.config.Parameters)
// Reconcile webhook secret
// This is done in the predicate function
logger.Infof("%s done reconciling", controllerName)
return reconcile.Result{}, nil
}
-40
View File
@@ -52,11 +52,6 @@ import (
"sigs.k8s.io/controller-runtime/pkg/cache"
"sigs.k8s.io/controller-runtime/pkg/manager"
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"
"sigs.k8s.io/controller-runtime/pkg/webhook"
)
const (
certDir = "/etc/webhook"
)
var (
@@ -66,16 +61,6 @@ var (
}
)
var (
webhookResources = []webhook.Validator{
&cephv1.CephCluster{},
&cephv1.CephBlockPool{},
&cephv1.CephObjectStore{},
&cephv1.CephBlockPoolRadosNamespace{},
&cephv1.CephFilesystemSubVolumeGroup{},
}
)
var (
// EnableMachineDisruptionBudget checks whether machine disruption budget is enabled
EnableMachineDisruptionBudget bool
@@ -165,9 +150,6 @@ func (o *Operator) startCRDManager(context context.Context, mgrErrorCh chan erro
BindAddress: "0",
},
Scheme: scheme,
WebhookServer: webhook.NewServer(webhook.Options{
CertDir: certDir,
}),
}
if o.config.NamespaceToWatch != "" {
@@ -183,28 +165,6 @@ func (o *Operator) startCRDManager(context context.Context, mgrErrorCh chan erro
return
}
// Add webhook if needed
isPresent, err := createWebhook(context, o.context)
if err != nil {
mgrErrorCh <- errors.Wrap(err, "failed to retrieve admission webhook secret")
return
}
if isPresent {
err := createWebhookService(context, o.context)
if err != nil {
mgrErrorCh <- errors.Wrap(err, "failed to create admission webhook service")
return
}
logger.Info("setting up admission webhooks")
for _, resource := range webhookResources {
err = ctrl.NewWebhookManagedBy(mgr).For(resource).Complete()
if err != nil {
mgrErrorCh <- errors.Wrapf(err, "failed to register webhook for %q", resource.GetObjectKind().GroupVersionKind().Kind)
return
}
}
}
// options to pass to the controllers
controllerOpts := &controllerconfig.Context{
ClusterdContext: o.context,
+1 -1
View File
@@ -170,7 +170,7 @@ func (r *ReconcileCephNFS) reconcile(request reconcile.Request) (reconcile.Resul
updateStatus(k8sutil.ObservedGenerationNotAvailable, r.client, request.NamespacedName, k8sutil.EmptyStatus)
}
if _, err := cephNFS.Spec.Security.Validate(); err != nil {
if err := cephNFS.Spec.Security.Validate(); err != nil {
return reconcile.Result{Requeue: true, RequeueAfter: 15 * time.Second}, *cephNFS,
errors.Wrapf(err, "failed to validate security spec for CephNFS %q",
types.NamespacedName{Namespace: cephNFS.Namespace, Name: cephNFS.Name})
+1 -1
View File
@@ -170,7 +170,7 @@ func (r *ReconcileBucketTopic) reconcile(request reconcile.Request) (reconcile.R
}
// validate the topic settings
_, err = cephBucketTopic.ValidateCreate()
err = cephBucketTopic.ValidateTopicSpec()
if err != nil {
r.updateStatus(k8sutil.ObservedGenerationNotAvailable, request.NamespacedName, k8sutil.ReconcileFailedStatus, nil)
return reconcile.Result{}, errors.Wrapf(err, "invalid CephBucketTopic %q", request.NamespacedName)
-51
View File
@@ -22,8 +22,6 @@ import (
"github.com/rook/rook/pkg/operator/ceph/controller"
v1 "k8s.io/api/core/v1"
kerrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/event"
"sigs.k8s.io/controller-runtime/pkg/predicate"
@@ -35,29 +33,7 @@ func predicateController(ctx context.Context, client client.Client) predicate.Fu
CreateFunc: func(e event.CreateEvent) bool {
if cm, ok := e.Object.(*v1.ConfigMap); ok {
return cm.Name == controller.OperatorSettingConfigMapName
} else if s, ok := e.Object.(*v1.Secret); ok {
if s.Name == admissionControllerAppName {
err := client.Get(ctx, types.NamespacedName{Name: admissionControllerAppName, Namespace: e.Object.GetNamespace()}, &v1.Service{})
if err != nil {
if kerrors.IsNotFound(err) {
// If the service is present we don't need to reload again. If we don't perform
// this check it will result in an infinite
// reconcile loop. CREATE event is only triggered when the controller is started
// no matter what.
// No need to ask for reconciliation since the context is going to be terminated when
// the signal is caught and the reconcile will run when the controller
// starts.
logger.Debug("webhook secret created reloading the manager to enable the webhook server")
controller.ReloadManager()
}
} else {
logger.Debug("webhook service already set up, not reloading the manager")
}
return false
}
}
return false
},
@@ -77,19 +53,6 @@ func predicateController(ctx context.Context, client client.Client) predicate.Fu
// We still want to reconcile the operator manager if the configmap is updated
return true
}
} else if s, ok := e.ObjectNew.(*v1.Secret); ok {
if s.Name == admissionControllerAppName {
logger.Debug("webhook secret updated, reloading the manager")
controller.ReloadManager()
// No need to ask for reconciliation since the context is going to be terminated when
// the signal is caught and the reconcile will run when the controller starts.
// If the admission controller secret is created or deleted we still need to reload and
// the webhook might be enabled or disabled
//
// The same goes the update, the secret changes we still need to reload the webhook
return false
}
}
}
@@ -103,20 +66,6 @@ func predicateController(ctx context.Context, client client.Client) predicate.Fu
return false
}
}
if s, ok := e.Object.(*v1.Secret); ok {
if s.Name == admissionControllerAppName {
logger.Debug("webhook secret deleted, reloading the manager")
controller.ReloadManager()
// No need to ask for reconciliation since the context is going to be terminated when
// the signal is caught and the reconcile will run when the controller starts.
// If the admission controller secret is created or deleted we still need to reload and
// the webhook might be enabled or disabled
//
// The same goes the update, the secret changes we still need to reload the webhook
return false
}
}
return false
},
+13 -2
View File
@@ -36,6 +36,17 @@ import (
const unknownKind = "<UnknownObjectKind>"
// Based on code from https://github.com/kubernetes/apimachinery/blob/master/pkg/api/meta/conditions.go
// A statusConditionGetter allows getting a pointer to an object's conditions.
type statusConditionGetter interface {
client.Object
// GetStatusConditions returns a pointer to the object's conditions compatible with
// SetStatusCondition and FindStatusCondition.
GetStatusConditions() *[]cephv1.Condition
}
// an object of a given type that has a nil reference is not the same as obj==nil (untyped nil)
// (e.g., var cluster cephv1.CephCluster = nil ), so we must also check for nil via reflection
func objIsNil(obj client.Object) bool {
@@ -164,7 +175,7 @@ func ReportReconcileResult(
// 2. as a condition on the object (added to the object's conditions list given)
// 3. as the returned error which should be included in the FailedReconcile message
func ReportDeletionBlockedDueToDependents(
ctx context.Context, logger *capnslog.PackageLogger, client client.Client, obj cephv1.StatusConditionGetter, deps *dependents.DependentList,
ctx context.Context, logger *capnslog.PackageLogger, client client.Client, obj statusConditionGetter, deps *dependents.DependentList,
) error {
kind := obj.GetObjectKind().GroupVersionKind().Kind
nsName := types.NamespacedName{
@@ -201,7 +212,7 @@ func ReportDeletionBlockedDueToDependents(
// 2. as an event on the object (via the given event recorder)
// 3. as a condition on the object (added to the object's conditions list given)
func ReportDeletionNotBlockedDueToDependents(
ctx context.Context, logger *capnslog.PackageLogger, client client.Client, recorder record.EventRecorder, obj cephv1.StatusConditionGetter,
ctx context.Context, logger *capnslog.PackageLogger, client client.Client, recorder record.EventRecorder, obj statusConditionGetter,
) {
kind := obj.GetObjectKind().GroupVersionKind().Kind
nsName := types.NamespacedName{
+1 -1
View File
@@ -51,7 +51,7 @@ func UpdateStatus(client client.Client, obj client.Object) error {
// UpdateStatusCondition updates (or adds to) the status condition to the given object. The object
// is updated with the latest version from the server on a successful update.
func UpdateStatusCondition(
client client.Client, obj cephv1.StatusConditionGetter, newCond cephv1.Condition,
client client.Client, obj statusConditionGetter, newCond cephv1.Condition,
) error {
kind := obj.GetObjectKind().GroupVersionKind().Kind
nsName := fmt.Sprintf("%s/%s", obj.GetNamespace(), obj.GetName())
-282
View File
@@ -1,282 +0,0 @@
/*
Copyright 2022 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package operator
import (
"context"
"fmt"
"time"
api "github.com/jetstack/cert-manager/pkg/apis/certmanager/v1"
v1 "github.com/jetstack/cert-manager/pkg/apis/meta/v1"
cs "github.com/jetstack/cert-manager/pkg/client/clientset/versioned/typed/certmanager/v1"
"github.com/pkg/errors"
"github.com/rook/rook/pkg/clusterd"
admv1 "k8s.io/api/admissionregistration/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
const (
issuerName = "selfsigned-issuer"
certificateName = "rook-admission-controller-cert"
webhookConfigName = "rook-ceph-webhook"
serviceCephClusterPath = "/validate-ceph-rook-io-v1-cephcluster"
serviceCephBlockPoolPath = "/validate-ceph-rook-io-v1-cephblockpool"
serviceCephObjectStorePath = "/validate-ceph-rook-io-v1-cephobjectstore"
serviceCephBlockPoolRadosNamespacePath = "/validate-ceph-rook-io-v1-cephblockpoolradosnamespace"
serviceCephFileSystemSubvolumeGroupPath = "/validate-ceph-rook-io-v1-cephfilesystemsubvolumegroup"
)
func fetchorCreateIssuer(ctx context.Context, certMgrClient *cs.CertmanagerV1Client) (*api.Issuer, error) {
logger.Infof("Fetching Issuer %s/%s.", namespace, issuerName)
issuer, err := certMgrClient.Issuers(namespace).Get(ctx, issuerName, metav1.GetOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
return createIssuers(ctx, certMgrClient)
}
logger.Errorf("failed to get issuer %s. %v", issuerName, err)
return issuer, err
}
return issuer, nil
}
func createIssuers(ctx context.Context, certMgrClient *cs.CertmanagerV1Client) (*api.Issuer, error) {
logger.Infof("Creating Issuer %s/%s.", namespace, issuerName)
issuer := &api.Issuer{
ObjectMeta: metav1.ObjectMeta{
Name: issuerName,
Namespace: namespace,
},
Spec: api.IssuerSpec{
IssuerConfig: api.IssuerConfig{
SelfSigned: &api.SelfSignedIssuer{},
},
},
}
issuers, err := certMgrClient.Issuers(namespace).Create(ctx, issuer, metav1.CreateOptions{})
if err != nil {
return nil, fmt.Errorf("failed to create issuer %s. %v", issuerName, err)
}
sleepTime := 1
attempts := 10
for i := 0; i < attempts; i++ {
issuers, err = certMgrClient.Issuers(namespace).Get(ctx, issuerName, metav1.GetOptions{})
if err != nil {
logger.Errorf("failed to get issuer %s. %v", issuerName, err)
return issuers, err
}
if len(issuers.Status.Conditions) != 0 {
if issuers.Status.Conditions[0].Reason == "IsReady" && issuers.Status.Conditions[0].Status == "True" {
return issuers, nil
}
logger.Debugf("webhook config %q status=%+v", issuerName, issuers.Status.Conditions[0].Status)
}
time.Sleep(time.Duration(sleepTime) * time.Second)
logger.Infof("waiting for webhook config %q to be in ready status", issuerName)
}
return issuers, nil
}
func fetchorCreateCertificate(ctx context.Context, certMgrClient *cs.CertmanagerV1Client, issuer *api.Issuer) error {
logger.Infof("Fetching certificate %s/%s.", namespace, certificateName)
_, err := certMgrClient.Certificates(namespace).Get(ctx, certificateName, metav1.GetOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
return createCertificate(ctx, certMgrClient, issuer)
}
logger.Errorf("failed to get certificate %s. %v", certificateName, err)
}
return nil
}
func createCertificate(ctx context.Context, certMgrClient *cs.CertmanagerV1Client, issuer *api.Issuer) error {
logger.Infof("Creating certificate %s/%s.", namespace, certificateName)
certificate := &api.Certificate{
ObjectMeta: metav1.ObjectMeta{
Name: certificateName,
Namespace: namespace,
},
Spec: api.CertificateSpec{
DNSNames: []string{
admissionControllerAppName,
fmt.Sprintf("%s.%s.svc", admissionControllerAppName, namespace),
fmt.Sprintf("%s.%s.svc.cluster.local", admissionControllerAppName, namespace)},
IssuerRef: v1.ObjectReference{
Name: issuer.Name,
Kind: "Issuer",
},
SecretName: admissionControllerAppName,
},
}
_, err := certMgrClient.Certificates(namespace).Create(ctx, certificate, metav1.CreateOptions{})
if err != nil {
return fmt.Errorf("failed to create certificate %s. %v", certificateName, err)
}
sleepTime := 1
attempts := 10
for i := 0; i < attempts; i++ {
cert, err := certMgrClient.Certificates(namespace).Get(ctx, certificateName, metav1.GetOptions{})
if err != nil {
logger.Errorf("failed to get certificate %s. %v", certificateName, err)
}
if len(cert.Status.Conditions) != 0 {
if cert.Status.Conditions[0].Reason == "Ready" && cert.Status.Conditions[0].Status == "True" {
return nil
}
logger.Debugf("webhook config %q status=%+v", certificateName, cert.Status.Conditions[0].Status)
}
logger.Infof("waiting for webhook config %q to be in ready status", certificateName)
time.Sleep(time.Duration(sleepTime) * time.Second)
}
return nil
}
func addValidatingWebhookConfig(ctx context.Context, clusterdContext *clusterd.Context) error {
resourcesWebhookName := []string{
fmt.Sprintf("cephcluster-wh-%s-%s.rook.io", admissionControllerAppName, namespace),
fmt.Sprintf("cephblockpool-wh-%s-%s.rook.io", admissionControllerAppName, namespace),
fmt.Sprintf("cephobjectstore-wh-%s-%s.rook.io", admissionControllerAppName, namespace),
fmt.Sprintf("cephblockpoolradosnamespace-wh-%s-%s.rook.io", admissionControllerAppName, namespace),
fmt.Sprintf("cephfilesystemsubvolumegroup-wh-%s-%s.rook.io", admissionControllerAppName, namespace),
}
resources := []string{
"cephclusters",
"cephblockpools",
"cephobjectstores",
"cephblockpoolradosnamespaces",
"cephfilesystemsubvolumegroups",
}
resourcesServicePath := []string{
serviceCephClusterPath,
serviceCephBlockPoolPath,
serviceCephObjectStorePath,
serviceCephBlockPoolRadosNamespacePath,
serviceCephFileSystemSubvolumeGroupPath,
}
logger.Infof("Creating webhook %s/%s.", namespace, webhookConfigName)
validatingWebhook := &admv1.ValidatingWebhookConfiguration{
ObjectMeta: metav1.ObjectMeta{
Name: webhookConfigName,
Namespace: namespace,
Annotations: map[string]string{
"cert-manager.io/inject-ca-from": fmt.Sprintf("%s/rook-admission-controller-cert", namespace),
},
},
}
for i := range resources {
validatingWebhook.Webhooks = append(validatingWebhook.Webhooks, webhookRules(resourcesWebhookName[i], resources[i], resourcesServicePath[i]))
}
webhookConfig, err := clusterdContext.Clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Get(ctx, validatingWebhook.Name, metav1.GetOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
_, err := clusterdContext.Clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Create(ctx, validatingWebhook, metav1.CreateOptions{})
if err != nil {
logger.Errorf("failed to create admission webhook %s. %v", webhookConfigName, err)
return errors.Wrapf(err, "failed to create admission webhook %q", validatingWebhook.Name)
}
return nil
}
return errors.Wrapf(err, "failed to get admission webhook config %q", validatingWebhook.Name)
}
validatingWebhook.SetResourceVersion(webhookConfig.GetResourceVersion())
logger.Info("admission webhook already exists. updating it")
_, err = clusterdContext.Clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Update(ctx, validatingWebhook, metav1.UpdateOptions{})
if err != nil {
logger.Errorf("failed to update admission webhook %s. %v", webhookConfigName, err)
return errors.Wrapf(err, "failed to update admission webhook %q", validatingWebhook.Name)
}
return nil
}
func webhookRules(name, resource, resourceServicePath string) admv1.ValidatingWebhook {
var timeout int32 = 5
sideEffects := admv1.SideEffectClassNone
return admv1.ValidatingWebhook{
Name: name,
Rules: []admv1.RuleWithOperations{
{
Rule: admv1.Rule{
APIGroups: []string{"ceph.rook.io"},
APIVersions: []string{"v1"},
Resources: []string{resource},
},
Operations: []admv1.OperationType{
admv1.Update,
admv1.Create,
admv1.Delete,
},
},
},
ClientConfig: admv1.WebhookClientConfig{
Service: &admv1.ServiceReference{
Name: admissionControllerAppName,
Namespace: namespace,
Path: &resourceServicePath,
},
},
AdmissionReviewVersions: []string{"v1", "v1beta1"},
SideEffects: &sideEffects,
TimeoutSeconds: &timeout,
}
}
// No need to return err here since we are just deleting and not handle errors
func deleteWebhookResources(ctx context.Context, certMgrClient *cs.CertmanagerV1Client, clusterdContext *clusterd.Context) {
logger.Infof("deleting validating webhook %s", webhookConfigName)
err := clusterdContext.Clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Delete(ctx, webhookConfigName, metav1.DeleteOptions{})
if err != nil && !apierrors.IsNotFound(err) {
logger.Errorf("failed to delete validating webhook %s. %v", webhookConfigName, err)
}
logger.Infof("deleting webhook cert manager Certificate %s", certificateName)
err = certMgrClient.Certificates(namespace).Delete(ctx, certificateName, metav1.DeleteOptions{})
if err != nil && !apierrors.IsNotFound(err) {
logger.Errorf("failed to delete webhook cert manager Certificate %s. %v", certificateName, err)
}
logger.Infof("deleting webhook cert manager Issuer %q", issuerName)
err = certMgrClient.Issuers(namespace).Delete(ctx, issuerName, metav1.DeleteOptions{})
if err != nil && !apierrors.IsNotFound(err) {
logger.Errorf("failed to delete webhook cert manager Issuer %s. %v", issuerName, err)
}
logger.Infof("deleting validating webhook service %q", admissionControllerAppName)
err = clusterdContext.Clientset.CoreV1().Services(namespace).Delete(ctx, admissionControllerAppName, metav1.DeleteOptions{})
if err != nil && !apierrors.IsNotFound(err) {
logger.Errorf("failed to delete validating webhook service %s. %v", admissionControllerAppName, err)
}
}
-146
View File
@@ -1,146 +0,0 @@
/*
Copyright 2020 The Rook Authors. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package operator
import (
"context"
"os"
"path"
cs "github.com/jetstack/cert-manager/pkg/client/clientset/versioned/typed/certmanager/v1"
"github.com/pkg/errors"
"github.com/rook/rook/pkg/clusterd"
opcontroller "github.com/rook/rook/pkg/operator/ceph/controller"
"github.com/rook/rook/pkg/operator/k8sutil"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/util/intstr"
"sigs.k8s.io/controller-runtime/pkg/webhook"
)
const (
admissionControllerAppName = "rook-ceph-admission-controller"
tlsPort int32 = 443
)
var (
namespace = os.Getenv(k8sutil.PodNamespaceEnvVar)
certManagerWebhookName = "cert-manager-webhook"
)
func createWebhook(ctx context.Context, context *clusterd.Context) (bool, error) {
certMgrClient, err := cs.NewForConfig(context.KubeConfig)
if err != nil {
logger.Errorf("failed to set config for cert-manager. %v", err)
return false, nil
}
value, err := k8sutil.GetOperatorSetting(ctx, context.Clientset, opcontroller.OperatorSettingConfigMapName, "ROOK_DISABLE_ADMISSION_CONTROLLER", "true")
if err != nil {
return false, err
}
if value == "true" {
logger.Info("delete webhook resources since webhook is disabled")
deleteWebhookResources(ctx, certMgrClient, context)
return false, nil
}
logger.Infof("Fetching webhook %s to see if cert-manager is installed.", certManagerWebhookName)
_, err = context.Clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Get(ctx, certManagerWebhookName, metav1.GetOptions{})
if err != nil {
logger.Info("failed to get cert manager")
return false, nil
}
issuer, err := fetchorCreateIssuer(ctx, certMgrClient)
if err != nil {
logger.Errorf("issuer creation failed %v", err)
return false, nil
}
err = fetchorCreateCertificate(ctx, certMgrClient, issuer)
if err != nil {
logger.Errorf("certificate creation failed %v", err)
return false, nil
}
logger.Infof("looking for admission webhook secret %q", admissionControllerAppName)
s, err := context.Clientset.CoreV1().Secrets(namespace).Get(ctx, admissionControllerAppName, metav1.GetOptions{})
if err != nil {
if apierrors.IsNotFound(err) {
// If secret is not found. All good ! Proceed with rook without admission controllers
logger.Info("delete Issuer and Certificate since secret is not found")
deleteWebhookResources(ctx, certMgrClient, context)
logger.Infof("admission webhook secret %q not found. proceeding without the admission controller", admissionControllerAppName)
return false, nil
}
return false, err
}
logger.Infof("admission webhook secret %q found", admissionControllerAppName)
err = addValidatingWebhookConfig(ctx, context)
if err != nil {
logger.Errorf("adding webhook failed %v", err)
return false, nil
}
for k, data := range s.Data {
filePath := path.Join(certDir, k)
// We must use 0600 mode so that the files can be overridden each time the Secret is fetched
// to keep an updated content
err := os.WriteFile(filePath, data, 0600)
if err != nil {
return false, errors.Wrapf(err, "failed to write secret content to file %q", filePath)
}
}
return true, nil
}
func createWebhookService(ctx context.Context, context *clusterd.Context) error {
webhookService := corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: admissionControllerAppName,
Namespace: namespace,
},
Spec: corev1.ServiceSpec{
Ports: []corev1.ServicePort{
{
Port: tlsPort,
TargetPort: intstr.IntOrString{
IntVal: int32(webhook.DefaultPort),
},
},
},
Selector: map[string]string{
k8sutil.AppAttr: "rook-ceph-operator",
},
},
}
_, err := k8sutil.CreateOrUpdateService(ctx, context.Clientset, namespace, &webhookService)
if err != nil {
return err
}
return nil
}
@@ -65,13 +65,11 @@ func (h *CephInstaller) configureRookOperatorViaHelm(upgrade bool) error {
"csiCephFSPluginResource": nil,
}
// create the operator namespace before the admission controller is created
// create the operator namespace
if err := h.k8shelper.CreateNamespace(h.settings.OperatorNamespace); err != nil {
return errors.Errorf("failed to create namespace %s. %v", h.settings.Namespace, err)
}
if err := h.startAdmissionController(); err != nil {
return errors.Errorf("failed to start admission controllers. %v", err)
}
if h.settings.RookVersion == LocalBuildTag {
if err := h.helmHelper.InstallLocalHelmChart(upgrade, h.settings.OperatorNamespace, OperatorChartName, values); err != nil {
return errors.Errorf("failed to install rook operator via helm, err : %v", err)
@@ -135,11 +135,6 @@ func (h *CephInstaller) CreateCephOperator() (err error) {
}
}
err = h.startAdmissionController()
if err != nil {
return errors.Errorf("Failed to start admission controllers: %v", err)
}
if err := h.CreateVolumeReplicationCRDs(); err != nil {
return errors.Wrap(err, "failed to create volume replication CRDs")
}
@@ -170,37 +165,6 @@ func (h *CephInstaller) CreateVolumeReplicationCRDs() (err error) {
return nil
}
func (h *CephInstaller) startAdmissionController() error {
if !h.k8shelper.VersionAtLeast("v1.16.0") {
logger.Info("skipping the admission controller on K8s version older than v1.16")
return nil
}
if !h.settings.EnableAdmissionController {
logger.Info("skipping admission controller for this test suite")
return nil
}
if utils.IsPlatformOpenShift() {
logger.Info("skipping the admission controller on OpenShift")
return nil
}
rootPath, err := utils.FindRookRoot()
if err != nil {
return errors.Errorf("failed to find rook root. %v", err)
}
userHome, err := os.UserHomeDir()
if err != nil {
return errors.Errorf("failed to find user home directory. %v", err)
}
scriptPath := path.Join(rootPath, "tests/scripts/deploy_cert_manager.sh")
err = h.k8shelper.MakeContext().Executor.ExecuteCommandWithEnv([]string{fmt.Sprintf("NAMESPACE=%s", h.settings.OperatorNamespace), fmt.Sprintf("HOME=%s", userHome)}, "bash", scriptPath)
if err != nil {
return err
}
return nil
}
func (h *CephInstaller) WaitForToolbox(namespace string) error {
if err := h.k8shelper.WaitForLabeledPodsToRun("app=rook-ceph-tools", namespace); err != nil {
return errors.Wrap(err, "Rook Toolbox couldn't start")
@@ -795,9 +759,6 @@ func (h *CephInstaller) UninstallRookFromMultipleNS(manifests ...CephManifests)
err = h.k8shelper.DeleteResourceAndWait(false, "namespace", h.settings.OperatorNamespace)
checkError(h.T(), err, fmt.Sprintf("cannot delete operator namespace %s", h.settings.OperatorNamespace))
err = h.k8shelper.Clientset.AdmissionregistrationV1().ValidatingWebhookConfigurations().Delete(ctx, "rook-ceph-webhook", metav1.DeleteOptions{})
checkError(h.T(), err, "failed to delete webhook configuration")
logger.Infof("done removing the operator from namespace %s", h.settings.OperatorNamespace)
logger.Infof("removing host data dir %s", h.hostPathToDelete)
// removing data dir if exists
@@ -39,7 +39,6 @@ type TestCephSettings struct {
MultipleMgrs bool
SkipOSDCreation bool
EnableDiscovery bool
EnableAdmissionController bool
IsExternal bool
SkipClusterCleanup bool
SkipCleanupPolicy bool
@@ -65,10 +64,6 @@ func (s *TestCephSettings) ApplyEnvVars() {
if os.Getenv("SKIP_CLEANUP_POLICY") == "false" {
s.SkipCleanupPolicy = false
}
err := os.Setenv("ROOK_DISABLE_ADMISSION_CONTROLLER", "false")
if err != nil {
logger.Errorf("failed to set ROOK_DISABLE_ADMISSION_CONTROLLER. %v", err)
}
}
func (s *TestCephSettings) readManifest(filename string) string {
@@ -91,7 +86,6 @@ func (s *TestCephSettings) replaceOperatorSettings(manifest string) string {
manifest = strings.ReplaceAll(manifest, `ROOK_ENABLE_DISCOVERY_DAEMON: "false"`, fmt.Sprintf(`ROOK_ENABLE_DISCOVERY_DAEMON: "%t"`, s.EnableDiscovery))
manifest = strings.ReplaceAll(manifest, `CSI_ENABLE_VOLUME_REPLICATION: "false"`, fmt.Sprintf(`CSI_ENABLE_VOLUME_REPLICATION: "%t"`, s.EnableVolumeReplication))
manifest = strings.ReplaceAll(manifest, `ROOK_CSI_ENABLE_NFS: "false"`, fmt.Sprintf(`ROOK_CSI_ENABLE_NFS: "%t"`, s.TestNFSCSI))
manifest = strings.ReplaceAll(manifest, `ROOK_DISABLE_ADMISSION_CONTROLLER: "true"`, `ROOK_DISABLE_ADMISSION_CONTROLLER: "false"`)
return manifest
}
+12 -13
View File
@@ -59,19 +59,18 @@ type HelmSuite struct {
func (h *HelmSuite) SetupSuite() {
namespace := "helm-ns"
h.settings = &installer.TestCephSettings{
Namespace: namespace,
OperatorNamespace: namespace,
StorageClassName: "",
UseHelm: true,
UsePVC: false,
Mons: 1,
SkipOSDCreation: false,
EnableAdmissionController: true,
EnableDiscovery: true,
ChangeHostName: true,
ConnectionsEncrypted: true,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.QuincyVersion,
Namespace: namespace,
OperatorNamespace: namespace,
StorageClassName: "",
UseHelm: true,
UsePVC: false,
Mons: 1,
SkipOSDCreation: false,
EnableDiscovery: true,
ChangeHostName: true,
ConnectionsEncrypted: true,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.QuincyVersion,
}
h.settings.ApplyEnvVars()
h.installer, h.k8shelper = StartTestCluster(h.T, h.settings)
+10 -11
View File
@@ -73,17 +73,16 @@ func (s *MultiClusterDeploySuite) SetupSuite() {
s.poolName = "multi-cluster-pool1"
coreNamespace := "multi-core"
s.settings = &installer.TestCephSettings{
ClusterName: "multi-cluster",
Namespace: coreNamespace,
OperatorNamespace: installer.SystemNamespace(coreNamespace),
StorageClassName: "manual",
UsePVC: installer.UsePVC(),
Mons: 1,
MultipleMgrs: true,
EnableAdmissionController: true,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.QuincyVersion,
RequireMsgr2: true,
ClusterName: "multi-cluster",
Namespace: coreNamespace,
OperatorNamespace: installer.SystemNamespace(coreNamespace),
StorageClassName: "manual",
UsePVC: installer.UsePVC(),
Mons: 1,
MultipleMgrs: true,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.QuincyVersion,
RequireMsgr2: true,
}
s.settings.ApplyEnvVars()
externalSettings := &installer.TestCephSettings{
+12 -13
View File
@@ -63,19 +63,18 @@ type ObjectSuite struct {
func (s *ObjectSuite) SetupSuite() {
namespace := "object-ns"
s.settings = &installer.TestCephSettings{
ClusterName: "object-cluster",
Namespace: namespace,
OperatorNamespace: installer.SystemNamespace(namespace),
StorageClassName: installer.StorageClassName(),
UseHelm: false,
UsePVC: installer.UsePVC(),
Mons: 3,
SkipOSDCreation: false,
EnableAdmissionController: true,
UseCrashPruner: true,
EnableVolumeReplication: false,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.ReturnCephVersion(),
ClusterName: "object-cluster",
Namespace: namespace,
OperatorNamespace: installer.SystemNamespace(namespace),
StorageClassName: installer.StorageClassName(),
UseHelm: false,
UsePVC: installer.UsePVC(),
Mons: 3,
SkipOSDCreation: false,
UseCrashPruner: true,
EnableVolumeReplication: false,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.ReturnCephVersion(),
}
s.settings.ApplyEnvVars()
s.installer, s.k8sh = StartTestCluster(s.T, s.settings)
+16 -120
View File
@@ -23,7 +23,6 @@ import (
"testing"
"time"
cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1"
"github.com/rook/rook/pkg/daemon/ceph/client"
opcontroller "github.com/rook/rook/pkg/operator/ceph/controller"
"github.com/rook/rook/tests/framework/clients"
@@ -34,7 +33,6 @@ import (
"github.com/stretchr/testify/suite"
appsv1 "k8s.io/api/apps/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
)
// ************************************************
@@ -81,23 +79,22 @@ type SmokeSuite struct {
func (s *SmokeSuite) SetupSuite() {
namespace := "smoke-ns"
s.settings = &installer.TestCephSettings{
ClusterName: "smoke-cluster",
Namespace: namespace,
OperatorNamespace: installer.SystemNamespace(namespace),
StorageClassName: installer.StorageClassName(),
UseHelm: false,
UsePVC: installer.UsePVC(),
Mons: 3,
SkipOSDCreation: false,
EnableAdmissionController: true,
ConnectionsEncrypted: true,
ConnectionsCompressed: true,
UseCrashPruner: true,
EnableVolumeReplication: true,
TestNFSCSI: true,
ChangeHostName: true,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.ReturnCephVersion(),
ClusterName: "smoke-cluster",
Namespace: namespace,
OperatorNamespace: installer.SystemNamespace(namespace),
StorageClassName: installer.StorageClassName(),
UseHelm: false,
UsePVC: installer.UsePVC(),
Mons: 3,
SkipOSDCreation: false,
ConnectionsEncrypted: true,
ConnectionsCompressed: true,
UseCrashPruner: true,
EnableVolumeReplication: true,
TestNFSCSI: true,
ChangeHostName: true,
RookVersion: installer.LocalBuildTag,
CephVersion: installer.ReturnCephVersion(),
}
s.settings.ApplyEnvVars()
s.installer, s.k8sh = StartTestCluster(s.T, s.settings)
@@ -348,104 +345,3 @@ func (s *SmokeSuite) getNonCanaryMonDeployments() ([]appsv1.Deployment, error) {
}
return nonCanaryMonDeployments, nil
}
// Smoke Test for Webhooks
func (s *SmokeSuite) TestWebhooks() {
logger.Info("Create webhook Smoke Test")
ctx := context.TODO()
t := s.T()
t.Run("During cephCluster update, hostPath cannot be changed", func(t *testing.T) {
cephCRList, err := s.k8sh.RookClientset.CephV1().CephClusters(s.settings.Namespace).List(ctx, metav1.ListOptions{})
require.NoError(s.T(), err)
cephCR := cephCRList.Items[0]
cephCR.Spec.DataDirHostPath = "/var"
_, err = s.k8sh.RookClientset.CephV1().CephClusters(s.settings.Namespace).Update(ctx, &cephCR, metav1.UpdateOptions{})
require.Error(s.T(), err)
})
t.Run("During blockPool creation, both erasurecoded and replicated fields cannot be set at the same time", func(t *testing.T) {
pool := &cephv1.CephBlockPool{
ObjectMeta: metav1.ObjectMeta{
Name: "webhook-pool",
Namespace: s.settings.Namespace,
UID: types.UID("c47cac40-9bee-4d52-823b-ccd803ba5bfe"),
},
Spec: cephv1.NamedBlockPoolSpec{
PoolSpec: cephv1.PoolSpec{
Replicated: cephv1.ReplicatedSpec{
Size: uint(1),
},
ErasureCoded: cephv1.ErasureCodedSpec{
CodingChunks: 1,
DataChunks: 1,
},
},
},
}
_, err := s.k8sh.RookClientset.CephV1().CephBlockPools(s.settings.Namespace).Create(ctx, pool, metav1.CreateOptions{})
require.Error(s.T(), err)
})
t.Run("During objectStore creation, either of port or securePort fields should be not be zero ", func(t *testing.T) {
objectStore := &cephv1.CephObjectStore{
ObjectMeta: metav1.ObjectMeta{
Name: "webhook-store",
Namespace: s.settings.Namespace,
},
TypeMeta: metav1.TypeMeta{
Kind: "CephObjectStore",
},
Spec: cephv1.ObjectStoreSpec{
Gateway: cephv1.GatewaySpec{
Port: 0,
SecurePort: 0,
},
},
}
_, err := s.k8sh.RookClientset.CephV1().CephObjectStores(s.settings.Namespace).Create(ctx, objectStore, metav1.CreateOptions{})
require.Error(s.T(), err)
})
t.Run("During filesystemSubVolumeGroup update filesystem name cannot be changed", func(t *testing.T) {
subVolumeGroup := &cephv1.CephFilesystemSubVolumeGroup{
ObjectMeta: metav1.ObjectMeta{
Name: "subvol-name",
},
Spec: cephv1.CephFilesystemSubVolumeGroupSpec{
FilesystemName: "filesystem-name",
},
}
subVolumeGroup, err := s.k8sh.RookClientset.CephV1().CephFilesystemSubVolumeGroups(s.settings.Namespace).Create(ctx, subVolumeGroup, metav1.CreateOptions{})
require.NoError(s.T(), err)
subVolumeGroup.Spec.FilesystemName = "new-fs"
_, err = s.k8sh.RookClientset.CephV1().CephFilesystemSubVolumeGroups(s.settings.Namespace).Update(ctx, subVolumeGroup, metav1.UpdateOptions{})
require.Error(s.T(), err)
// Delete the subVolumeGroup for cleanup
err = s.k8sh.RookClientset.CephV1().CephFilesystemSubVolumeGroups(s.settings.Namespace).Delete(ctx, subVolumeGroup.Name, metav1.DeleteOptions{})
require.NoError(s.T(), err)
})
t.Run("During blockPoolRadosNamespace update blockpool name cannot be changed", func(t *testing.T) {
rn := &cephv1.CephBlockPoolRadosNamespace{
ObjectMeta: metav1.ObjectMeta{
Name: "rados-name",
},
Spec: cephv1.CephBlockPoolRadosNamespaceSpec{
BlockPoolName: "pool-name",
},
}
rn, err := s.k8sh.RookClientset.CephV1().CephBlockPoolRadosNamespaces(s.settings.Namespace).Create(ctx, rn, metav1.CreateOptions{})
require.NoError(s.T(), err)
rn.Spec.BlockPoolName = "new-pool"
_, err = s.k8sh.RookClientset.CephV1().CephBlockPoolRadosNamespaces(s.settings.Namespace).Update(ctx, rn, metav1.UpdateOptions{})
require.Error(s.T(), err)
// Delete the radosNamespace for cleanup
err = s.k8sh.RookClientset.CephV1().CephBlockPoolRadosNamespaces(s.settings.Namespace).Delete(ctx, rn.Name, metav1.DeleteOptions{})
require.NoError(s.T(), err)
})
}
-61
View File
@@ -1,61 +0,0 @@
#!/usr/bin/env bash
# Sets up the cert-manager for CI
set -eEo pipefail
function error_log() {
set +e -x
kubectl -n rook-ceph get issuer
kubectl -n rook-ceph get certificate
kubectl -n rook-ceph get secret | grep rook-ceph-admission-controller
kubectl -n rook-ceph get validatingwebhookconfigurations.admissionregistration.k8s.io
kubectl describe validatingwebhookconfigurations.admissionregistration.k8s.io cert-manager-webhook
kubectl describe validatingwebhookconfigurations.admissionregistration.k8s.io rook-ceph-webhook
kubectl -n cert-manager logs deploy/cert-manager-webhook --tail=10
kubectl -n cert-manager logs deploy/cert-manager-cainjector --tail=10
set -e +x
}
trap error_log ERR
# Set our known directories and parameters.
BASE_DIR=$(cd "$(dirname "$0")"; pwd)
CERT_VERSION="v1.3.1"
[ -z "${NAMESPACE}" ] && NAMESPACE="rook-ceph"
export NAMESPACE
export WEBHOOK_CONFIG_NAME="rook-ceph-webhook"
export SERVICE_NAME="rook-ceph-admission-controller"
echo "$BASE_DIR"
echo "Deploying cert-manager"
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/$CERT_VERSION/cert-manager.yaml
timeout 150 bash <<-'EOF'
until [ $(kubectl -n cert-manager get pods --field-selector=status.phase=Running | grep -c ^cert-) -eq 3 ]; do
echo "waiting for cert-manager pods to be in running state"
sleep 1
done
EOF
timeout 20 bash <<-'EOF'
until [ $(kubectl -n cert-manager get pods -o custom-columns=READY:status.containerStatuses[*].ready | grep -c true) -eq 3 ]; do
echo "waiting for the pods to be in ready state"
sleep 1
done
EOF
timeout 25 bash <<-'EOF'
until [ $(kubectl get validatingwebhookconfigurations cert-manager-webhook -o jsonpath='{.webhooks[*].clientConfig.caBundle}' | wc -c) -gt 1 ]; do
echo "waiting for caInjector to inject in caBundle for cert-manager validating webhook"
sleep 1
done
EOF
timeout 25 bash <<-'EOF'
until [ $(kubectl get mutatingwebhookconfigurations cert-manager-webhook -o jsonpath='{.webhooks[*].clientConfig.caBundle}' | wc -c) -gt 1 ]; do
echo "waiting for caInjector to inject in caBundle for cert-managers mutating webhook"
sleep 1
done
EOF
echo "Successfully deployed cert-manager"