ceph: update deprecated CRD apiextensions.k8s.io/v1beta1 to v1

the apiextensions.k8s.io/v1beta1 version of CustomResourceDefinition
is deprecated in Kubernetes v1.16 and will no longer be supported from
v1.19. For now, we changing only for ceph and it's related documented.

Signed-off-by: subhamkrai <srai@redhat.com>
Co-authored-by: travisn <tnielsen@redhat.com>
Signed-off-by: subhamkrai <srai@redhat.com>
This commit is contained in:
subhamkraiandtravisn committed 2020-10-29 06:26:10 +05:30
1 parent 0134a3c453
commit 1b2c15041f
13 files changed
+4378 -528

No files matched your search

+8
View File
@@ -29,6 +29,7 @@ If after trying the suggestions found on this page and the problem is not resolv
* [Too few PGs per OSD warning is shown](#too-few-pgs-per-osd-warning-is-shown)
* [LVM metadata can be corrupted with OSD on LV-backed PVC](#lvm-metadata-can-be-corrupted-with-osd-on-lv-backed-pvc)
* [OSD prepare job fails due to low aio-max-nr setting](#osd-prepare-job-fails-due-to-low-aio-max-nr-setting)
* [Failed to create CRDs](#failed-to-create-crds)
## Troubleshooting Techniques
@@ -833,3 +834,10 @@ To overcome this, you need to increase the value of `fs.aio-max-nr` of your sysc
You can do this with your favorite configuration management system.
Alternatively, you can have a [DaemonSet](https://github.com/rook/rook/issues/6279#issuecomment-694390514) to apply the configuration for you on all your nodes.
## Failed to create CRDs
If you are using Kubernetes version is v1.15 or older, you will see an error like this:
```
unable to recognize "STDIN": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1"
```
You need to create the CRDs found in `cluster/examples/kubernetes/ceph/pre-k8s-1.16`. Note that these pre-1.16 `apiextensions.k8s.io/v1beta1` CRDs are deprecated in k8s v1.16 and will no longer be supported from k8s v1.22.
+7
View File
@@ -16,6 +16,8 @@ from other pods running in your cluster.
Kubernetes **v1.11** or higher is supported by Rook.
**Important** If you are using K8s 1.15 or older, you will need to create a different version of the Rook CRDs. Create the `crd.yaml` found in the [pre-k8s-1.16](https://github.com/rook/rook/blob/{{ branchName }}/cluster/examples/kubernetes/ceph/pre-k8s-1.16) subfolder of the example manifests.
## Prerequisites
To make sure you have a Kubernetes cluster that is ready for `Rook`, you can [follow these instructions](k8s-pre-reqs.md).
@@ -76,6 +78,11 @@ kubectl create -f operator.yaml
kubectl -n rook-ceph get pod
```
**NOTE**
If you are using kubernetes v1.15 or older you need to create CRDs found here `/cluster/examples/kubernetes/ceph` as apiextension v1beta1 version of CustomResourceDefinition
is deprecated in Kubernetes v1.16 and will no longer be supported from k8s v1.22.
You can also deploy the operator with the [Rook Helm Chart](helm-operator.md).
## Create a Rook Ceph Cluster
+3 -1
View File
@@ -18,7 +18,9 @@ you can quickly set one up using [Minikube](#minikube), [Kubeadm](#kubeadm) or [
## Minimum Version
Kubernetes v1.11 or higher is supported by Rook.
Kubernetes **v1.11** or higher is supported by Rook.
**Important** If you are using K8s 1.15 or older, you will need to create a different version of the Rook CRDs. Create the `crd.yaml` found in the [pre-k8s-1.16](https://github.com/rook/rook/blob/{{ branchName }}/cluster/examples/kubernetes/ceph/pre-k8s-1.16) subfolder of the example manifests.
## Ceph Prerequisites
+1
View File
@@ -11,6 +11,7 @@ v1.5...
### Ceph
- Ceph mons require an odd number for a healthy quorum. An even number of mons is now disallowed.
- Update deprecated CRD apiextensions.k8s.io/v1beta1 to v1 ([#6424](https://github.com/rook/rook/pull/6424))
## Features
File diff suppressed because it is too large. Load diff
@@ -54,7 +54,8 @@ spec:
# placement: # is ignored here in favor of Rook's placement below
# host_pattern: # (deprecated in Ceph) is similarly ignored here in favor of Rook's placement
# service_id: # is always set to the Drive Group name ("all_nodes" here)
placement: # example: omitting 'placement' or leaving it empty matches all untainted nodes
# omitting 'placement' or leaving it empty matches all untainted nodes
#placement: # omitting 'placement' or leaving it empty matches all untainted nodes
# # Further examples
# - name: nodes_by_hostname
File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -43,4 +43,4 @@ spec:
# cpu: "500m"
# memory: "1024Mi"
# the priority class to set to influence the scheduler's pod preemption
priorityClassName:
#priorityClassName:
@@ -0,0 +1,752 @@
###################################################################################################################
# Create the common resources that are necessary to start the operator and the ceph cluster.
# These resources *must* be created before the operator.yaml and cluster.yaml or their variants.
# The samples all assume that a single operator will manage a single cluster crd in the same "rook-ceph" namespace.
#
# If the operator needs to manage multiple clusters (in different namespaces), see the section below
# for "cluster-specific resources". The resources below that section will need to be created for each namespace
# where the operator needs to manage the cluster. The resources above that section do not be created again.
#
# Most of the sections are prefixed with a 'OLM' keyword which is used to build our CSV for an OLM (Operator Life Cycle manager)
###################################################################################################################
# Namespace where the operator and other rook resources are created
apiVersion: v1
kind: Namespace
metadata:
name: rook-ceph
# The CRD declarations
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephclusters.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephCluster
listKind: CephClusterList
plural: cephclusters
singular: cephcluster
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
annotations: {}
cephVersion:
properties:
allowUnsupported:
type: boolean
image:
type: string
dashboard:
properties:
enabled:
type: boolean
urlPrefix:
type: string
port:
type: integer
minimum: 0
maximum: 65535
ssl:
type: boolean
dataDirHostPath:
pattern: ^/(\S+)
type: string
disruptionManagement:
properties:
machineDisruptionBudgetNamespace:
type: string
managePodBudgets:
type: boolean
osdMaintenanceTimeout:
type: integer
manageMachineDisruptionBudgets:
type: boolean
skipUpgradeChecks:
type: boolean
continueUpgradeAfterChecksEvenIfNotHealthy:
type: boolean
mon:
properties:
allowMultiplePerNode:
type: boolean
count:
maximum: 9
minimum: 0
type: integer
volumeClaimTemplate: {}
mgr:
properties:
modules:
items:
properties:
name:
type: string
enabled:
type: boolean
network:
properties:
hostNetwork:
type: boolean
provider:
type: string
selectors: {}
storage:
properties:
disruptionManagement:
properties:
machineDisruptionBudgetNamespace:
type: string
managePodBudgets:
type: boolean
osdMaintenanceTimeout:
type: integer
manageMachineDisruptionBudgets:
type: boolean
useAllNodes:
type: boolean
nodes:
items:
properties:
name:
type: string
config:
properties:
metadataDevice:
type: string
storeType:
type: string
pattern: ^(bluestore)$
databaseSizeMB:
type: string
walSizeMB:
type: string
journalSizeMB:
type: string
osdsPerDevice:
type: string
encryptedDevice:
type: string
pattern: ^(true|false)$
useAllDevices:
type: boolean
deviceFilter:
type: string
devicePathFilter:
type: string
devices:
type: array
items:
properties:
name:
type: string
config: {}
resources: {}
type: array
useAllDevices:
type: boolean
deviceFilter:
type: string
devicePathFilter:
type: string
config: {}
storageClassDeviceSets: {}
driveGroups:
type: array
nullable: true
items:
properties:
name:
type: string
spec: {}
placement: {}
required:
- name
- spec
monitoring:
properties:
enabled:
type: boolean
rulesNamespace:
type: string
externalMgrEndpoints:
type: array
items:
properties:
ip:
type: string
removeOSDsIfOutAndSafeToRemove:
type: boolean
external:
properties:
enable:
type: boolean
cleanupPolicy:
properties:
confirmation:
type: string
pattern: ^$|^yes-really-destroy-data$
sanitizeDisks:
properties:
method:
type: string
pattern: ^(complete|quick)$
dataSource:
type: string
pattern: ^(zero|random)$
iteration:
type: integer
format: int32
placement: {}
resources: {}
healthCheck: {}
subresources:
status: {}
additionalPrinterColumns:
- name: DataDirHostPath
type: string
description: Directory used on the K8s nodes
JSONPath: .spec.dataDirHostPath
- name: MonCount
type: string
description: Number of MONs
JSONPath: .spec.mon.count
- name: Age
type: date
JSONPath: .metadata.creationTimestamp
- name: Phase
type: string
description: Phase
JSONPath: .status.phase
- name: Message
type: string
description: Message
JSONPath: .status.message
- name: Health
type: string
description: Ceph Health
JSONPath: .status.ceph.health
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephclients.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephClient
listKind: CephClientList
plural: cephclients
singular: cephclient
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
caps:
type: object
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephrbdmirrors.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephRBDMirror
listKind: CephRBDMirrorList
plural: cephrbdmirrors
singular: cephrbdmirror
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
count:
type: integer
minimum: 1
maximum: 100
peers:
properties:
secretNames:
type: array
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephfilesystems.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephFilesystem
listKind: CephFilesystemList
plural: cephfilesystems
singular: cephfilesystem
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
metadataServer:
properties:
activeCount:
minimum: 1
maximum: 10
type: integer
activeStandby:
type: boolean
annotations: {}
placement: {}
resources: {}
metadataPool:
properties:
failureDomain:
type: string
crushRoot:
type: string
replicated:
properties:
size:
minimum: 0
maximum: 10
type: integer
requireSafeReplicaSize:
type: boolean
erasureCoded:
properties:
dataChunks:
minimum: 0
maximum: 10
type: integer
codingChunks:
minimum: 0
maximum: 10
type: integer
compressionMode:
type: string
enum:
- ""
- none
- passive
- aggressive
- force
dataPools:
type: array
items:
properties:
failureDomain:
type: string
crushRoot:
type: string
replicated:
properties:
size:
minimum: 0
maximum: 10
type: integer
requireSafeReplicaSize:
type: boolean
erasureCoded:
properties:
dataChunks:
minimum: 0
maximum: 10
type: integer
codingChunks:
minimum: 0
maximum: 10
type: integer
compressionMode:
type: string
enum:
- ""
- none
- passive
- aggressive
- force
parameters:
type: object
preservePoolsOnDelete:
type: boolean
additionalPrinterColumns:
- name: ActiveMDS
type: string
description: Number of desired active MDS daemons
JSONPath: .spec.metadataServer.activeCount
- name: Age
type: date
JSONPath: .metadata.creationTimestamp
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephnfses.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephNFS
listKind: CephNFSList
plural: cephnfses
singular: cephnfs
shortNames:
- nfs
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
rados:
properties:
pool:
type: string
namespace:
type: string
server:
properties:
active:
type: integer
annotations: {}
placement: {}
resources: {}
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephobjectstores.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephObjectStore
listKind: CephObjectStoreList
plural: cephobjectstores
singular: cephobjectstore
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
gateway:
properties:
type:
type: string
sslCertificateRef: {}
port:
type: integer
minimum: 0
maximum: 65535
securePort:
type: integer
minimum: 0
maximum: 65535
instances:
type: integer
externalRgwEndpoints:
type: array
items:
properties:
ip:
type: string
annotations: {}
placement: {}
resources: {}
metadataPool:
properties:
failureDomain:
type: string
crushRoot:
type: string
replicated:
properties:
size:
type: integer
requireSafeReplicaSize:
type: boolean
erasureCoded:
properties:
dataChunks:
type: integer
codingChunks:
type: integer
compressionMode:
type: string
enum:
- ""
- none
- passive
- aggressive
- force
parameters:
type: object
dataPool:
properties:
failureDomain:
type: string
crushRoot:
type: string
replicated:
properties:
size:
type: integer
requireSafeReplicaSize:
type: boolean
erasureCoded:
properties:
dataChunks:
type: integer
codingChunks:
type: integer
compressionMode:
type: string
enum:
- ""
- none
- passive
- aggressive
- force
parameters:
type: object
preservePoolsOnDelete:
type: boolean
healthCheck:
properties:
bucket:
properties:
enabled:
type: boolean
interval:
type: string
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephobjectstoreusers.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephObjectStoreUser
listKind: CephObjectStoreUserList
plural: cephobjectstoreusers
singular: cephobjectstoreuser
shortNames:
- rcou
- objectuser
scope: Namespaced
version: v1
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephobjectrealms.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephObjectRealm
listKind: CephObjectRealmList
plural: cephobjectrealms
singular: cephobjectrealm
scope: Namespaced
version: v1
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephobjectzonegroups.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephObjectZoneGroup
listKind: CephObjectZoneGroupList
plural: cephobjectzonegroups
singular: cephobjectzonegroup
scope: Namespaced
version: v1
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephobjectzones.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephObjectZone
listKind: CephObjectZoneList
plural: cephobjectzones
singular: cephobjectzone
scope: Namespaced
version: v1
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: cephblockpools.ceph.rook.io
spec:
group: ceph.rook.io
names:
kind: CephBlockPool
listKind: CephBlockPoolList
plural: cephblockpools
singular: cephblockpool
scope: Namespaced
version: v1
validation:
openAPIV3Schema:
properties:
spec:
properties:
failureDomain:
type: string
crushRoot:
type: string
replicated:
properties:
size:
type: integer
minimum: 0
maximum: 9
targetSizeRatio:
type: number
requireSafeReplicaSize:
type: boolean
erasureCoded:
properties:
dataChunks:
type: integer
minimum: 0
maximum: 9
codingChunks:
type: integer
minimum: 0
maximum: 9
compressionMode:
type: string
enum:
- ""
- none
- passive
- aggressive
- force
enableRBDStats:
description: EnableRBDStats is used to enable gathering of statistics
for all RBD images in the pool
type: boolean
parameters:
type: object
mirroring:
properties:
enabled:
type: boolean
mode:
type: string
enum:
- image
- pool
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: volumes.rook.io
spec:
group: rook.io
names:
kind: Volume
listKind: VolumeList
plural: volumes
singular: volume
shortNames:
- rv
scope: Namespaced
version: v1alpha2
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: objectbuckets.objectbucket.io
spec:
group: objectbucket.io
versions:
- name: v1alpha1
served: true
storage: true
names:
kind: ObjectBucket
listKind: ObjectBucketList
plural: objectbuckets
singular: objectbucket
shortNames:
- ob
- obs
scope: Cluster
subresources:
status: {}
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: objectbucketclaims.objectbucket.io
spec:
versions:
- name: v1alpha1
served: true
storage: true
group: objectbucket.io
names:
kind: ObjectBucketClaim
listKind: ObjectBucketClaimList
plural: objectbucketclaims
singular: objectbucketclaim
shortNames:
- obc
- obcs
scope: Namespaced
subresources:
status: {}
-1
View File
@@ -53,7 +53,6 @@ spec:
port: 80
securePort: 443
instances: 3
allNodes: false
```
Now create the object store.
+6 -1
View File
@@ -80,9 +80,14 @@ func (h *CephInstaller) CreateCephOperator(namespace string) (err error) {
// creating clusterrolebinding for kubeadm env.
h.k8shelper.CreateAnonSystemClusterBinding()
var v1ExtensionsSupported bool
minVersion := "v1.16.0"
if h.k8shelper.VersionAtLeast(minVersion) {
v1ExtensionsSupported = true
}
// creating rook resources
logger.Info("Creating Rook CRDs")
resources := h.Manifests.GetRookCRDs()
resources := h.Manifests.GetRookCRDs(v1ExtensionsSupported)
if _, err = h.k8shelper.KubectlWithStdin(resources, createFromStdinArgs...); err != nil {
return err
}
File diff suppressed because it is too large. Load diff
@@ -26,7 +26,7 @@ type CephManifestsV1_2 struct {
imageTag string
}
func (m *CephManifestsV1_2) GetRookCRDs() string {
func (m *CephManifestsV1_2) GetRookCRDs(v1ExtensionsSupported bool) string {
return `
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
@@ -1874,7 +1874,6 @@ spec:
sslCertificateRef:
port: ` + strconv.Itoa(port) + `
instances: ` + strconv.Itoa(replicaCount) + `
allNodes: false
`
}