Merge pull request #1644 from travisn/remove-tpr-support
Remove support for TPRs and K8s 1.6
This commit is contained in:
43 files changed
+46
-827
No files matched your search
@@ -81,11 +81,11 @@ Below are the settings available, both at the cluster and individual node level,
|
||||
|
||||
Placement configuration for the cluster services. It includes the following keys: `mgr`, `mon`, `osd` and `all`. Each service will have its placement configuration generated by merging the generic configuration under `all` with the most specific one (which will override any attributes).
|
||||
|
||||
A Placement configuration is specified (according to the kubernetes [PodSpec](https://kubernetes.io/docs/api-reference/v1.6/#podspec-v1-core)) as:
|
||||
- `nodeAffinity`: kubernetes [NodeAffinity](https://kubernetes.io/docs/api-reference/v1.6/#nodeaffinity-v1-core)
|
||||
- `podAffinity`: kubernetes [PodAffinity](https://kubernetes.io/docs/api-reference/v1.6/#podaffinity-v1-core)
|
||||
- `podAntiAffinity`: kubernetes [PodAntiAffinity](https://kubernetes.io/docs/api-reference/v1.6/#podantiaffinity-v1-core)
|
||||
- `tolerations`: list of kubernetes [Toleration](https://kubernetes.io/docs/api-reference/v1.6/#toleration-v1-core)
|
||||
A Placement configuration is specified (according to the kubernetes PodSpec) as:
|
||||
- `nodeAffinity`: kubernetes [NodeAffinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature)
|
||||
- `podAffinity`: kubernetes [PodAffinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature)
|
||||
- `podAntiAffinity`: kubernetes [PodAntiAffinity](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature)
|
||||
- `tolerations`: list of kubernetes [Toleration](https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/)
|
||||
|
||||
The `mon` pod does not allow `Pod` affinity or anti-affinity.
|
||||
This is because of the mons having built-in anti-affinity with each other through the operator. The operator chooses which nodes are to run a mon on. Each mon is then tied to a node with a node selector using a hostname.
|
||||
|
||||
@@ -88,7 +88,6 @@ There are many Ceph sub-commands to look at and manipulate Ceph objects, well be
|
||||
* Persistent Volume and/or Claim are failing to be created and bound: [Volume Creation](#volume-creation)
|
||||
* `rook-agent` pod is failing to mount and format the volume: [Rook Agent Mounting](#volume-mounting)
|
||||
* You are using Kubernetes 1.7.x or earlier and the Kubelet has not been restarted after `rook-agent` is in the `Running` status: [Restart Kubelet](#kubelet-restart)
|
||||
* You are using Kubernetes 1.6.x and the attach-detach controller has not been disabled: [Disable attach-detach controller](./quickstart.md#disable-attacher-detacher-controller)
|
||||
|
||||
## Investigation Details
|
||||
If you see some of the symptoms above, it's because the requested Rook storage for your pod is not being created and mounted successfully.
|
||||
@@ -142,8 +141,8 @@ kubectl -n rook-system delete pod -l app=rook-operator
|
||||
```
|
||||
|
||||
### Kubelet Restart
|
||||
#### **Kubernetes 1.7.x and earlier only**
|
||||
If the `rook-agent` pods are all in the `Running` state then another thing to confirm is that **if you are running on Kubernetes 1.7.x or earlier**, the Kubelet must be restarted after the `rook-agent` pods are running.
|
||||
#### **Kubernetes 1.7.x only**
|
||||
If the `rook-agent` pods are all in the `Running` state then another thing to confirm is that **if you are running on Kubernetes 1.7.x**, the Kubelet must be restarted after the `rook-agent` pods are running.
|
||||
|
||||
A symptom of this can be found in the Kubelet's log/journal, with the following error saying `no volume plugin matched`:
|
||||
```console
|
||||
|
||||
@@ -14,7 +14,7 @@ This chart bootstraps a [rook-operator](https://github.com/rook/rook) deployment
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Kubernetes 1.6+
|
||||
- Kubernetes 1.7+
|
||||
|
||||
### RBAC
|
||||
|
||||
|
||||
@@ -9,10 +9,7 @@ Rook can be installed on any existing Kubernetes clusters as long as it meets th
|
||||
|
||||
## Minimum Version
|
||||
|
||||
Kubernetes v1.6 or higher is targeted by Rook (while Rook is in alpha it will track the latest release to use the latest features).
|
||||
|
||||
Support is available for Kubernetes v1.5.2, although your mileage may vary.
|
||||
You will need to use the yaml files from the [1.5 folder](/cluster/examples/kubernetes/1.5).
|
||||
Kubernetes v1.7 or higher is supported by Rook.
|
||||
|
||||
## Privileges and RBAC
|
||||
|
||||
|
||||
@@ -10,8 +10,6 @@ If you do not have Prometheus running, follow the steps below to enable monitori
|
||||
contains a Prometheus instance, it will automatically discover Rooks scrape endpoint using the standard
|
||||
`prometheus.io/scrape` and `prometheus.io/port` annotations.
|
||||
|
||||
Note that these steps work only for Kubernetes 1.7.x or higher. For Kubernetes 1.6.x or older, refer to the Rook 0.5.0 [documentation](https://github.com/rook/rook/blob/release-0.5/Documentation/monitoring.md) and use these [manifest files](https://github.com/rook/rook/tree/release-0.5/cluster/examples/kubernetes/monitoring).
|
||||
|
||||
|
||||
## Prometheus Operator
|
||||
|
||||
|
||||
@@ -57,12 +57,6 @@ spec:
|
||||
|
||||
## Object Store Settings
|
||||
|
||||
### Kind
|
||||
If you are using a version of Kubernetes **earlier than 1.7**, you will need to slightly modify the `kind` to be compatible with TPRs (deprecated in 1.7). Notice the different casing.
|
||||
```yaml
|
||||
kind: Objectstore
|
||||
```
|
||||
|
||||
### Metadata
|
||||
|
||||
- `name`: The name of the object store to create, which will be reflected in the pool and other resource names.
|
||||
|
||||
@@ -13,10 +13,7 @@ from other pods running in your cluster.
|
||||
|
||||
## Minimum Version
|
||||
|
||||
Kubernetes **v1.6** or higher is targeted by Rook (while Rook is in alpha it will track the latest release to use the latest features).
|
||||
|
||||
Support is available for Kubernetes **v1.5.2**, although your mileage may vary.
|
||||
You will need to use the yaml files from the [1.5 folder](/cluster/examples/kubernetes/1.5).
|
||||
Kubernetes **v1.7** or higher is supported by Rook.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -51,15 +48,10 @@ You can also deploy the operator with the [Rook Helm Chart](helm-operator.md).
|
||||
|
||||
---
|
||||
### **Restart Kubelet**
|
||||
**(K8S 1.7.x and older only)**
|
||||
**(K8S 1.7.x only)**
|
||||
|
||||
For versions of Kubernetes prior to 1.8, the Kubelet process on all nodes will require a restart after the Rook operator and Rook agents have been deployed. As part of their initial setup, the Rook agents deploy and configure a Flexvolume plugin in order to integrate with Kubernetes' volume controller framework. In Kubernetes v1.8+, the [dynamic Flexvolume plugin discovery](https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md#dynamic-plugin-discovery) will find and initialize our plugin, but in older versions of Kubernetes a manual restart of the Kubelet will be required.
|
||||
|
||||
### **Disable Attacher-detacher controller**
|
||||
**(K8S 1.6.x only)**
|
||||
|
||||
For Kubernetes 1.6, it is also necessary to pass the `--enable-controller-attach-detach=false` flag to Kubelet when you restart it. This is a workaround for a [Kubernetes issue](https://github.com/kubernetes/kubernetes/issues/47109) that only affects 1.6.
|
||||
|
||||
---
|
||||
|
||||
## Create a Rook Cluster
|
||||
|
||||
Vendored
-2
@@ -67,8 +67,6 @@ pipeline {
|
||||
stash name: 'repo-amd64',includes: 'rook-amd64.tar,build/common.sh,_output/tests/linux_amd64/,_output/charts/,tests/scripts/'
|
||||
script{
|
||||
def data = [
|
||||
"gce_1.6.1": "v1.6.1",
|
||||
"gce_1.6.x": "v1.6.13",
|
||||
"aws_1.7.x": "v1.7.11",
|
||||
"aws_1.8.x": "v1.8.5",
|
||||
"gce_1.9.x": "v1.9.1"
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
- Rook-Operator no longer creates the resources CRD's or TPR's at the runtime. Instead, those resources are provisioned during deployment via `helm` or `kubectl`.
|
||||
|
||||
## Breaking Changes
|
||||
- Removed support for Kubernetes 1.6, including the legacy Third Party Resources (TPRs).
|
||||
|
||||
### Removal of the API service and rookctl tool
|
||||
The [REST API service](https://github.com/rook/rook/issues/1122) has been removed. All cluster configuration is now accomplished through the
|
||||
|
||||
@@ -42,18 +42,6 @@ rules:
|
||||
- create
|
||||
- update
|
||||
- delete
|
||||
- apiGroups:
|
||||
- extensions
|
||||
resources:
|
||||
- thirdpartyresources
|
||||
resourceNames:
|
||||
- cluster.rook.io
|
||||
- pool.rook.io
|
||||
- objectstore.rook.io
|
||||
- filesystem.rook.io
|
||||
- volumeattachment.rook.io
|
||||
verbs:
|
||||
- get
|
||||
- apiGroups:
|
||||
- apiextensions.k8s.io
|
||||
resources:
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
{{- if .Capabilities.APIVersions.Has "apiextensions.k8s.io/v1beta1" }}
|
||||
---
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
@@ -70,46 +69,3 @@ spec:
|
||||
scope: Namespaced
|
||||
version: v1alpha1
|
||||
---
|
||||
{{- else }}
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for cluster
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: cluster.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for filesystem
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: filesystem.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for objectstore
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: objectstore.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for pool
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: pool.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for volumeattachment
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: volumeattachment.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
{{- end }}
|
||||
@@ -22,21 +22,12 @@ spec:
|
||||
command:
|
||||
- ./kubectl
|
||||
- get
|
||||
{{- if .Capabilities.APIVersions.Has "apiextensions.k8s.io/v1beta1" }}
|
||||
- customresourcedefinitions
|
||||
- clusters.rook.io
|
||||
- pools.rook.io
|
||||
- objectstores.rook.io
|
||||
- filesystems.rook.io
|
||||
- volumeattachments.rook.io
|
||||
{{- else }}
|
||||
- thirdpartyresources
|
||||
- cluster.rook.io
|
||||
- pool.rook.io
|
||||
- objectstore.rook.io
|
||||
- filesystem.rook.io
|
||||
- volumeattachment.rook.io
|
||||
{{- end }}
|
||||
restartPolicy: OnFailure
|
||||
{{- if .Values.rbacEnable }}
|
||||
serviceAccountName: rook-operator
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
../kube-registry.yaml
|
||||
@@ -1,60 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: wordpress-mysql
|
||||
labels:
|
||||
app: wordpress
|
||||
spec:
|
||||
ports:
|
||||
- port: 3306
|
||||
selector:
|
||||
app: wordpress
|
||||
tier: mysql
|
||||
clusterIP: None
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: mysql-pv-claim
|
||||
annotations:
|
||||
volume.beta.kubernetes.io/storage-class: rook-block
|
||||
labels:
|
||||
app: wordpress
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: wordpress-mysql
|
||||
labels:
|
||||
app: wordpress
|
||||
spec:
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: wordpress
|
||||
tier: mysql
|
||||
spec:
|
||||
containers:
|
||||
- image: mysql:5.6
|
||||
name: mysql
|
||||
env:
|
||||
- name: MYSQL_ROOT_PASSWORD
|
||||
value: changeme
|
||||
ports:
|
||||
- containerPort: 3306
|
||||
name: mysql
|
||||
volumeMounts:
|
||||
- name: mysql-persistent-storage
|
||||
mountPath: /var/lib/mysql
|
||||
volumes:
|
||||
- name: mysql-persistent-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: mysql-pv-claim
|
||||
@@ -1 +0,0 @@
|
||||
../rgw-external.yaml
|
||||
@@ -1 +0,0 @@
|
||||
../rook-client.yaml
|
||||
@@ -1 +0,0 @@
|
||||
../rook-cluster.yaml
|
||||
@@ -1,37 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: rook-system
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: rook-operator
|
||||
namespace: rook-system
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
name: rook-operator
|
||||
spec:
|
||||
containers:
|
||||
- name: rook-operator
|
||||
image: rook/rook:master
|
||||
args: ["operator"]
|
||||
env:
|
||||
# The interval to check if every mon is in the quorum.
|
||||
- name: ROOK_MON_HEALTHCHECK_INTERVAL
|
||||
value: "45s"
|
||||
# The duration to wait before trying to failover or remove/replace the
|
||||
# current mon with a new mon (useful for compensating flapping network).
|
||||
- name: ROOK_MON_OUT_TIMEOUT
|
||||
value: "300s"
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
- name: NODE_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: spec.nodeName
|
||||
@@ -1 +0,0 @@
|
||||
../rook-pool.yaml
|
||||
@@ -1,27 +0,0 @@
|
||||
apiVersion: rook.io/v1alpha1
|
||||
kind: Pool
|
||||
metadata:
|
||||
name: replicapool
|
||||
namespace: rook
|
||||
spec:
|
||||
replicated:
|
||||
size: 1
|
||||
# For an erasure-coded pool, comment out the replication size above and uncomment the following settings.
|
||||
# Make sure you have enough OSDs to support the replica size or erasure code chunks.
|
||||
#erasureCoded:
|
||||
# dataChunks: 2
|
||||
# codingChunks: 1
|
||||
---
|
||||
apiVersion: storage.k8s.io/v1beta1
|
||||
kind: StorageClass
|
||||
metadata:
|
||||
name: rook-block
|
||||
provisioner: rook.io/block
|
||||
parameters:
|
||||
pool: replicapool
|
||||
# Specify the Rook cluster from which to create volumes.
|
||||
# If not specified, it will use `rook` as the name of the cluster.
|
||||
# This is also the namespace where the cluster will be
|
||||
clusterName: rook
|
||||
# Specify the filesystem type of the volume. If not specified, it will use `ext4`.
|
||||
# fstype: ext4
|
||||
@@ -1 +0,0 @@
|
||||
../rook-tools.yaml
|
||||
@@ -1,62 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: wordpress
|
||||
labels:
|
||||
app: wordpress
|
||||
spec:
|
||||
ports:
|
||||
- port: 80
|
||||
selector:
|
||||
app: wordpress
|
||||
tier: frontend
|
||||
type: LoadBalancer
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: wp-pv-claim
|
||||
annotations:
|
||||
volume.beta.kubernetes.io/storage-class: rook-block
|
||||
labels:
|
||||
app: wordpress
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 20Gi
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: wordpress
|
||||
labels:
|
||||
app: wordpress
|
||||
spec:
|
||||
strategy:
|
||||
type: Recreate
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: wordpress
|
||||
tier: frontend
|
||||
spec:
|
||||
containers:
|
||||
- image: wordpress:4.6.1-apache
|
||||
name: wordpress
|
||||
env:
|
||||
- name: WORDPRESS_DB_HOST
|
||||
value: wordpress-mysql
|
||||
- name: WORDPRESS_DB_PASSWORD
|
||||
value: changeme
|
||||
ports:
|
||||
- containerPort: 80
|
||||
name: wordpress
|
||||
volumeMounts:
|
||||
- name: wordpress-persistent-storage
|
||||
mountPath: /var/www/html
|
||||
volumes:
|
||||
- name: wordpress-persistent-storage
|
||||
persistentVolumeClaim:
|
||||
claimName: wp-pv-claim
|
||||
@@ -1,45 +0,0 @@
|
||||
apiVersion: rook.io/v1alpha1
|
||||
# For earlier versions than K8s 1.7, the kind must be Objectstore
|
||||
kind: Objectstore
|
||||
metadata:
|
||||
name: my-store
|
||||
namespace: rook
|
||||
spec:
|
||||
# The pool spec used to create the metadata pools
|
||||
metadataPool:
|
||||
replicated:
|
||||
size: 2
|
||||
# The pool spec used to create the data pool
|
||||
dataPool:
|
||||
erasureCoded:
|
||||
dataChunks: 2
|
||||
codingChunks: 1
|
||||
# The gaeteway service configuration
|
||||
gateway:
|
||||
# type of the gateway (s3)
|
||||
type: s3
|
||||
# A reference to the secret in the rook namespace where the ssl certificate is stored
|
||||
sslCertificateRef:
|
||||
# The port that RGW pods will listen on (http)
|
||||
port: 80
|
||||
# The port that RGW pods will listen on (https). An ssl certificate is required.
|
||||
securePort: 443
|
||||
# The number of pods in the rgw deployment (ignored if allNodes=true)
|
||||
instances: 1
|
||||
# Whether the rgw pods should be deployed on all nodes as a daemonset
|
||||
allNodes: false
|
||||
# The affinity rules to apply to the rgw deployment or daemonset.
|
||||
placement:
|
||||
# nodeAffinity:
|
||||
# requiredDuringSchedulingIgnoredDuringExecution:
|
||||
# nodeSelectorTerms:
|
||||
# - matchExpressions:
|
||||
# - key: role
|
||||
# operator: In
|
||||
# values:
|
||||
# - rgw-node
|
||||
# tolerations:
|
||||
# - key: rgw-node
|
||||
# operator: Exists
|
||||
# podAffinity:
|
||||
# podAntiAffinity:
|
||||
@@ -1,173 +0,0 @@
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for cluster
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: cluster.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for filesystem
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: filesystem.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for objectstore
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: objectstore.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for pool
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: pool.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for volumeattachment
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: volumeattachment.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: rook-system
|
||||
---
|
||||
kind: ClusterRole
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: rook-operator
|
||||
rules:
|
||||
- apiGroups:
|
||||
- ""
|
||||
resources:
|
||||
- namespaces
|
||||
- serviceaccounts
|
||||
- secrets
|
||||
- pods
|
||||
- services
|
||||
- nodes
|
||||
- nodes/proxy
|
||||
- configmaps
|
||||
- events
|
||||
- persistentvolumes
|
||||
- persistentvolumeclaims
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- patch
|
||||
- create
|
||||
- update
|
||||
- delete
|
||||
- apiGroups:
|
||||
- rbac.authorization.k8s.io
|
||||
resources:
|
||||
- clusterroles
|
||||
- clusterrolebindings
|
||||
- roles
|
||||
- rolebindings
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- create
|
||||
- update
|
||||
- delete
|
||||
- apiGroups:
|
||||
- storage.k8s.io
|
||||
resources:
|
||||
- storageclasses
|
||||
verbs:
|
||||
- get
|
||||
- list
|
||||
- watch
|
||||
- delete
|
||||
- apiGroups:
|
||||
- rook.io
|
||||
resources:
|
||||
- "*"
|
||||
verbs:
|
||||
- "*"
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: rook-operator
|
||||
namespace: rook-system
|
||||
---
|
||||
kind: ClusterRoleBinding
|
||||
apiVersion: rbac.authorization.k8s.io/v1beta1
|
||||
metadata:
|
||||
name: rook-operator
|
||||
namespace: rook-system
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: rook-operator
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: rook-operator
|
||||
namespace: rook-system
|
||||
---
|
||||
apiVersion: apps/v1beta1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: rook-operator
|
||||
namespace: rook-system
|
||||
spec:
|
||||
replicas: 1
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: rook-operator
|
||||
spec:
|
||||
serviceAccountName: rook-operator
|
||||
containers:
|
||||
- name: rook-operator
|
||||
image: rook/rook:master
|
||||
args: ["operator"]
|
||||
env:
|
||||
# To disable RBAC, uncomment the following:
|
||||
# - name: RBAC_ENABLED
|
||||
# value: "false"
|
||||
# Rook Agent toleration. Will tolerate all taints with all keys.
|
||||
# Choose between NoSchedule, PreferNoSchedule and NoExecute:
|
||||
# - name: AGENT_TOLERATION
|
||||
# value: "NoSchedule"
|
||||
# (Optional) Rook Agent toleration key. Set this to the key of the taint you want to tolerate
|
||||
# - name: AGENT_TOLERATION_KEY
|
||||
# value: "<KeyOfTheTaintToTolerate>"
|
||||
# Set the path where the Rook agent can find the flex volumes
|
||||
# - name: FLEXVOLUME_DIR_PATH
|
||||
# value: "<PathToFlexVolumes>"
|
||||
# The interval to check if every mon is in the quorum.
|
||||
- name: ROOK_MON_HEALTHCHECK_INTERVAL
|
||||
value: "45s"
|
||||
# The duration to wait before trying to failover or remove/replace the
|
||||
# current mon with a new mon (useful for compensating flapping network).
|
||||
- name: ROOK_MON_OUT_TIMEOUT
|
||||
value: "300s"
|
||||
- name: NODE_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: spec.nodeName
|
||||
- name: POD_NAME
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.name
|
||||
- name: POD_NAMESPACE
|
||||
valueFrom:
|
||||
fieldRef:
|
||||
fieldPath: metadata.namespace
|
||||
@@ -1,5 +1,4 @@
|
||||
apiVersion: rook.io/v1alpha1
|
||||
# For earlier versions than K8s 1.7, the kind must be Objectstore
|
||||
kind: ObjectStore
|
||||
metadata:
|
||||
name: my-store
|
||||
|
||||
@@ -99,7 +99,7 @@ Furthermore, if users want to consume a Rook volume outside of the default names
|
||||
|
||||
Both of these issues are handled by this proposed design for the Rook agent, greatly streamlining the Rook storage experience in Kubernetes and removing the common causes for errors and failure. This is a big win for our users.
|
||||
|
||||
This proposed design also allows Rook to normalize on a single path for providing persistent volumes across all Kubernetes versions that it supports (1.5+).
|
||||
This proposed design also allows Rook to normalize on a single path for providing persistent volumes across all Kubernetes versions that it supports.
|
||||
Since this design is fully out-of-tree for Kubernetes, it is not tied to Kubernetes release timelines.
|
||||
Updates, fixes and new features can be released through the normal Rook release process and timelines.
|
||||
|
||||
|
||||
@@ -61,8 +61,6 @@ func addKnownTypes(scheme *runtime.Scheme) error {
|
||||
&FilesystemList{},
|
||||
&ObjectStore{},
|
||||
&ObjectStoreList{},
|
||||
// TODO: Remove ObjectstoreList after TPRs are no longer supported
|
||||
&ObjectstoreList{},
|
||||
&VolumeAttachment{},
|
||||
&VolumeAttachmentList{},
|
||||
)
|
||||
|
||||
@@ -288,15 +288,6 @@ type ObjectStoreList struct {
|
||||
Items []ObjectStore `json:"items"`
|
||||
}
|
||||
|
||||
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
|
||||
|
||||
// ObjectstoreList is the definition of a list of object stores for TPRs (pre-1.7)
|
||||
type ObjectstoreList struct {
|
||||
metav1.TypeMeta `json:",inline"`
|
||||
metav1.ListMeta `json:"metadata"`
|
||||
Items []ObjectStore `json:"items"`
|
||||
}
|
||||
|
||||
// ObjectStoreSpec represent the spec of a pool
|
||||
type ObjectStoreSpec struct {
|
||||
// The metadata pool settings
|
||||
|
||||
@@ -430,40 +430,6 @@ func (in *ObjectStoreSpec) DeepCopy() *ObjectStoreSpec {
|
||||
return out
|
||||
}
|
||||
|
||||
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
|
||||
func (in *ObjectstoreList) DeepCopyInto(out *ObjectstoreList) {
|
||||
*out = *in
|
||||
out.TypeMeta = in.TypeMeta
|
||||
out.ListMeta = in.ListMeta
|
||||
if in.Items != nil {
|
||||
in, out := &in.Items, &out.Items
|
||||
*out = make([]ObjectStore, len(*in))
|
||||
for i := range *in {
|
||||
(*in)[i].DeepCopyInto(&(*out)[i])
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ObjectstoreList.
|
||||
func (in *ObjectstoreList) DeepCopy() *ObjectstoreList {
|
||||
if in == nil {
|
||||
return nil
|
||||
}
|
||||
out := new(ObjectstoreList)
|
||||
in.DeepCopyInto(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
|
||||
func (in *ObjectstoreList) DeepCopyObject() runtime.Object {
|
||||
if c := in.DeepCopy(); c != nil {
|
||||
return c
|
||||
} else {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
|
||||
func (in *Placement) DeepCopyInto(out *Placement) {
|
||||
*out = *in
|
||||
|
||||
@@ -17,25 +17,15 @@ limitations under the License.
|
||||
package attachment
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
|
||||
"github.com/coreos/pkg/capnslog"
|
||||
rookalpha "github.com/rook/rook/pkg/apis/rook.io/v1alpha1"
|
||||
"github.com/rook/rook/pkg/clusterd"
|
||||
"github.com/rook/rook/pkg/operator/k8sutil"
|
||||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
|
||||
"k8s.io/kubernetes/pkg/util/version"
|
||||
)
|
||||
|
||||
var logger = capnslog.NewPackageLogger("github.com/rook/rook", "rook-agent-crd")
|
||||
|
||||
const (
|
||||
serverVersionV170 = "v1.7.0"
|
||||
)
|
||||
|
||||
// Attachment handles custom resource VolumeAttachment storage operations.
|
||||
// This interface goes away when there is no longer a need to support TPRs since
|
||||
// we can call the RookClientset directly.
|
||||
type Attachment interface {
|
||||
Create(volumeAttachment *rookalpha.VolumeAttachment) error
|
||||
Get(namespace, name string) (*rookalpha.VolumeAttachment, error)
|
||||
@@ -51,22 +41,7 @@ type crd struct {
|
||||
|
||||
// CreateController creates a new controller for volume attachment
|
||||
func New(context *clusterd.Context) (Attachment, error) {
|
||||
|
||||
// CRD is available on v1.7.0. TPR became deprecated on v1.7.0
|
||||
// Remove this code when TPR is not longer supported
|
||||
kubeVersion, err := k8sutil.GetK8SVersion(context.Clientset)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Error getting server version: %v", err)
|
||||
}
|
||||
if kubeVersion.AtLeast(version.MustParseSemantic(serverVersionV170)) {
|
||||
return &crd{
|
||||
context: context,
|
||||
}, nil
|
||||
}
|
||||
|
||||
return &tpr{
|
||||
clientset: context.Clientset,
|
||||
}, nil
|
||||
return &crd{context: context}, nil
|
||||
}
|
||||
|
||||
// Get queries the VolumeAttachment CRD from Kubernetes
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
/*
|
||||
Copyright 2017 The Rook Authors. All rights reserved.
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
*/
|
||||
|
||||
package attachment
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
|
||||
"github.com/coreos/pkg/capnslog"
|
||||
rookalpha "github.com/rook/rook/pkg/apis/rook.io/v1alpha1"
|
||||
"k8s.io/client-go/kubernetes"
|
||||
)
|
||||
|
||||
const (
|
||||
tprKind = "Volumeattachment"
|
||||
)
|
||||
|
||||
var tprlogger = capnslog.NewPackageLogger("github.com/rook/rook", "rook-agent-tpr")
|
||||
|
||||
// TPR is a controller to manage VolumeAttachment TPR objects
|
||||
// CRD is available on v1.7.0. TPR became deprecated on v1.7.0
|
||||
// Remove this code when TPR is not longer supported
|
||||
type tpr struct {
|
||||
clientset kubernetes.Interface
|
||||
}
|
||||
|
||||
// Get queries the VolumeAttachment TPR from Kubernetes
|
||||
func (t *tpr) Get(namespace, name string) (*rookalpha.VolumeAttachment, error) {
|
||||
|
||||
var result rookalpha.VolumeAttachment
|
||||
uri := fmt.Sprintf("apis/%s/%s/namespaces/%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version, namespace, CustomResourceNamePlural)
|
||||
return &result, t.clientset.CoreV1().RESTClient().Get().
|
||||
RequestURI(uri).
|
||||
Name(name).
|
||||
Do().
|
||||
Into(&result)
|
||||
}
|
||||
|
||||
// List lists all the volume attachment TPR resources in the given namespace
|
||||
func (t *tpr) List(namespace string) (*rookalpha.VolumeAttachmentList, error) {
|
||||
|
||||
var result rookalpha.VolumeAttachmentList
|
||||
uri := fmt.Sprintf("apis/%s/%s/namespaces/%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version, namespace, CustomResourceNamePlural)
|
||||
return &result, t.clientset.CoreV1().RESTClient().Get().
|
||||
RequestURI(uri).
|
||||
Do().
|
||||
Into(&result)
|
||||
}
|
||||
|
||||
// Create creates the volume attach TPR resource in Kubernetes
|
||||
func (t *tpr) Create(volumeAttachment *rookalpha.VolumeAttachment) error {
|
||||
volumeAttachment.APIVersion = fmt.Sprintf("%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version)
|
||||
volumeAttachment.Kind = tprKind
|
||||
body, _ := json.Marshal(volumeAttachment)
|
||||
uri := fmt.Sprintf("apis/%s/%s/namespaces/%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version, volumeAttachment.Namespace, CustomResourceNamePlural)
|
||||
return t.clientset.CoreV1().RESTClient().Post().
|
||||
RequestURI(uri).
|
||||
Body(body).
|
||||
Do().Error()
|
||||
}
|
||||
|
||||
// Update updates VolumeAttachment TPR resource
|
||||
func (t *tpr) Update(volumeAttachment *rookalpha.VolumeAttachment) error {
|
||||
volumeAttachment.APIVersion = fmt.Sprintf("%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version)
|
||||
volumeAttachment.Kind = tprKind
|
||||
body, _ := json.Marshal(volumeAttachment)
|
||||
uri := fmt.Sprintf("apis/%s/%s/namespaces/%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version, volumeAttachment.Namespace, CustomResourceNamePlural)
|
||||
err := t.clientset.CoreV1().RESTClient().Put().
|
||||
RequestURI(uri).
|
||||
Name(volumeAttachment.Name).
|
||||
Body(body).
|
||||
Do().Error()
|
||||
if err != nil {
|
||||
tprlogger.Errorf("failed to update VolumeAttachment CRD. %+v", err)
|
||||
return err
|
||||
}
|
||||
tprlogger.Infof("updated Volumeattach TPR %s", volumeAttachment.ObjectMeta.Name)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Delete deletes the volume attach TPR resource in Kubernetes
|
||||
func (t *tpr) Delete(namespace, name string) error {
|
||||
uri := fmt.Sprintf("apis/%s/%s/namespaces/%s/%s", rookalpha.CustomResourceGroup, rookalpha.Version, namespace, CustomResourceNamePlural)
|
||||
return t.clientset.CoreV1().RESTClient().Delete().
|
||||
RequestURI(uri).
|
||||
Name(name).
|
||||
Do().
|
||||
Error()
|
||||
}
|
||||
@@ -44,7 +44,6 @@ import (
|
||||
"k8s.io/apimachinery/pkg/types"
|
||||
"k8s.io/apimachinery/pkg/util/wait"
|
||||
"k8s.io/client-go/tools/cache"
|
||||
"k8s.io/kubernetes/pkg/util/version"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -297,17 +296,9 @@ func (c *ClusterController) onDelete(obj interface{}) {
|
||||
}
|
||||
|
||||
func (c *ClusterController) addFinalizer(clust *rookalpha.Cluster) error {
|
||||
kubeVersion, err := k8sutil.GetK8SVersion(c.context.Clientset)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error getting server version: %v", err)
|
||||
}
|
||||
if !kubeVersion.AtLeast(version.MustParseSemantic(k8sutil.FirstCRDVersion)) {
|
||||
logger.Infof("not adding finalizer for cluster TPR")
|
||||
return nil
|
||||
}
|
||||
|
||||
// get the latest cluster object since we probably updated it before we got to this point (e.g. by updating its status)
|
||||
clust, err = c.context.RookClientset.RookV1alpha1().Clusters(clust.Namespace).Get(clust.Name, metav1.GetOptions{})
|
||||
clust, err := c.context.RookClientset.RookV1alpha1().Clusters(clust.Namespace).Get(clust.Name, metav1.GetOptions{})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -45,8 +45,6 @@ const (
|
||||
PodNamespaceEnvVar = "POD_NAMESPACE"
|
||||
// NodeNameEnvVar is the env variable for getting the node via downward api
|
||||
NodeNameEnvVar = "NODE_NAME"
|
||||
// FirstCRDVersion is the first K8s version with CRDs to replace TPRs
|
||||
FirstCRDVersion = "v1.7.0"
|
||||
)
|
||||
|
||||
// GetK8SVersion gets the version of the running K8S cluster
|
||||
|
||||
@@ -97,8 +97,6 @@ func (o *Operator) Run() error {
|
||||
signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM)
|
||||
|
||||
// Run volume provisioner
|
||||
// The controller needs to know what the server version is because out-of-tree
|
||||
// provisioners aren't officially supported until 1.5
|
||||
serverVersion, err := o.context.Clientset.Discovery().ServerVersion()
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error getting server version: %v", err)
|
||||
|
||||
+2
-6
@@ -56,11 +56,7 @@ tests/scripts/kubeadm.sh clean
|
||||
#### Alternate Kubernetes Versions
|
||||
These two scripts can install any version of Kubernetes you wish based on the `KUBE_VERSION` environment variable.
|
||||
To use an alternate version, simply set this variable before running the relevant `up` command from above.
|
||||
For example, if you wanted to use `v1.6.7`, you would run `export KUBE_VERSION=v1.6.7` first before running `up`.
|
||||
|
||||
By default, the following versions of Kubernetes are used:
|
||||
* `minikube.sh`: `v1.9.1`
|
||||
* `kubeadm.sh`: `v1.8.5`
|
||||
For example, if you wanted to use `v1.9.6`, you would run `export KUBE_VERSION=v1.9.6` first before running `up`.
|
||||
|
||||
### Install Helm
|
||||
Use [helm.sh](/tests/scripts/helm.sh) to install helm and set up rook charts defined under `_output/charts` (generated by build).
|
||||
@@ -101,7 +97,7 @@ In addition to standard go tests parameters, the following custom parameters are
|
||||
Parameter | Description | Possible values | Default
|
||||
--- |--- | --- | ---
|
||||
rook_platform| platform rook needs to be installed on | kubernetes | kubernetes
|
||||
k8s_version | version of Kubernetes to be installed | v1.6+ | v1.8.5
|
||||
k8s_version | version of Kubernetes to be installed | v1.7+ | v1.8.5
|
||||
rook_image | rook image name to be installed | valid image name | rook/rook
|
||||
toolbox_image | toolbox image name to be installed | valid image name | rook/toolbox
|
||||
skip_install_rook | skips installing rook (if already installed) | true or false | false
|
||||
|
||||
@@ -40,13 +40,9 @@ func CreateObjectOperation(k8sh *utils.K8sHelper) *ObjectOperation {
|
||||
// ObjectCreate Function to create a object store in rook
|
||||
func (ro *ObjectOperation) Create(namespace, storeName string, replicaCount int32) error {
|
||||
|
||||
kind := "ObjectStore"
|
||||
if !ro.k8sh.VersionAtLeast("1.7.0") {
|
||||
kind = "Objectstore"
|
||||
}
|
||||
logger.Infof("creating the object store via CRD")
|
||||
storeSpec := fmt.Sprintf(`apiVersion: rook.io/v1alpha1
|
||||
kind: %s
|
||||
kind: ObjectStore
|
||||
metadata:
|
||||
name: %s
|
||||
namespace: %s
|
||||
@@ -64,7 +60,7 @@ spec:
|
||||
securePort:
|
||||
instances: %d
|
||||
allNodes: false
|
||||
`, kind, storeName, namespace, rgwPort, replicaCount)
|
||||
`, storeName, namespace, rgwPort, replicaCount)
|
||||
|
||||
if _, err := ro.k8sh.ResourceOperation("create", storeSpec); err != nil {
|
||||
return err
|
||||
@@ -80,10 +76,6 @@ spec:
|
||||
|
||||
func (ro *ObjectOperation) Delete(namespace, storeName string, replicaCount int32) error {
|
||||
|
||||
if !ro.k8sh.VersionAtLeast("1.7.0") {
|
||||
// The operator fails to process the Objectstore TPR upon deletion. Support for 1.6 is going away, so just disable this test in that case.
|
||||
return nil
|
||||
}
|
||||
logger.Infof("Deleting the object store via CRD")
|
||||
storeSpec := fmt.Sprintf(`apiVersion: rook.io/v1alpha1
|
||||
kind: ObjectStore
|
||||
|
||||
@@ -29,48 +29,6 @@ func NewK8sInstallData() *InstallData {
|
||||
return &InstallData{}
|
||||
}
|
||||
|
||||
func (i *InstallData) GetRookTPRs(namespace string) string {
|
||||
return `apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for cluster
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: cluster.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for filesystem
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: filesystem.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for objectstore
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: objectstore.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for pool
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: pool.rook.io
|
||||
versions:
|
||||
- name: v1alpha1
|
||||
---
|
||||
apiVersion: extensions/v1beta1
|
||||
description: ThirdPartyResource for volumeattachment
|
||||
kind: ThirdPartyResource
|
||||
metadata:
|
||||
name: volumeattachment.rook.io
|
||||
versions:
|
||||
- name: v1alpha1`
|
||||
}
|
||||
|
||||
func (i *InstallData) GetRookCRDs(namespace string) string {
|
||||
return `apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
|
||||
@@ -65,15 +65,9 @@ type InstallHelper struct {
|
||||
|
||||
func (h *InstallHelper) CreateK8sRookResources(namespace string) (err error) {
|
||||
var resources string
|
||||
if h.k8shelper.VersionAtLeast("v1.7.0") {
|
||||
logger.Info("Creating Rook CRD's")
|
||||
logger.Info("Creating Rook CRD's")
|
||||
|
||||
resources = h.installData.GetRookCRDs(namespace)
|
||||
} else {
|
||||
logger.Info("Creating Rook TPR's")
|
||||
|
||||
resources = h.installData.GetRookTPRs(namespace)
|
||||
}
|
||||
resources = h.installData.GetRookCRDs(namespace)
|
||||
|
||||
_, err = h.k8shelper.KubectlWithStdin(resources, createArgs...)
|
||||
|
||||
@@ -98,14 +92,8 @@ func (h *InstallHelper) CreateK8sRookOperator(namespace string) (err error) {
|
||||
return fmt.Errorf("Failed to create rook-operator pod : %v ", err)
|
||||
}
|
||||
|
||||
if h.k8shelper.VersionAtLeast("v1.7.0") {
|
||||
if !h.k8shelper.IsCRDPresent(rookOperatorCreatedCrd) {
|
||||
return fmt.Errorf("Failed to start Rook Operator; k8s CustomResourceDefinition did not appear")
|
||||
}
|
||||
} else {
|
||||
if !h.k8shelper.IsThirdPartyResourcePresent(rookOperatorCreatedTpr) {
|
||||
return fmt.Errorf("Failed to start Rook Operator; k8s thirdpartyresource did not appear")
|
||||
}
|
||||
if !h.k8shelper.IsCRDPresent(rookOperatorCreatedCrd) {
|
||||
return fmt.Errorf("Failed to start Rook Operator; k8s CustomResourceDefinition did not appear")
|
||||
}
|
||||
|
||||
logger.Infof("Rook Operator started")
|
||||
@@ -130,14 +118,8 @@ func (h *InstallHelper) CreateK8sRookOperatorViaHelm(namespace string) error {
|
||||
|
||||
}
|
||||
|
||||
if h.k8shelper.VersionAtLeast("v1.7.0") {
|
||||
if !h.k8shelper.IsCRDPresent(rookOperatorCreatedCrd) {
|
||||
return fmt.Errorf("Failed to start Rook Operator; k8s CustomResourceDefinition did not appear")
|
||||
}
|
||||
} else {
|
||||
if !h.k8shelper.IsThirdPartyResourcePresent(rookOperatorCreatedTpr) {
|
||||
return fmt.Errorf("Failed to start Rook Operator; k8s thirdpartyresource did not appear")
|
||||
}
|
||||
if !h.k8shelper.IsCRDPresent(rookOperatorCreatedCrd) {
|
||||
return fmt.Errorf("Failed to start Rook Operator; k8s CustomResourceDefinition did not appear")
|
||||
}
|
||||
|
||||
return nil
|
||||
@@ -349,13 +331,8 @@ func (h *InstallHelper) UninstallRookFromMultipleNS(helmInstalled bool, systemNa
|
||||
}
|
||||
|
||||
logger.Infof("removing the operator from namespace %s", systemNamespace)
|
||||
if h.k8shelper.VersionAtLeast("v1.7.0") {
|
||||
_, err = h.k8shelper.DeleteResource([]string{"crd", "clusters.rook.io", "pools.rook.io", "objectstores.rook.io", "filesystems.rook.io", "volumeattachments.rook.io"})
|
||||
h.checkError(err, "cannot delete CRDs")
|
||||
} else {
|
||||
_, err = h.k8shelper.DeleteResource([]string{"thirdpartyresources", "cluster.rook.io", "pool.rook.io", "objectstore.rook.io", "filesystem.rook.io", "volumeattachment.rook.io"})
|
||||
h.checkError(err, "cannot delete TPRs")
|
||||
}
|
||||
_, err = h.k8shelper.DeleteResource([]string{"crd", "clusters.rook.io", "pools.rook.io", "objectstores.rook.io", "filesystems.rook.io", "volumeattachments.rook.io"})
|
||||
h.checkError(err, "cannot delete CRDs")
|
||||
|
||||
if helmInstalled {
|
||||
err = h.helmHelper.DeleteLocalRookHelmChart(helmDeployName)
|
||||
@@ -381,7 +358,6 @@ func (h *InstallHelper) checkError(err error, message string) {
|
||||
|
||||
func (h *InstallHelper) waitForCustomResourceDeletion(namespace string) error {
|
||||
if !h.k8shelper.VersionAtLeast("v1.8.0") {
|
||||
// v1.6 does not have finalizers for TPRs
|
||||
// v1.7 has an intermittent issue with long delay to delete resources so we will skip waiting
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -453,23 +453,6 @@ func (k8sh *K8sHelper) GetService(servicename string, namespace string) (*v1.Ser
|
||||
return result, nil
|
||||
}
|
||||
|
||||
//IsThirdPartyResourcePresent returns true if Third party resource is present
|
||||
func (k8sh *K8sHelper) IsThirdPartyResourcePresent(tprname string) bool {
|
||||
args := []string{"get", "thirdpartyresources", tprname}
|
||||
inc := 0
|
||||
for inc < RetryLoop {
|
||||
_, err := k8sh.Kubectl(args...)
|
||||
if err == nil {
|
||||
k8slogger.Infof("Found the thirdparty resource: " + tprname)
|
||||
return true
|
||||
}
|
||||
time.Sleep(RetryInterval * time.Second)
|
||||
inc++
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
//IsCRDPresent returns true if custom resource definition is present
|
||||
func (k8sh *K8sHelper) IsCRDPresent(crdName string) bool {
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ KUBE_INSTALL_VERSION="${KUBE_VERSION/v/$null_str}"-00
|
||||
sudo swapoff -a
|
||||
|
||||
# init flexvolume
|
||||
if [[ $KUBE_VERSION == v1.5* ]] || [[ $KUBE_VERSION == v1.6* ]] || [[ $KUBE_VERSION == v1.7* ]] ;
|
||||
if [[ $KUBE_VERSION == v1.7* ]] ;
|
||||
then
|
||||
sudo mkdir -p /usr/libexec/kubernetes/kubelet-plugins/volume/exec/rook.io~rook
|
||||
cat << EOF | sudo tee -a /usr/libexec/kubernetes/kubelet-plugins/volume/exec/rook.io~rook/rook
|
||||
|
||||
+16
-18
@@ -20,14 +20,7 @@ usage(){
|
||||
#install k8s master node
|
||||
install_master(){
|
||||
|
||||
# This is needed on K8S 1.6 to fix a regression. https://github.com/kubernetes/kubernetes/issues/47109
|
||||
if [[ $KUBE_VERSION == v1.6* ]] ; then
|
||||
cat << EOF | sudo tee -a /etc/systemd/system/kubelet.service.d/11-disable_attachdetach_controller.conf
|
||||
[Service]
|
||||
Environment="KUBELET_SYSTEM_PODS_ARGS=--pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --enable-controller-attach-detach=false"
|
||||
EOF
|
||||
sudo systemctl daemon-reload
|
||||
elif [[ $KUBE_VERSION == v1.8* ]] ; then
|
||||
if [[ $KUBE_VERSION == v1.8* ]] ; then
|
||||
# for k8s 1.8, use a non default value for volume plugins
|
||||
cat << EOF | sudo tee -a /etc/systemd/system/kubelet.service.d/11-volume_plugin_dir.conf
|
||||
[Service]
|
||||
@@ -45,17 +38,19 @@ EOF
|
||||
kubectl taint nodes --all node-role.kubernetes.io/master-
|
||||
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
|
||||
|
||||
echo "wait for K8s master node to be Ready"
|
||||
kube_ready=$(kubectl get node -o jsonpath='{.items[0].status.conditions[3].status}')
|
||||
echo "wait for K8s master node to be Ready"
|
||||
INC=0
|
||||
until [[ "${kube_ready}" == "True" || $INC -gt 20 ]]; do
|
||||
while [[ $INC -lt 20 ]]; do
|
||||
kube_ready=$(kubectl get node -o jsonpath='{.items[0].status.conditions[3].status}')
|
||||
if [ "${kube_ready}" == "True" ]; then
|
||||
break
|
||||
fi
|
||||
echo "."
|
||||
sleep 10
|
||||
((++INC))
|
||||
kube_ready=$(kubectl get node -o jsonpath='{.items[0].status.conditions[3].status}')
|
||||
done
|
||||
|
||||
if [ "${kube_ready}" == "False" ]; then
|
||||
if [ "${kube_ready}" != "True" ]; then
|
||||
echo "k8s master node never went to Ready status"
|
||||
exit 1
|
||||
fi
|
||||
@@ -90,17 +85,20 @@ wait_for_ready(){
|
||||
export KUBECONFIG=$HOME/admin.conf
|
||||
|
||||
until [[ $count -eq $numberOfNode ]]; do
|
||||
echo "wait for K8s node $count to be Ready"
|
||||
kube_ready=$(kubectl get node -o jsonpath='{.items['$count'].status.conditions[3].status}')
|
||||
echo "wait for K8s node $count to be Ready."
|
||||
INC=0
|
||||
until [[ "${kube_ready}" == "True" || $INC -gt 90 ]]; do
|
||||
while [[ $INC -lt 90 ]]; do
|
||||
kubectl get node -o jsonpath='{.items['$count'].status.conditions[?(@.reason == "KubeletReady")]}'
|
||||
kube_ready=$(kubectl get node -o jsonpath='{.items['$count'].status.conditions[3].status}')
|
||||
if [ "${kube_ready}" != "True" ]; then
|
||||
break
|
||||
fi
|
||||
echo -n "."
|
||||
sleep 10
|
||||
((++INC))
|
||||
kube_ready=$(kubectl get node -o jsonpath='{.items['$count'].status.conditions[3].status}')
|
||||
done
|
||||
echo
|
||||
if [ "${kube_ready}" == "False" ]; then
|
||||
if [ "${kube_ready}" != "True" ]; then
|
||||
echo "k8s node ${count} never went to Ready status"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
@@ -96,7 +96,7 @@ MEMORY=${MEMORY:-"3000"}
|
||||
case "${1:-}" in
|
||||
up)
|
||||
# Use kubeadm bootstrapper for 1.9+ since localkube was deprecated in 1.8
|
||||
if [[ $KUBE_VERSION == v1.5* ]] || [[ $KUBE_VERSION == v1.6* ]] || [[ $KUBE_VERSION == v1.7* ]] ; then
|
||||
if [[ $KUBE_VERSION == v1.7* ]] ; then
|
||||
echo "starting minikube with localkube bootstrapper"
|
||||
minikube start --memory=${MEMORY} --kubernetes-version ${KUBE_VERSION} --extra-config=apiserver.Authorization.Mode=RBAC
|
||||
wait_for_ssh
|
||||
|
||||
@@ -29,8 +29,7 @@ function purge_rook_pods {
|
||||
kubectl -n kube-system delete secret rook-admin || true
|
||||
kubectl delete -f kube-registry.yaml || true
|
||||
kubectl delete -n rook cluster rook || true
|
||||
kubectl delete thirdpartyresources cluster.rook.io pool.rook.io objectstore.rook.io filesystem.rook.io volumeattachment.rook.io || true # ignore errors if on K8s 1.7+
|
||||
kubectl delete crd clusters.rook.io pools.rook.io objectstores.rook.io filesystems.rook.io volumeattachments.rook.io || true # ignore errors if on K8s 1.5 and 1.6
|
||||
kubectl delete crd clusters.rook.io pools.rook.io objectstores.rook.io filesystems.rook.io volumeattachments.rook.io || true
|
||||
kubectl delete -n rook-system daemonset rook-agent || true
|
||||
kubectl delete -f rook-operator.yaml || true
|
||||
kubectl delete clusterroles rook-agent || true
|
||||
|
||||
Reference in new issue
Block a user