73 files changed
+101
-101
No files matched your search
@@ -19,7 +19,7 @@
|
||||
|
||||
**File(s) to submit**:
|
||||
|
||||
* Cluster CR (custom ressource), typically called `cluster.yaml`, if necessary
|
||||
* Cluster CR (custom resource), typically called `cluster.yaml`, if necessary
|
||||
* Operator's logs, if necessary
|
||||
* Crashing pod(s) logs, if necessary
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ This script will help us achieve the following tasks
|
||||
1. Creates self-signed certificate.
|
||||
1. Creates a Certificate Signing Request(CSR) for the certificate and gets it approved from the Kubernetes cluster.
|
||||
1. Stores these certificates as a Kubernetes Secret.
|
||||
1. Creates a Service Account, ClusterRole and ClusterRoleBindings for running the webhook server with minimal priviledges.
|
||||
1. Creates a Service Account, ClusterRole and ClusterRoleBindings for running the webhook server with minimal privileges.
|
||||
1. Creates ValidatingWebhookConfig and fills the CA bundle with the appropriate value from the cluster.
|
||||
|
||||
Run the following commands:
|
||||
|
||||
@@ -757,7 +757,7 @@ Also, if you are in the need of specific features and wonder if CSI is capable o
|
||||
|
||||
### Investigation
|
||||
|
||||
This hapens when the following conditions are satisfied.
|
||||
This happens when the following conditions are satisfied.
|
||||
|
||||
- The problematic RBD device and the corresponding OSDs are co-located.
|
||||
- There is an XFS filesystem on top of this device.
|
||||
@@ -808,7 +808,7 @@ Please refer to [Configuring Pools](ceph-advanced-configuration.md#configuring-p
|
||||
|
||||
### Symptoms
|
||||
|
||||
There is a critical flaw in OSD on LV-backed PVC. LVM metadata can be corrupted if both the host and OSD container modify it simultaneously. For example, the administrator might modify it on the host, while the OSD initialization process in a container could modify it too. In addition, if `lvmetad` is running, the possibility of occurence gets higher. In this case, the change of LVM metadata in OSD container is not reflected to LVM metadata cache in host for a while.
|
||||
There is a critical flaw in OSD on LV-backed PVC. LVM metadata can be corrupted if both the host and OSD container modify it simultaneously. For example, the administrator might modify it on the host, while the OSD initialization process in a container could modify it too. In addition, if `lvmetad` is running, the possibility of occurrence gets higher. In this case, the change of LVM metadata in OSD container is not reflected to LVM metadata cache in host for a while.
|
||||
|
||||
If you still decide to configure an OSD on LVM, please keep the following in mind to reduce the probability of this issue.
|
||||
|
||||
|
||||
@@ -29,8 +29,8 @@ name needs to be set in both the storageclass and snapshotclass.
|
||||
|
||||
## Liveness Sidecar
|
||||
|
||||
All CSI pods are deployed with a sidecar container that provides a prometheus metric for tracking if the CSI plugin is alive and runnning.
|
||||
These metrics are meant to be collected by prometheus but can be acceses through a GET request to a specific node ip.
|
||||
All CSI pods are deployed with a sidecar container that provides a prometheus metric for tracking if the CSI plugin is alive and running.
|
||||
These metrics are meant to be collected by prometheus but can be accesses through a GET request to a specific node ip.
|
||||
for example `curl -X get http://[pod ip]:[liveness-port][liveness-path] 2>/dev/null | grep csi`
|
||||
the expected output should be
|
||||
|
||||
|
||||
@@ -92,7 +92,7 @@ ceph-mon \
|
||||
--public-bind-addr=$ROOK_POD_IP
|
||||
```
|
||||
|
||||
(be sure to remove the single quotes around the `--log-stderr-prefix` flag and the paranthesis around the variables being passed ROOK_CEPH_MON_HOST, ROOK_CEPH_MON_INITIAL_MEMBERS and ROOK_POD_IP )
|
||||
(be sure to remove the single quotes around the `--log-stderr-prefix` flag and the parenthesis around the variables being passed ROOK_CEPH_MON_HOST, ROOK_CEPH_MON_INITIAL_MEMBERS and ROOK_POD_IP )
|
||||
|
||||
Patch the `rook-ceph-mon-b` Deployment to run a sleep instead of the `ceph mon` command:
|
||||
|
||||
@@ -339,7 +339,7 @@ Assuming `dataHostPathData` is `/var/lib/rook`, and the `CephCluster` trying to
|
||||
|
||||
## Backing up and restoring a cluster based on PVCs into a new Kubernetes cluster
|
||||
|
||||
It is possible to migrate/restore an rook/ceph cluster from an exising Kubernetes cluster to a new one without resorting to SSH access or ceph tooling. This allows doing the migration using standard kubernetes resources only. This guide assumes the following
|
||||
It is possible to migrate/restore an rook/ceph cluster from an existing Kubernetes cluster to a new one without resorting to SSH access or ceph tooling. This allows doing the migration using standard kubernetes resources only. This guide assumes the following
|
||||
1. You have a CephCluster that uses PVCs to persist mon and osd data. Let's call it the "old cluster"
|
||||
1. You can restore the PVCs as-is in the new cluster. Usually this is done by taking regular snapshots of the PVC volumes and using a tool that can re-create PVCs from these snapshots in the underlying cloud provider. Velero is one such tool. (https://github.com/vmware-tanzu/velero)
|
||||
1. You have regular backups of the secrets and configmaps in the rook-ceph namespace. Velero provides this functionality too.
|
||||
|
||||
@@ -60,7 +60,7 @@ To begin doing this, the admin needs 2 pieces of information:
|
||||
|
||||
To pull a Ceph realm from a remote Ceph cluster, an `endpoint` must be added to the CephObjectRealm's `pull` section in the `spec`. This endpoint must be from the master zone in the master zone group of that realm.
|
||||
|
||||
If an admin does not know of an endpoint that fits this critera, the admin can find such an endpoint on the remote Ceph cluster (via the tool box if it is a Rook Ceph Cluster) by runnning:
|
||||
If an admin does not know of an endpoint that fits this criteria, the admin can find such an endpoint on the remote Ceph cluster (via the tool box if it is a Rook Ceph Cluster) by running:
|
||||
|
||||
```
|
||||
radosgw-admin zonegroup get --rgw-realm=$REALM_NAME --rgw-zonegroup=$MASTER_ZONEGROUP_NAME
|
||||
@@ -280,5 +280,5 @@ When an object-store (created in a zone) is deleted, the endpoint for that objec
|
||||
kubectl delete -f object-store.yaml
|
||||
```
|
||||
|
||||
Removing object store(s) from the master zone of the master zone group should be done with caution. When all of theses object-stores are deleted the period cannot be updated and that realm cannot be pulled.
|
||||
Removing object store(s) from the master zone of the master zone group should be done with caution. When all of these object-stores are deleted the period cannot be updated and that realm cannot be pulled.
|
||||
|
||||
@@ -113,8 +113,8 @@ Settings can be specified at the global level to apply to the cluster as a whole
|
||||
- **WARNING**: For test scenarios, if you delete a cluster and start a new cluster on the same hosts, the path used by `dataDirHostPath` must be deleted. Otherwise, stale information and other config will remain from the previous cluster and the new target will fail to start.
|
||||
If this value is empty, each pod will get an ephemeral directory to store their config files that is tied to the lifetime of the pod running on that node. More details can be found in the Kubernetes [empty dir docs](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir).
|
||||
- `dataVolumeSize`: Alternative to `dataDirHostPath`. If defined then Cluster CRD operator will disregard `dataDirHostPath` setting and instead will automatically claim persistent volume. If `storage` settings not provided then provisioned volume will also be used as a storage device for Target pods (automatic provisioning via `rtlfs`).
|
||||
- `sysRepCount`: overrides the default (3) system replication count value. Can be set to 1 or 2 for a cluster with limited number of failure domains. For example, a signle node setup with two disks can provide up to 2 replicas per chunk and requires `sysRepCount` to be set to 1 or 2.
|
||||
- `failureDomain`: identifies the way chunk replicas are distributed accross cluster's disks. The `device` domain requires each replicas to resides on different disks, the `host` domains implies one replica per node and the `zone` domain is for one replica per zone. If `failureDomain` option isn't specified, then the failure domain is set to `host` or `zone` depending on nodes config (see below). The `failureDomain` allows to specify the failure domain explicitly. For example, a single-node cluster with multiple nodes requires the `failureDomain` set to `device` if the `sysRepCount` > 1.
|
||||
- `sysRepCount`: overrides the default (3) system replication count value. Can be set to 1 or 2 for a cluster with limited number of failure domains. For example, a single node setup with two disks can provide up to 2 replicas per chunk and requires `sysRepCount` to be set to 1 or 2.
|
||||
- `failureDomain`: identifies the way chunk replicas are distributed across cluster's disks. The `device` domain requires each replicas to resides on different disks, the `host` domains implies one replica per node and the `zone` domain is for one replica per zone. If `failureDomain` option isn't specified, then the failure domain is set to `host` or `zone` depending on nodes config (see below). The `failureDomain` allows to specify the failure domain explicitly. For example, a single-node cluster with multiple nodes requires the `failureDomain` set to `device` if the `sysRepCount` > 1.
|
||||
- `dashboard`: This specification may be used to override and enable additional [EdgeFS UI Dashboard](edgefs-ui.md) functionality.
|
||||
- `localAddr`: Specifies local IP address to be used as Kubernetes external IP.
|
||||
- `network`: [network configuration settings](#network-configuration-settings)
|
||||
@@ -187,8 +187,8 @@ The following storage selection settings are specific to EdgeFS and do not apply
|
||||
- `useMetadataMask`: Defines what parts of metadata needs to be stored on offloaded devices. Default is 0xff, offload all metadata. To save SSD/NVMe capacity, set it to 0x7d to offload all except second level manifests. Applicable only to `rtrd`.
|
||||
- `useBCache`: When `useMetadataOffload` is true, enable use of BCache. Default is false. Applicable only to `rtrd` and when host has "bcache" kernel module preloaded.
|
||||
- `useBCacheWB`: When `useMetadataOffload` and `useBCache` is true, this option can enable use of BCache write-back cache. By default BCache only used as read cache in front of HDD. Applicable only to `rtrd`.
|
||||
- `useAllSSD`: When set to true, only SSD/NVMe non rotational devices will be used. Default is false and if `useMetadataOffload` not defined then only rotational devices (HDDs) will be picked up during node provisioning phase. Is not applicabel to `rtkvs`.
|
||||
- `rtPLevelOverride`: In case of large devices or directories, it will be automatically partitioned into smaller parts around 500GB each. In case of embedded use cases, lowering the value would allow to operate with smaller memory footprint devices at the cost of performance. This option allows partitioning number override. Default is automatic. Typical and recommended range is 1 - 32. For `rtkvs` recomended plevel is 16.
|
||||
- `useAllSSD`: When set to true, only SSD/NVMe non rotational devices will be used. Default is false and if `useMetadataOffload` not defined then only rotational devices (HDDs) will be picked up during node provisioning phase. Is not applicable to `rtkvs`.
|
||||
- `rtPLevelOverride`: In case of large devices or directories, it will be automatically partitioned into smaller parts around 500GB each. In case of embedded use cases, lowering the value would allow to operate with smaller memory footprint devices at the cost of performance. This option allows partitioning number override. Default is automatic. Typical and recommended range is 1 - 32. For `rtkvs` recommended plevel is 16.
|
||||
- `hddReadAhead`: For all HDD or hybrid (SSD/HDD) use cases, adjusting hddReadAhead may provide significant boost in performance. Set to a value higher then 0, in KBs. Not applicable to `rtkvs`
|
||||
- `mdReserved`: For hybrid (SSD/HDD) use case, adjusting mdReserved can be necessary when combined with BCache read/write caches. Allowed range 10-99% of automatically calcuated slice. Not applicable to `rtkvs`
|
||||
- `rtVerifyChid`: Verify transferred or read payload. Payload can be data or metadata chunk of flexible size between 4K and 8MB. EdgeFS uses SHA-3 variant to cryptographically sign each chunk and uses it for self validation, self healing and FlexHash addressing. In case of low CPU systems verification after networking transfer prior to write can be disabled by setting this parameter to 0. In case of high CPU systems, verification after read but before networking transfer can be enabled by setting this parameter to 2. Default is 1, i.e. verify after networking transfer only. Setting it to 0 may improve CPU utilization at the cost of reduced availability. However, for objects with 3 or more replicas, availability isn't going to be visibly affected.
|
||||
@@ -207,7 +207,7 @@ The following storage selection settings are specific to EdgeFS and do not apply
|
||||
- `payloadS3Region`: S3 server region. Default is `us-east-1`. Only applicable when the `payloadS3URL` is set.
|
||||
- `payloadS3MinKb`: a minimal payload chunk size to trigger the forwarding to the S3 bucket. Data chunks smaller than this value will be stored locally. Only applicable when the `payloadS3URL` is set.
|
||||
- `payloadS3CapacityGB`: capacity of the external S3 bucket expressed in GB. This is an artificial value used to report accurate usage summary and to limit storage size. Only applicable when the `payloadS3URL` is set.
|
||||
- `payloadS3Secret`: name of a Kubernetes secret to be used as an external S3 bucket credential. The secret needs to be pre-creted before deployment of the EdgeFS cluster and resides in the same namespace as the cluster. Secret's key neeed to be set to `cred` and value format is as follow: `<aws_key>,<aws_secret>`. See an example in the s3PayloadSecret.yaml
|
||||
- `payloadS3Secret`: name of a Kubernetes secret to be used as an external S3 bucket credential. The secret needs to be pre-creted before deployment of the EdgeFS cluster and resides in the same namespace as the cluster. Secret's key need to be set to `cred` and value format is as follow: `<aws_key>,<aws_secret>`. See an example in the s3PayloadSecret.yaml
|
||||
|
||||
|
||||
### Placement Configuration Settings
|
||||
|
||||
@@ -55,7 +55,7 @@ spec:
|
||||
* `name`: The name of the NFS system to create, which must match existing EdgeFS service.
|
||||
* `namespace`: The namespace of the Rook cluster where the NFS service is created.
|
||||
* `instances`: The number of active NFS service instances. EdgeFS NFS service is Multi-Head capable, such so that multiple PODs can mount same tenant's buckets via different endpoints. [EdgeFS CSI provisioner](edgefs-csi.md) orchestrates distribution and load balancing across NFS service instances in round-robin or random policy ways.
|
||||
* `relaxedDirUpdates`: If set to `true` then it will significantly improve performance of directory operations by defering updates, guaranteeing eventual directory consistency. This option is recommended when a bucket exported via single NFS instance and it is not a destination for ISGW Link synchronization.
|
||||
* `relaxedDirUpdates`: If set to `true` then it will significantly improve performance of directory operations by deferring updates, guaranteeing eventual directory consistency. This option is recommended when a bucket exported via single NFS instance and it is not a destination for ISGW Link synchronization.
|
||||
* `chunkCacheSize`: Limit amount of memory allocated for dynamic chunk cache. By default NFS pod uses up to 75% of available memory as chunk caching area. This option can influence this allocation strategy.
|
||||
* `annotations`: Key value pair list of annotations to add.
|
||||
* `placement`: The NFS PODs can be given standard Kubernetes placement restrictions with `nodeAffinity`, `tolerations`, `podAffinity`, and `podAntiAffinity` similar to placement defined for daemons configured by the [cluster CRD](/cluster/examples/kubernetes/edgefs/cluster.yaml).
|
||||
|
||||
@@ -61,7 +61,7 @@ spec:
|
||||
* `name`: The name of the SMB system to create, which must match existing EdgeFS service.
|
||||
* `namespace`: The namespace of the Rook cluster where the SMB service is created.
|
||||
* `instances`: The number of active SMB service instances. EdgeFS SMB service is Multi-Head capable, such so that multiple PODs can mount same tenant's buckets via different endpoints.
|
||||
* `relaxedDirUpdates`: If set to `true` then it will significantly improve performance of directory operations by defering updates, guaranteeing eventual directory consistency. This option is recommended when a bucket exported via single SMB instance and it is not a destination for ISGW Link synchronization.
|
||||
* `relaxedDirUpdates`: If set to `true` then it will significantly improve performance of directory operations by deferring updates, guaranteeing eventual directory consistency. This option is recommended when a bucket exported via single SMB instance and it is not a destination for ISGW Link synchronization.
|
||||
* `chunkCacheSize`: Limit amount of memory allocated for dynamic chunk cache. By default SMB pod uses up to 75% of available memory as chunk caching area. This option can influence this allocation strategy.
|
||||
* `annotations`: Key value pair list of annotations to add.
|
||||
* `placement`: The SMB PODs can be given standard Kubernetes placement restrictions with `nodeAffinity`, `tolerations`, `podAffinity`, and `podAntiAffinity` similar to placement defined for daemons configured by the [cluster CRD](/cluster/examples/kubernetes/edgefs/cluster.yaml).
|
||||
|
||||
@@ -58,7 +58,7 @@ And replace EdgeFS operator to new stable 'v1' version
|
||||
kubectl -n rook-edgefs-system set image deploy/rook-edgefs-operator rook-edgefs-operator=rook/edgefs:v1.1.0
|
||||
```
|
||||
|
||||
Then you could update your cluster's EdgeFS image for latest one as discribed below.
|
||||
Then you could update your cluster's EdgeFS image for latest one as described below.
|
||||
|
||||
## EdgeFS Version Upgrade
|
||||
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ The following high level features are targeted for Rook v1.5 (November 2020). Fo
|
||||
* RGW Multi-site replication improvements towards declaring the feature stable
|
||||
* Multus networking configuration declared stable
|
||||
* Support IPv6 single stack configuration [#3850](https://github.com/rook/rook/issues/3850)
|
||||
* Enabing the admission controller by default [#6242](https://github.com/rook/rook/issues/6242)
|
||||
* Enabling the admission controller by default [#6242](https://github.com/rook/rook/issues/6242)
|
||||
* NFS
|
||||
* A quota for disk usage can now be set and enforced for each NFS PV [#5788](https://github.com/rook/rook/issues/5788)
|
||||
* NFS PV's can be deleted and cleaned up [#3074](https://github.com/rook/rook/issues/3074)
|
||||
|
||||
+1
-1
@@ -42,7 +42,7 @@ function check_git() {
|
||||
# see https://github.com/git/git/blob/master/Documentation/RelNotes/2.8.3.txt#L33
|
||||
local gitversion=$(git --version | cut -d" " -f3)
|
||||
if (( $(ver ${gitversion}) > $(ver 2.6.6) && $(ver ${gitversion}) < $(ver 2.8.3) )); then
|
||||
echo WARN: your running git version ${gitversion} which has a bug realted to relative
|
||||
echo WARN: your running git version ${gitversion} which has a bug related to relative
|
||||
echo WARN: submodule paths. Please consider upgrading to 2.8.3 or later
|
||||
fi
|
||||
}
|
||||
|
||||
@@ -91,7 +91,7 @@ endif
|
||||
# the build. Please keep them in sync.
|
||||
|
||||
# we run go build with -i which on most system's would want to install packages
|
||||
# into the system's root dir. using our own pkg dir avoid thats
|
||||
# into the system's root dir. using our own pkg dir avoid this
|
||||
ifneq ($(GO_PKG_DIR),)
|
||||
GO_PKG_BASE_DIR := $(abspath $(GO_PKG_DIR)/$(PLATFORM))
|
||||
GO_PKG_STATIC_FLAGS := -pkgdir $(GO_PKG_BASE_DIR)_static
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
# * restricted (for all logged in users)
|
||||
#
|
||||
# If we name it as `rook-ceph-operator`, it comes next to `restricted` PSP alphabetically,
|
||||
# and applies `restricted` capabilities to `rook-system`. Thats reason this is named with `00-rook-ceph-operator`,
|
||||
# and applies `restricted` capabilities to `rook-system`. That's reason this is named with `00-rook-ceph-operator`,
|
||||
# so it stays somewhere close to top and `rook-system` gets the intended PSP.
|
||||
#
|
||||
# More info on PSP ordering : https://kubernetes.io/docs/concepts/policy/pod-security-policy/#policy-order
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Service acccount for the operator
|
||||
# Service account for the operator
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
|
||||
@@ -779,7 +779,7 @@ if __name__ == '__main__':
|
||||
try:
|
||||
rjObj.main()
|
||||
except ExecutionFailureException as err:
|
||||
print("Excecution Failed: {}".format(err))
|
||||
print("Execution Failed: {}".format(err))
|
||||
except KeyError as kErr:
|
||||
print("KeyError: %s", kErr)
|
||||
except OSError as osErr:
|
||||
|
||||
@@ -433,7 +433,7 @@ spec:
|
||||
- name: ROOK_DISABLE_DEVICE_HOTPLUG
|
||||
value: "false"
|
||||
# Provide customised regex as the values using comma. For eg. regex for rbd based volume, value will be like "(?i)rbd[0-9]+".
|
||||
# In case of more than one regex, use comma to seperate between them.
|
||||
# In case of more than one regex, use comma to separate between them.
|
||||
# Default regex will be "(?i)dm-[0-9]+,(?i)rbd[0-9]+,(?i)nbd[0-9]+"
|
||||
# add regex expression after putting a comma to blacklist a disk
|
||||
# If value is empty, the default regex will be used.
|
||||
|
||||
@@ -384,7 +384,7 @@ spec:
|
||||
value: "false"
|
||||
|
||||
# Provide customised regex as the values using comma. For eg. regex for rbd based volume, value will be like "(?i)rbd[0-9]+".
|
||||
# In case of more than one regex, use comma to seperate between them.
|
||||
# In case of more than one regex, use comma to separate between them.
|
||||
# Default regex will be "(?i)dm-[0-9]+,(?i)rbd[0-9]+,(?i)nbd[0-9]+"
|
||||
# Add regex expression after putting a comma to blacklist a disk
|
||||
# If value is empty, the default regex will be used.
|
||||
|
||||
@@ -12,6 +12,6 @@ cluster: cltest #substitution edgefs cluster name for csi operations
|
||||
tenant: test #substitution edgefs tenant name for csi operations
|
||||
bucket: bk1 #substitution edgefs bucket name for csi operations
|
||||
|
||||
# Edgefs connnection options
|
||||
# Edgefs connection options
|
||||
username: admin #edgefs k8s cluster grpc connection username
|
||||
password: admin #edgefs k8s cluster grpc connection password
|
||||
@@ -10,8 +10,8 @@ k8sEdgefsMgmtPrefix: rook-edgefs-mgr # edgefs cluster management prefix
|
||||
#Edgefs csi operatins options
|
||||
cluster: cltest #substitution edgefs cluster name for csi operations
|
||||
tenant: test #substitution edgefs tenant name for csi operations
|
||||
#serviceFilter: "nfs01" #List of comma delimeted allowed service names to filter
|
||||
#serviceFilter: "nfs01" #List of comma delimited allowed service names to filter
|
||||
|
||||
# Edgefs connnection options
|
||||
# Edgefs connection options
|
||||
username: admin #edgefs k8s cluster grpc connection username
|
||||
password: admin #edgefs k8s cluster grpc connection password
|
||||
@@ -133,7 +133,7 @@ that could be made to improve the Rook and CSI integration:
|
||||
|
||||
Extend CephBlockPool and CephFilesystem CRDs to automatically provision Storage
|
||||
Classes when so configured. Instead of requiring an administrator to create a
|
||||
CRD and a Storage Class, add metatdata to the CRD such that Rook will
|
||||
CRD and a Storage Class, add metadata to the CRD such that Rook will
|
||||
automatically create storage classes based on that additional metadata.
|
||||
|
||||
#### Select Flex Provisioning or CSI based on CephCluster CRD
|
||||
|
||||
@@ -4,10 +4,10 @@
|
||||
and [RGW] backends. It can export Ceph's filesystem namespaces and Object
|
||||
gateway namespaces over NFSv4 protocol.
|
||||
|
||||
Rook already orchestrates Ceph filesytem and Object store (or RGW) on
|
||||
Rook already orchestrates Ceph filesystem and Object store (or RGW) on
|
||||
Kubernetes (k8s). It can be extended to orchestrate NFS-Ganesha server daemons
|
||||
as highly available and scalable NFS gateway pods to the Ceph filesytem and
|
||||
Object Store. This will allow NFS client applications to use the Ceph filesytem
|
||||
as highly available and scalable NFS gateway pods to the Ceph filesystem and
|
||||
Object Store. This will allow NFS client applications to use the Ceph filesystem
|
||||
and object store setup by rook.
|
||||
|
||||
This feature mainly differs from the feature to add NFS as an another
|
||||
|
||||
@@ -6,7 +6,7 @@ A Rook Ceph cluster.
|
||||
|
||||
## Ceph Object Realm Walkthrough
|
||||
|
||||
The resource descibed in this design document represents the realm in the [Ceph Multisite data model](/design/ceph/object/ceph-multisite-overview.md).
|
||||
The resource described in this design document represents the realm in the [Ceph Multisite data model](/design/ceph/object/ceph-multisite-overview.md).
|
||||
|
||||
### Creating an Object Realm
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ A Rook Ceph cluster. Ideally a ceph-object-realm resource would have been starte
|
||||
|
||||
## Ceph Object Zone Group Walkthrough
|
||||
|
||||
The resource descibed in this design document represents the zone group in the [Ceph Multisite data model](/design/ceph/object/ceph-multisite-overview.md).
|
||||
The resource described in this design document represents the zone group in the [Ceph Multisite data model](/design/ceph/object/ceph-multisite-overview.md).
|
||||
|
||||
|
||||
### Creating an Ceph Object Zone Group
|
||||
|
||||
@@ -6,7 +6,7 @@ A Rook Ceph cluster. Ideally a ceph-object-realm and a ceph-object-zone-group re
|
||||
|
||||
## Ceph Object Zone Walkthrough
|
||||
|
||||
The resource descibed in this design document represents the zone in the [Ceph Multisite data model](/design/ceph/object/ceph-multisite-overview.md).
|
||||
The resource described in this design document represents the zone in the [Ceph Multisite data model](/design/ceph/object/ceph-multisite-overview.md).
|
||||
|
||||
### Creating an Ceph Object Zone
|
||||
|
||||
|
||||
@@ -108,7 +108,7 @@ Orchestrator CLI or Dashboard interface.
|
||||
|
||||
#### Reporting Status
|
||||
Rook should report failure messages back to the Rook Orchestrator Mgr Module via a `CephCluster`
|
||||
`status` mechanism so that the Orchestrator Module can take appropriate actions ot notify the user
|
||||
`status` mechanism so that the Orchestrator Module can take appropriate actions to notify the user
|
||||
of failure. Rook may report successful statuses back if needed by the Orchestrator Module; however,
|
||||
limiting the information reported back will keep the `CephCluster` resource's `status` field smaller
|
||||
and less unwieldy for users.
|
||||
|
||||
@@ -67,7 +67,7 @@ In the CephClusterStatus, we can either remove the `Status.State` and `Status.Me
|
||||
|
||||
### Examples
|
||||
|
||||
Consider a cluster is being created. The RookConditions is an array that can store multiple Conditions. So the progression of the cluster being created can be seen in the RookConditions as shown in the example below. The Ceph Cluster gets created after it establishes a succesfull Connection. The `RookCondition` will show in the slice that the `Connecting` Condition will be in `Condition.Status` False. The `Connected` and `Progressing` Types will be set to True.
|
||||
Consider a cluster is being created. The RookConditions is an array that can store multiple Conditions. So the progression of the cluster being created can be seen in the RookConditions as shown in the example below. The Ceph Cluster gets created after it establishes a successful Connection. The `RookCondition` will show in the slice that the `Connecting` Condition will be in `Condition.Status` False. The `Connected` and `Progressing` Types will be set to True.
|
||||
|
||||
Before:
|
||||
ClusterStatus{
|
||||
|
||||
@@ -34,7 +34,7 @@ default to pod networking.
|
||||
|
||||
Rook operators can define storage cluster's network using network provider.
|
||||
Network provider example includes host, and multus. To configure the cluster
|
||||
network, cluster CRD needs to tell the network provider the appropiate
|
||||
network, cluster CRD needs to tell the network provider the appropriate
|
||||
`NetworkInterfaceSelector`. `NetworkInterfaceSelector` will be provided as list
|
||||
of `interfaces` key-value.
|
||||
|
||||
@@ -77,7 +77,7 @@ On the other hand, multi-plugin CNI plugin needs to know the network attachment
|
||||
definition's name and vice versa. Multi-plugin such as multus may seem to follow
|
||||
_Network Attachment Selection Annotation_ documented at [Kubernetes Network
|
||||
Custom Resource Definition De-facto Standard][network-crd]. However, their
|
||||
implemtation has extra features not covered by the standard such as
|
||||
implementation has extra features not covered by the standard such as
|
||||
`@<interface-name>` notation or `interfaceRequest` property documented
|
||||
[here][multus-annotation].
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ We welcome contributions! In the meantime, features that are not yet implemented
|
||||
### Prerequisites
|
||||
|
||||
- A Rook storage cluster must be configured and running in Kubernetes. In this example, it is assumed the cluster is in the `rook` namespace.
|
||||
- The following resources, or equivilant, need to be created:
|
||||
- The following resources, or equivalent, need to be created:
|
||||
- [common](/cluster/examples/kubernetes/ceph/common.yaml)
|
||||
- [operator](/cluster/examples/kubernetes/ceph/operator.yaml)
|
||||
- [cluster](/cluster/examples/kubernetes/ceph/cluster-test.yaml)
|
||||
|
||||
@@ -220,10 +220,10 @@ spec:
|
||||
features: # these are not components but confiuration groups
|
||||
prometheus: true
|
||||
tracing: true
|
||||
ozoneVersion: #see Ceph versioning doc for mor details
|
||||
ozoneVersion: #see Ceph versioning doc for more details
|
||||
image: apache/ozone:0.4.1
|
||||
allowUnsupported: false
|
||||
upgradePolicy: #see Ceph versioning doc for mor details (Won't be implmented in Phase 1)
|
||||
upgradePolicy: #see Ceph versioning doc for more details (Won't be implemented in Phase 1)
|
||||
ozoneVersion:
|
||||
image: apache/ozone:0.5.0
|
||||
allowUnsupported: false
|
||||
@@ -238,7 +238,7 @@ spec:
|
||||
generated-with: rook
|
||||
```
|
||||
|
||||
The type of the `storage` attribute is `StorageScopeSpec` which is a [common Rook type](https://github.com/rook/rook/blob/818ad9c7ca4d352a9bed202e619d47b05f1a4dab/pkg/apis/rook.io/v1alpha2/types.go), and can be used to specifiy the details of the storage (eg. volumeClaimTemplates, nodeCounts, etc.)
|
||||
The type of the `storage` attribute is `StorageScopeSpec` which is a [common Rook type](https://github.com/rook/rook/blob/818ad9c7ca4d352a9bed202e619d47b05f1a4dab/pkg/apis/rook.io/v1alpha2/types.go), and can be used to specify the details of the storage (eg. volumeClaimTemplates, nodeCounts, etc.)
|
||||
|
||||
## Operator capabilities
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ write_endpoints() {
|
||||
endpoints=$(cat ${MON_CONFIG})
|
||||
|
||||
# filter out the mon names
|
||||
# external cluster can have numbers or hypens in mon names, handling them in regex
|
||||
# external cluster can have numbers or hyphens in mon names, handling them in regex
|
||||
mon_endpoints=$(echo ${endpoints} | sed 's/[a-z0-9_-]\+=//g')
|
||||
|
||||
DATE=$(date)
|
||||
|
||||
@@ -26,10 +26,10 @@ const (
|
||||
// SanitizeMethodComplete will sanitize everything on the disk
|
||||
SanitizeMethodComplete SanitizeMethodProperty = "complete"
|
||||
|
||||
// SanitizeMethodQuick will sanitize metdata only on the disk
|
||||
// SanitizeMethodQuick will sanitize metadata only on the disk
|
||||
SanitizeMethodQuick SanitizeMethodProperty = "quick"
|
||||
|
||||
// DeleteDataDirOnHostsConfirmation represents the validation to destry dataDirHostPath
|
||||
// DeleteDataDirOnHostsConfirmation represents the validation to destroy dataDirHostPath
|
||||
DeleteDataDirOnHostsConfirmation CleanupConfirmationProperty = "yes-really-destroy-data"
|
||||
)
|
||||
|
||||
|
||||
@@ -274,7 +274,7 @@ func TestClusterDeleteMultiAttachmentRace(t *testing.T) {
|
||||
clusterToDelete := &cephv1.CephCluster{ObjectMeta: metav1.ObjectMeta{Namespace: clusterName}}
|
||||
controller.handleClusterDelete(clusterToDelete, time.Millisecond)
|
||||
|
||||
// both attachments should have made it all the way through the clean up process, meaing that Delete
|
||||
// both attachments should have made it all the way through the clean up process, meaning that Delete
|
||||
// (which is idempotent) should have been called twice.
|
||||
assert.Equal(t, 2, deleteCount)
|
||||
}
|
||||
|
||||
@@ -253,7 +253,7 @@ func getMDSRank(status CephStatus, fsName string) (int, error) {
|
||||
return mdsRank, nil
|
||||
}
|
||||
|
||||
// MdsActiveOrStandbyReplay returns wether a given MDS is active or in standby
|
||||
// MdsActiveOrStandbyReplay returns whether a given MDS is active or in standby
|
||||
func MdsActiveOrStandbyReplay(context *clusterd.Context, clusterInfo *ClusterInfo, fsName string) error {
|
||||
status, err := Status(context, clusterInfo)
|
||||
if err != nil {
|
||||
|
||||
@@ -367,7 +367,7 @@ func buildHostListFromTree(tree OsdTree) (OsdTree, error) {
|
||||
return osdList, nil
|
||||
}
|
||||
|
||||
// osdDoNothing determines wether we should perform upgrade pre-check and post-checks for the OSD daemon
|
||||
// osdDoNothing determines whether we should perform upgrade pre-check and post-checks for the OSD daemon
|
||||
// it checks for various cluster info like number of OSD and their placement
|
||||
// it returns 'true' if we need to do nothing and false and we should pre-check/post-check
|
||||
func osdDoNothing(context *clusterd.Context, clusterInfo *ClusterInfo) bool {
|
||||
|
||||
@@ -240,7 +240,7 @@ func Provision(context *clusterd.Context, agent *OsdAgent, crushLocation string)
|
||||
rawDevices = append(rawDevices, rawDevice)
|
||||
}
|
||||
} else {
|
||||
// We still need to use 'lsblk' as the underlaying way to discover devices
|
||||
// We still need to use 'lsblk' as the underlying way to discover devices
|
||||
// Ideally, we would use the "ceph-volume inventory" command instead
|
||||
// However, it suffers from some limitation such as exposing available partitions and LVs
|
||||
// See: https://tracker.ceph.com/issues/43579
|
||||
@@ -374,7 +374,7 @@ func getAvailableDevices(context *clusterd.Context, agent *OsdAgent) (*DeviceOsd
|
||||
// udevadm does not support device path different than /dev or /sys
|
||||
//
|
||||
// So earlier lsblk extracted the '/dev' path, hence the device.Name property
|
||||
// device.Name can be 'xvdca', later this is formated to '/dev/xvdca'
|
||||
// device.Name can be 'xvdca', later this is formatted to '/dev/xvdca'
|
||||
var err error
|
||||
var isAvailable bool
|
||||
rejectedReason := ""
|
||||
|
||||
@@ -55,7 +55,7 @@ func (a *OsdAgent) configureDriveGroups(context *clusterd.Context) error {
|
||||
|
||||
func (a *OsdAgent) driveGroupsAreSupported(context *clusterd.Context) bool {
|
||||
// Call `ceph-volume drive-group` command with no args:
|
||||
// if supported, will return 0 (and help message which we ingore), else returns nonzero (error)
|
||||
// if supported, will return 0 (and help message which we ignore), else returns nonzero (error)
|
||||
_, err := callCephVolume(context, true, cvDriveGroupsCommand)
|
||||
return err == nil
|
||||
}
|
||||
@@ -150,7 +150,7 @@ func matchUdevEvent(text string, matches, exclusions []string) (bool, error) {
|
||||
func rawUdevBlockMonitor(c chan string, matches, exclusions []string) {
|
||||
defer close(c)
|
||||
|
||||
// stdbuf -oL performs line bufferred output
|
||||
// stdbuf -oL performs line buffered output
|
||||
cmd := exec.Command("stdbuf", "-oL", "udevadm", "monitor", "-u", "-k", "-s", "block")
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
|
||||
@@ -149,7 +149,7 @@ func New(
|
||||
//Deletion handling:
|
||||
// Atm, the only thing left behind will be the state, ie
|
||||
// the PVCs that the StatefulSets don't erase.
|
||||
// This behaviour may actually be preferrable to deleting them,
|
||||
// This behaviour may actually be preferable to deleting them,
|
||||
// since it ensures that no data will be lost if someone accidentally
|
||||
// deletes the cluster.
|
||||
})
|
||||
|
||||
@@ -217,7 +217,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler, context *clusterd.Context)
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileCephCluster) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile. %v", err)
|
||||
|
||||
@@ -342,7 +342,7 @@ func genSecret(executor exec.Executor, configDir, name string, args []string) (s
|
||||
return ExtractKey(string(contents))
|
||||
}
|
||||
|
||||
// ExtractKey retrives mon secret key from the keyring file
|
||||
// ExtractKey retrieves mon secret key from the keyring file
|
||||
func ExtractKey(contents string) (string, error) {
|
||||
secret := ""
|
||||
slice := strings.Fields(sys.Grep(string(contents), "key"))
|
||||
|
||||
@@ -944,7 +944,7 @@ func (c *Cluster) applyUpgradeOSDFunctionality() {
|
||||
|
||||
// If length is one, this clearly indicates that all the osds are running the same version
|
||||
// If this is the first time we are creating a cluster length will be 0
|
||||
// On an initial OSD boostrap, by the time we reach this code, the OSDs haven't registered yet
|
||||
// On an initial OSD bootstrap, by the time we reach this code, the OSDs haven't registered yet
|
||||
// Basically, this task is happening too quickly and OSD pods are not running yet.
|
||||
// That's not an issue since it's an initial bootstrap and not an update.
|
||||
if len(versions.Osd) == 1 {
|
||||
|
||||
@@ -139,7 +139,7 @@ if [ -b "$DM_PATH" ]; then
|
||||
if [[ "$field" =~ ^[0-9]+\:[0-9]+ ]]; then
|
||||
underlaying_block="/sys/dev/block/$field"
|
||||
if [ ! -d "$underlaying_block" ]; then
|
||||
echo "Underlaying block device $underlaying_block of crypt $DM_NAME disappeared!"
|
||||
echo "Underlying block device $underlaying_block of crypt $DM_NAME disappeared!"
|
||||
echo "Removing stale dm device $DM_NAME"
|
||||
dmsetup remove --force "$DM_NAME"
|
||||
open_encrypted_block
|
||||
|
||||
@@ -136,7 +136,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileCephRBDMirror) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
updateStatus(r.client, request.NamespacedName, k8sutil.FailedStatus)
|
||||
|
||||
@@ -64,7 +64,7 @@ func TestDiffImageSpecAndClusterRunningVersion(t *testing.T) {
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, m)
|
||||
|
||||
// 3nd test - spec version is lower than running cluster? what's going on?
|
||||
// 3rd test - spec version is lower than running cluster? what's going on?
|
||||
fakeRunningVersions = []byte(`
|
||||
{
|
||||
"overall": {
|
||||
|
||||
@@ -100,7 +100,7 @@ func (c *clientCluster) onK8sNode(object runtime.Object) bool {
|
||||
|
||||
// If they are OSDs in the CRUSH map and if the host exists in the CRUSH map, don't reconcile
|
||||
if osds != "" {
|
||||
// This is Debug level because the node receives frequent updates and this will polute the logs
|
||||
// This is Debug level because the node receives frequent updates and this will pollute the logs
|
||||
logger.Debugf("node watcher: node %q is already an OSD node with %q", nodeName, osds)
|
||||
} else {
|
||||
logger.Infof("node watcher: adding node %q to cluster %q", node.Labels[v1.LabelHostname], cluster.Namespace)
|
||||
|
||||
@@ -63,15 +63,15 @@ const (
|
||||
)
|
||||
|
||||
var (
|
||||
// VarLibCephDir is simply "/var/lib/ceph". It is made overwriteable only for unit tests where it
|
||||
// VarLibCephDir is simply "/var/lib/ceph". It is made overwritable only for unit tests where it
|
||||
// may be needed to send data intended for /var/lib/ceph to a temporary test dir.
|
||||
VarLibCephDir = "/var/lib/ceph"
|
||||
|
||||
// EtcCephDir is simply "/etc/ceph". It is made overwriteable only for unit tests where it
|
||||
// EtcCephDir is simply "/etc/ceph". It is made overwritable only for unit tests where it
|
||||
// may be needed to send data intended for /etc/ceph to a temporary test dir.
|
||||
EtcCephDir = "/etc/ceph"
|
||||
|
||||
// VarLogCephDir defines Ceph logging directory. It is made overwriteable only for unit tests where it
|
||||
// VarLogCephDir defines Ceph logging directory. It is made overwritable only for unit tests where it
|
||||
// may be needed to send data intended for /var/log/ceph to a temporary test dir.
|
||||
VarLogCephDir = "/var/log/ceph"
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ const (
|
||||
)
|
||||
|
||||
// Add the Ceph version to the given labels. This should *not* be used on pod specifications,
|
||||
// because this will result in the deployment/daemonset/ect. recreating all of its pods even if an
|
||||
// because this will result in the deployment/daemonset/etc. recreating all of its pods even if an
|
||||
// update wouldn't otherwise be required. Upgrading unnecessarily increases risk for loss of data
|
||||
// reliability, even if only briefly.
|
||||
func addCephVersionLabel(cephVersion version.CephVersion, labels map[string]string) {
|
||||
|
||||
@@ -101,7 +101,7 @@ func Add(mgr manager.Manager, context *controllerconfig.Context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// Watch for changes to the osd pod nodename and enqueue thier nodes
|
||||
// Watch for changes to the osd pod nodename and enqueue their nodes
|
||||
err = c.Watch(
|
||||
&source.Kind{Type: &corev1.Pod{}},
|
||||
&handler.EnqueueRequestsFromMapFunc{
|
||||
|
||||
@@ -61,7 +61,7 @@ type ReconcileNode struct {
|
||||
// The Controller will requeue the Request to be processed again if an error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileNode) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
result, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Error(err)
|
||||
@@ -155,7 +155,7 @@ func (r *ReconcileNode) reconcile(request reconcile.Request) (reconcile.Result,
|
||||
// CreateOrUpdate the deployment
|
||||
mutateFunc := func() error {
|
||||
|
||||
// lablels for the pod, the deployment, and the deploymentSelector
|
||||
// labels for the pod, the deployment, and the deploymentSelector
|
||||
selectorLabels := map[string]string{
|
||||
corev1.LabelHostname: nodeHostnameLabel,
|
||||
k8sutil.AppAttr: CanaryAppName,
|
||||
|
||||
@@ -128,7 +128,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileCephFilesystem) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile %v", err)
|
||||
|
||||
@@ -267,7 +267,7 @@ func (f *Filesystem) doFilesystemCreate(context *clusterd.Context, clusterInfo *
|
||||
}
|
||||
|
||||
// create the filesystem ('fs new' needs to be forced in order to reuse pre-existing pools)
|
||||
// if only one pool is created new it wont work (to avoid inconsistencies).
|
||||
// if only one pool is created new it won't work (to avoid inconsistencies).
|
||||
if err := client.CreateFilesystem(context, clusterInfo, f.Name, metadataPoolName, dataPoolNames, !poolsCreated); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -128,7 +128,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileCephNFS) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile %v", err)
|
||||
|
||||
@@ -144,7 +144,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileCephObjectStore) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile %v", err)
|
||||
|
||||
@@ -125,7 +125,7 @@ const (
|
||||
type PolicyStatement struct {
|
||||
// Sid (optional) is the PolicyStatement's unique identifier
|
||||
Sid string `json:"Sid"`
|
||||
// Effect determins whether the Action(s) are 'Allow'ed or 'Deny'ed.
|
||||
// Effect determines whether the Action(s) are 'Allow'ed or 'Deny'ed.
|
||||
Effect effect `json:"Effect"`
|
||||
// Principle is/are the Ceph user names affected by this PolicyStatement
|
||||
// Must be in the format of 'arn:aws:iam:::user/<ceph-user>'
|
||||
|
||||
@@ -119,7 +119,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileObjectRealm) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile: %v", err)
|
||||
|
||||
@@ -125,7 +125,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileObjectStoreUser) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile %v", err)
|
||||
|
||||
@@ -114,7 +114,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileObjectZone) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile: %v", err)
|
||||
|
||||
@@ -113,7 +113,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileObjectZoneGroup) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile: %v", err)
|
||||
|
||||
@@ -41,7 +41,7 @@ const (
|
||||
)
|
||||
|
||||
func (r *ReconcileCephBlockPool) createBootstrapPeerSecret(cephBlockPool *cephv1.CephBlockPool, namespacedName types.NamespacedName) (reconcile.Result, error) {
|
||||
// Create rbd mirror boostrap peer token
|
||||
// Create rbd mirror bootstrap peer token
|
||||
boostrapToken, err := cephclient.CreateRBDMirrorBootstrapPeer(r.context, r.clusterInfo, cephBlockPool.Name)
|
||||
if err != nil {
|
||||
return opcontroller.ImmediateRetryResult, errors.Wrap(err, "failed to create rbd-mirror bootstrap peer")
|
||||
|
||||
@@ -117,7 +117,7 @@ func add(mgr manager.Manager, r reconcile.Reconciler) error {
|
||||
// The Controller will requeue the Request to be processed again if the returned error is non-nil or
|
||||
// Result.Requeue is true, otherwise upon completion it will remove the work from the queue.
|
||||
func (r *ReconcileCephBlockPool) Reconcile(request reconcile.Request) (reconcile.Result, error) {
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface
|
||||
// workaround because the rook logging mechanism is not compatible with the controller-runtime logging interface
|
||||
reconcileResponse, err := r.reconcile(request)
|
||||
if err != nil {
|
||||
logger.Errorf("failed to reconcile. %v", err)
|
||||
|
||||
@@ -26,10 +26,10 @@ import (
|
||||
func TestToString(t *testing.T) {
|
||||
assert.Equal(t, "14.0.0-0 nautilus", Nautilus.String())
|
||||
assert.Equal(t, "15.0.0-0 octopus", Octopus.String())
|
||||
receieved := CephVersion{-1, 0, 0, 0}
|
||||
received := CephVersion{-1, 0, 0, 0}
|
||||
|
||||
expected := fmt.Sprintf("-1.0.0-0 %s", unknownVersionString)
|
||||
assert.Equal(t, expected, receieved.String())
|
||||
assert.Equal(t, expected, received.String())
|
||||
}
|
||||
|
||||
func TestCephVersionFormatted(t *testing.T) {
|
||||
|
||||
@@ -131,10 +131,10 @@ func (c *cluster) createInstance(rookImage string, isClusterUpdate bool) (bool,
|
||||
dro := ParseDevicesResurrectMode(c.Spec.DevicesResurrectMode)
|
||||
logger.Infof("DevicesResurrect options: %+v", dro)
|
||||
|
||||
// Retrive existing cluster config from Kubernetes ConfigMap
|
||||
// Retrieve existing cluster config from Kubernetes ConfigMap
|
||||
existingConfig, err := c.retrieveDeploymentConfig()
|
||||
if err != nil {
|
||||
return true, fmt.Errorf("Failed to retrive DeploymentConfig for cluster [%s]. Error: %s", c.Namespace, err)
|
||||
return true, fmt.Errorf("Failed to retrieve DeploymentConfig for cluster [%s]. Error: %s", c.Namespace, err)
|
||||
}
|
||||
|
||||
clusterReconfiguration, err := c.createClusterReconfigurationSpec(existingConfig, clusterNodes, dro)
|
||||
|
||||
@@ -79,7 +79,7 @@ func (c *cluster) createClusterReconfigurationSpec(existingConfig edgefsv1.Clust
|
||||
if nodeDevConfig, ok := existingConfig.DevConfig[node.Name]; ok {
|
||||
devicesConfig = nodeDevConfig
|
||||
} else {
|
||||
// create ned devices config for new node
|
||||
// create new devices config for new node
|
||||
devicesConfig, err = c.createDevicesConfig(deploymentType, node, dro)
|
||||
if err != nil {
|
||||
logger.Warningf("Can't create DevicesConfig for %s node, Error: %+v", node.Name, err)
|
||||
@@ -150,7 +150,7 @@ func (c *cluster) getClusterFailureDomain() (string, error) {
|
||||
case "zone":
|
||||
failureDomain = "zone"
|
||||
default:
|
||||
return "", fmt.Errorf("Unknow failure domain %s, skipped", c.Spec.FailureDomain)
|
||||
return "", fmt.Errorf("Unknown failure domain %s, skipped", c.Spec.FailureDomain)
|
||||
}
|
||||
}
|
||||
return failureDomain, nil
|
||||
|
||||
@@ -112,7 +112,7 @@ func (c *cluster) createClusterConfigMap(deploymentConfig edgefsv1.ClusterDeploy
|
||||
// In resurrection case we only need to adjust networking selections
|
||||
// in ccow.json, ccowd.json and corosync.conf. And keep device transport
|
||||
// same as before. Resurrection is "best effort" feature, we cannot
|
||||
// guarnatee that cluster can be reconfigured, but at least we do try.
|
||||
// guarantee that cluster can be reconfigured, but at least we do try.
|
||||
|
||||
rtDevices = make([]edgefsv1.RTDevice, 0)
|
||||
rtSlaveDevices = make([]edgefsv1.RTDevices, 0)
|
||||
@@ -132,7 +132,7 @@ func (c *cluster) createClusterConfigMap(deploymentConfig edgefsv1.ClusterDeploy
|
||||
case "zone":
|
||||
failureDomain = 2
|
||||
default:
|
||||
logger.Infof("Unknow failure domain %s, skipped", c.Spec.FailureDomain)
|
||||
logger.Infof("Unknown failure domain %s, skipped", c.Spec.FailureDomain)
|
||||
}
|
||||
}
|
||||
commitWait := 1
|
||||
|
||||
@@ -105,7 +105,7 @@ func GetContainers(maxContainerCapacity int64, devices []sys.LocalDisk, storeCon
|
||||
return nil, fmt.Errorf("No available disks for container")
|
||||
}
|
||||
|
||||
// calculates max containers count for avalable devices
|
||||
// calculates max containers count for available devices
|
||||
numContainers := int(math.Ceil(float64(totalCapacity) / float64(maxCap)))
|
||||
maxSSDPerContainer := int(math.Ceil(float64(len(ssds)) / float64(numContainers)))
|
||||
maxHDDPerContainer := int(math.Ceil(float64(len(hdds)) / float64(numContainers)))
|
||||
|
||||
@@ -81,7 +81,7 @@ func GetK8SVersion(clientset kubernetes.Interface) (*version.Version, error) {
|
||||
|
||||
// Hash stableName computes a stable pseudorandom string suitable for inclusion in a Kubernetes object name from the given seed string.
|
||||
// Do **NOT** edit this function in a way that would change its output as it needs to
|
||||
// provide consistent mappings from string to hash accross versions of rook.
|
||||
// provide consistent mappings from string to hash across versions of rook.
|
||||
func Hash(s string) string {
|
||||
h := sha256.Sum256([]byte(s))
|
||||
return hex.EncodeToString(h[:16])
|
||||
@@ -94,7 +94,7 @@ func Hash(s string) string {
|
||||
// https://stackoverflow.com/a/50451893
|
||||
// https://stackoverflow.com/a/32294443
|
||||
// Do **NOT** edit this function in a way that would change its output as it needs to
|
||||
// provide consistent mappings from string to hash accross versions of rook.
|
||||
// provide consistent mappings from string to hash across versions of rook.
|
||||
func TruncateNodeName(format, nodeName string) string {
|
||||
if len(nodeName)+len(fmt.Sprintf(format, "")) > validation.DNS1035LabelMaxLength {
|
||||
hashed := Hash(nodeName)
|
||||
@@ -148,7 +148,7 @@ func deleteResourceAndWait(namespace, name, resourceType string,
|
||||
}
|
||||
|
||||
// Add the rook version to the labels. This should *not* be used on pod specifications, because this
|
||||
// will result in the deployment/daemonset/ect. recreating all of its pods even if an update
|
||||
// will result in the deployment/daemonset/etc. recreating all of its pods even if an update
|
||||
// wouldn't otherwise be required. Upgrading unnecessarily increases risk for loss of data
|
||||
// reliability, even if only briefly.
|
||||
//
|
||||
|
||||
@@ -67,7 +67,7 @@ func newServiceForNFSServer(cr *nfsv1alpha1.NFSServer) *corev1.Service {
|
||||
}
|
||||
|
||||
func newStatefulSetForNFSServer(cr *nfsv1alpha1.NFSServer) *appsv1.StatefulSet {
|
||||
privilaged := true
|
||||
privileged := true
|
||||
replicas := int32(cr.Spec.Replicas)
|
||||
return &appsv1.StatefulSet{
|
||||
ObjectMeta: metav1.ObjectMeta{
|
||||
@@ -117,7 +117,7 @@ func newStatefulSetForNFSServer(cr *nfsv1alpha1.NFSServer) *appsv1.StatefulSet {
|
||||
TerminationMessagePath: "/dev/termination-log",
|
||||
TerminationMessagePolicy: corev1.TerminationMessageReadFile,
|
||||
SecurityContext: &corev1.SecurityContext{
|
||||
Privileged: &privilaged,
|
||||
Privileged: &privileged,
|
||||
},
|
||||
},
|
||||
},
|
||||
|
||||
@@ -34,7 +34,7 @@ import (
|
||||
// Executor is the main interface for all the exec commands
|
||||
type Executor interface {
|
||||
ExecuteCommand(command string, arg ...string) error
|
||||
ExecuteCommandWithEnv(env []string, comand string, arg ...string) error
|
||||
ExecuteCommandWithEnv(env []string, command string, arg ...string) error
|
||||
ExecuteCommandWithOutput(command string, arg ...string) (string, error)
|
||||
ExecuteCommandWithCombinedOutput(command string, arg ...string) (string, error)
|
||||
ExecuteCommandWithOutputFile(command, outfileArg string, arg ...string) (string, error)
|
||||
|
||||
@@ -50,9 +50,9 @@ type CephManifests interface {
|
||||
GetPVCClone(cloneClaimName, parentClaimName, namespace, storageClassName, accessModes, size string) string
|
||||
GetSnapshot(snapshotName, claimName, snapshotClassName, namespace string) string
|
||||
GetPod(podName, claimName, namespace, mountPoint string, readOnly bool) string
|
||||
GetFilesystem(namepace, name string, activeCount int) string
|
||||
GetNFS(namepace, name, pool string, daemonCount int) string
|
||||
GetRBDMirror(namepace, name string, daemonCount int) string
|
||||
GetFilesystem(namespace, name string, activeCount int) string
|
||||
GetNFS(namespace, name, pool string, daemonCount int) string
|
||||
GetRBDMirror(namespace, name string, daemonCount int) string
|
||||
GetObjectStore(namespace, name string, replicaCount, port int) string
|
||||
GetObjectStoreUser(namespace, name, displayName, store string) string
|
||||
GetBucketStorageClass(namespace, storeName, storageClassName, reclaimPolicy, region string) string
|
||||
|
||||
@@ -50,7 +50,7 @@ func (y *YugabyteDBInstaller) InstallYugabyteDB(systemNS, ns string, count int)
|
||||
} else if !isDefStorageClassPresent {
|
||||
logger.Info("Default storage class not set. Creating one.")
|
||||
|
||||
// Mark the installationa attempt of a host path provisioner, for removal later.
|
||||
// Mark the installation attempt of a host path provisioner, for removal later.
|
||||
y.hostPathProvisionerInstalled = true
|
||||
|
||||
if err := InstallHostPathProvisioner(y.k8sHelper); err != nil {
|
||||
|
||||
@@ -44,7 +44,7 @@ import (
|
||||
storagev1util "k8s.io/kubernetes/pkg/apis/storage/v1/util"
|
||||
)
|
||||
|
||||
// K8sHelper is a helper for common kubectl commads
|
||||
// K8sHelper is a helper for common kubectl commands
|
||||
type K8sHelper struct {
|
||||
executor *exec.CommandExecutor
|
||||
Clientset *kubernetes.Clientset
|
||||
|
||||
@@ -11,7 +11,7 @@ This script uses k8s' CertificateSigningRequest API to a generate a
|
||||
certificate signed by k8s CA suitable for use with trace-context-injector webhook
|
||||
services. This requires permissions to create and approve CSR. See
|
||||
https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster for
|
||||
detailed explantion and additional instructions.
|
||||
detailed explanation and additional instructions.
|
||||
|
||||
The server key/cert k8s CA cert are stored in a k8s secret.
|
||||
|
||||
|
||||
Reference in new issue
Block a user