forked from rook/rook
noobaa: remove design docs due to inactivity
The NooBaa operator was proposed for addition to Rook, but the team went a different direction and created an independent operator. Removing the stale docs. Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit is contained in:
Travis Nielsen
@@ -1,14 +0,0 @@
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: noobaabackingstores.noobaa.rook.io
|
||||
spec:
|
||||
names:
|
||||
kind: NooBaaBackingStore
|
||||
plural: noobaabackingstores
|
||||
singular: noobaabackingstore
|
||||
listKind: NooBaaBackingStoreList
|
||||
shortNames: ["nbstore"]
|
||||
scope: Namespaced
|
||||
group: noobaa.rook.io
|
||||
version: v1alpha1
|
||||
@@ -1,10 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: aws-s3
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
type: aws-s3
|
||||
region: us-east-1
|
||||
bucketName: noobaa1-aws-backing-store
|
||||
credentialsSecretName: aws-credentials-secret
|
||||
@@ -1,9 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: azure-blob
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
type: azure-blob
|
||||
bucketName: noobaa1-azure-blob-backing-store
|
||||
credentialsSecretName: azure-blob-credentials-secret
|
||||
-10
@@ -1,10 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: gcs
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
type: google-cloud-storage
|
||||
region: us-west1
|
||||
bucketName: noobaa1-gcs-backing-store
|
||||
credentialsSecretName: gcs-credentials-secret
|
||||
@@ -1,9 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: aws-obc
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
type: obc
|
||||
bucketName: aws-obc-backing-store
|
||||
storageClassName: aws-obc-provisioner
|
||||
@@ -1,11 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: local-pv
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
type: pvc
|
||||
storageClassName: ceph-block-provisioner
|
||||
count: 3
|
||||
size: 30
|
||||
sizeUnits: GB
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: rgw
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
type: s3-compatible
|
||||
endpoint: s3.rook-ceph
|
||||
bucketName: noobaa1-rgw-backing-store
|
||||
credentialsSecretName: rgw-credentials-secret
|
||||
options:
|
||||
sslEnabled: false
|
||||
s3ForcePathStyle: true
|
||||
signatureVersion: v2
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: noobaabucketclasses.noobaa.rook.io
|
||||
spec:
|
||||
names:
|
||||
kind: NooBaaBucketClass
|
||||
plural: noobaabucketclasses
|
||||
singular: noobaabucketclass
|
||||
listKind: NooBaaBucketClassList
|
||||
shortNames: ["nbclass"]
|
||||
scope: Namespaced
|
||||
group: noobaa.rook.io
|
||||
version: v1alpha1
|
||||
@@ -1,16 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBucketClass
|
||||
metadata:
|
||||
name: noobaa-cloud-mirror-class
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
placementPolicy:
|
||||
tiers:
|
||||
- tier:
|
||||
mirrors:
|
||||
- mirror:
|
||||
spread:
|
||||
- noobaa-ceph-resource
|
||||
- mirror:
|
||||
spread:
|
||||
- noobaa-aws-resource
|
||||
@@ -1,13 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBucketClass
|
||||
metadata:
|
||||
name: noobaa-default-class
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
placementPolicy:
|
||||
tiers:
|
||||
- tier:
|
||||
mirrors:
|
||||
- mirror:
|
||||
spread:
|
||||
- noobaa-aws-resource
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: apiextensions.k8s.io/v1beta1
|
||||
kind: CustomResourceDefinition
|
||||
metadata:
|
||||
name: noobaasystems.noobaa.rook.io
|
||||
spec:
|
||||
names:
|
||||
kind: NooBaaSystem
|
||||
plural: noobaasystems
|
||||
singular: noobaasystem
|
||||
listKind: NooBaaSystemList
|
||||
shortNames: ["nb", "noobaa"]
|
||||
scope: Namespaced
|
||||
group: noobaa.rook.io
|
||||
version: v1alpha1
|
||||
@@ -1,18 +0,0 @@
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaSystem
|
||||
metadata:
|
||||
name: noobaa-system
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
|
||||
# Image is the container image to use for the deployment.
|
||||
image: noobaa/noobaa-core:4.0.0
|
||||
|
||||
endpoints:
|
||||
# Endpoints autoscale is used to define the HorizontalPodAutoscaler (HPA) settings for S3 endpoints.
|
||||
# See https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough
|
||||
# NooBaa endpoints are stateless components that consume CPU, memory, and network resources, and are designed to scale horizontally.
|
||||
# The operator will define the metrics and thresholds for scaling.
|
||||
autoscale:
|
||||
minReplicas: 1
|
||||
maxReplicas: 10
|
||||
@@ -1,22 +0,0 @@
|
||||
# ♜ Rook NooBaa Design
|
||||
|
||||
NooBaa is an object data service for hybrid and multi cloud environments. NooBaa runs on kubernetes, provides an S3 object store service (and Lambda with bucket triggers) to clients both inside and outside the cluster, and uses storage resources from within or outside the cluster, with flexible placement policies to automate data use cases.
|
||||
|
||||
[About NooBaa](noobaa-about.md)
|
||||
|
||||
# Operator Design
|
||||
|
||||
CRDs
|
||||
- [NooBaaSystem](noobaa-system.md) - The basic CRD to deploy a NooBaa service.
|
||||
- [NooBaaBackingStore](noobaa-backing-store.md) - Connection to cloud or local storage to use in policies.
|
||||
- [NooBaaBucketClass](noobaa-bucket-class.md) - Policies applied to a class of buckets.
|
||||
|
||||
Applications
|
||||
- [S3 Account](noobaa-s3-account.md) - Method to obtain S3 credentials for native S3 applications.
|
||||
- [OBC Provisioner](noobaa-obc-provisioner.md) - Method to claim a new/existing bucket.
|
||||
|
||||
Roadmap
|
||||
- Operator Lifecycle Manager integration [](noobaa-olm.md)
|
||||
- Scaling up/down S3 endpoints [](noobaa-endpoints.md)
|
||||
- Multi-cluster [](noobaa-multicluster.md)
|
||||
- Security [](noobaa-security.md)
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 484 KiB |
@@ -1,22 +0,0 @@
|
||||
♜ [Rook NooBaa Design](README.md) /
|
||||
# About NooBaa
|
||||
|
||||
- [NooBaa Github Project](https://github.com/noobaa/noobaa-core)
|
||||
- [NooBaa On Ceph Tech Talk (youtube)](https://www.youtube.com/watch?v=uW6NvsYFX-s)
|
||||
- [NooBaa From Zero To Multi Cloud (youtube)](https://www.youtube.com/watch?v=fuTKXBMwOes)
|
||||
- [OBC Design Document](https://github.com/yard-turkey/lib-bucket-provisioner/blob/master/doc/design/object-bucket-lib.md)
|
||||
|
||||
|
||||
# Terminology:
|
||||
|
||||
| Term | a.k.a | Description |
|
||||
|------|-------|-------------|
|
||||
| NooBaa Core | "The Brain" | |
|
||||
| NooBaa Endpoint | "S3 server" | stateless |
|
||||
| NooBaa Agent | "The Daemon" or "Storage Node" | stateful for local FS volume, embeds endpoint |
|
||||
|
||||
|
||||
# Main Components:
|
||||
|
||||
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
♜ [Rook NooBaa Design](README.md) /
|
||||
# NooBaaBackingStore
|
||||
|
||||
NooBaaBackingStore is a CRD representing a storage target to be used as underlying storage for the data in NooBaa buckets.
|
||||
These storage targets are used to store deduped+compressed+encrypted chunks of data (encryption keys are stored separatly).
|
||||
Backing-stores are referred to by name when defining [NooBaaBucketClass](noobaa-bucket-class.md).
|
||||
|
||||
Multiple types of backing-stores are supported: aws-s3, s3-compatible, google-cloud-storage, azure-blob, obc, pvc.
|
||||
Adding support for a new type of backing-store is rather easy as it requires just GET/PUT key-value store, see [Backing-stores supported by NooBaa](https://github.com/noobaa/noobaa-core/tree/master/src/agent/block_store_services).
|
||||
|
||||
|
||||
# Definitions
|
||||
|
||||
- CRD: [noobaa-backing-store-crd.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-crd.yaml)
|
||||
- Examples:
|
||||
1. [noobaa-backing-store-example-aws-s3.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-example-aws-s3.yaml)
|
||||
1. [noobaa-backing-store-example-s3-compatible.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-example-s3-compatible.yaml)
|
||||
1. [noobaa-backing-store-example-google-cloud-storage.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-example-google-cloud-storage.yaml)
|
||||
1. [noobaa-backing-store-example-azure-blob.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-example-azure-blob.yaml)
|
||||
1. [noobaa-backing-store-example-obc.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-example-obc.yaml)
|
||||
1. [noobaa-backing-store-example-pvc.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-backing-store-example-pvc.yaml)
|
||||
|
||||
|
||||
# Reconcile
|
||||
|
||||
#### OBC type
|
||||
|
||||
The operator will create a claim and the appropriate provisioner will create a new bucket or connect to existing one depending on the obc options. Once the claim is ready its details will be used to configure a cloud resource in NooBaa.
|
||||
|
||||
#### PVC type
|
||||
|
||||
Create a NooBaa storage agent StatefulSet with PVC mounted in each pod. Each agent will connect to the NooBaa brain and provide the PV filesystem storage to be used for storing encrypted chunks of data.
|
||||
|
||||
#### Credentials change
|
||||
|
||||
In case the credentials of a backing-store need to be updated due to a periodic security policy or concern, the appropriate secret should be updated by the user, and the operator will be responsible for watching changes in those secrets and propagating the new credential update to the NooBaa system server.
|
||||
|
||||
|
||||
# Read Status
|
||||
|
||||
Here is an example healthy status (see below example of non-healthy status):
|
||||
|
||||
```yaml
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: aws-s3
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
...
|
||||
status:
|
||||
health: OK
|
||||
issues: []
|
||||
```
|
||||
|
||||
|
||||
# Delete
|
||||
|
||||
Backing-stores are used for data persistency, therefore there is a cleanup process before they can be deleted.
|
||||
The operator will use the `finalizer` pattern as explained in the link below, and set a finalizer on every backing-store to mark that external cleanup is needed before it can be delete:
|
||||
|
||||
https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#finalizers
|
||||
|
||||
After marking a backing-store for deletion, the operator will notify the NooBaa server on the deletion which will enter a *decommissioning* state, in which NooBaa will attempt to rebuild the data to a new backing-store location. Once the decomissioning process completes the operator will remove the finalizer and allow the CR to be deleted.
|
||||
|
||||
There are cases where the decommissioning cannot complete due to inability to read the data from the backing-store that is already not serving - for example if the target bucket was already deleted or the credentials were invalidated or there is no network from the system to the backing-store service. In such cases the system status will be used to report these issues and suggest manual resolution for example:
|
||||
|
||||
```yaml
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBackingStore
|
||||
metadata:
|
||||
name: aws-s3
|
||||
namespace: rook-noobaa
|
||||
finalizers:
|
||||
- finalizer.noobaa.rook.io
|
||||
spec:
|
||||
...
|
||||
status:
|
||||
health: WARNING
|
||||
issues:
|
||||
- title: Backing-Store "aws" - Target bucket is missing / access denied
|
||||
createTime: "2019-06-04T13:05:35.473Z"
|
||||
lastTime: "2019-06-04T13:05:35.473Z"
|
||||
- title: Backing-Store "aws" - Cannot remove `finalizer.noobaa.rook.io` to complete deletion until the data rebuild process completes
|
||||
createTime: "2019-06-04T13:05:35.473Z"
|
||||
lastTime: "2019-06-04T13:05:35.473Z"
|
||||
troubleshooting: "https://github.com/noobaa/noobaa-core/wiki/Backing-store-finalizer-troubleshooting"
|
||||
```
|
||||
@@ -1,76 +0,0 @@
|
||||
♜ [Rook NooBaa Design](README.md) /
|
||||
# NooBaaBucketClass
|
||||
|
||||
NooBaaBucketClass is a CRD representing a class for buckets that defines policies for data placement and more.
|
||||
|
||||
Data placement capabilities are built as a multi-layer structure, here are the layers bottom-up:
|
||||
- Spread Layer - list of backing-stores, aggregates the storage of multiple stores.
|
||||
- Mirroring Layer - list of spread-layers, async-mirroring to all mirrors, with locality optimization.
|
||||
- Tiering Layer - list of mirroring-layers, push cold data to next tier.
|
||||
|
||||
For more information on using bucket-classes from S3 see [S3 Account](noobaa-s3-account.md).
|
||||
|
||||
# Definitions
|
||||
|
||||
- CRD: [noobaa-bucket-class-crd.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-bucket-class-crd.yaml)
|
||||
- Examples:
|
||||
- [noobaa-bucket-class-example-single.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-bucket-class-example-single.yaml)
|
||||
- [noobaa-bucket-class-example-mirror.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-bucket-class-example-mirror.yaml)
|
||||
|
||||
|
||||
# Reconcile
|
||||
|
||||
- The operator will verify that bucket-class is valid - i.e. that the backing-stores exist and can be used.
|
||||
- Changes to a bucket-class spec will be propagated to buckets that were instantiated from it.
|
||||
- Other than that the bucket-class is passive, just waiting there for new buckets to use it.
|
||||
|
||||
# Read Status
|
||||
|
||||
Here is an example of healthy status:
|
||||
|
||||
```yaml
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBucketClass
|
||||
metadata:
|
||||
name: noobaa-default-class
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
...
|
||||
status:
|
||||
health: OK
|
||||
buckets: 31
|
||||
issues: []
|
||||
```
|
||||
|
||||
# Delete
|
||||
|
||||
Deleting a bucket-class should not be possible as long as there are buckets referring to it.
|
||||
Since CRD's do not offer this level of control, the operator will use the `finalizer` pattern as explained in the link below, and set a finalizer on every bucket-class to mark that external cleanup is needed before it can be delete:
|
||||
|
||||
https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#finalizers
|
||||
|
||||
The status of the bucket-class will show the remaining buckets that prevent if from being deleted:
|
||||
|
||||
```yaml
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaBucketClass
|
||||
metadata:
|
||||
name: noobaa-default-class
|
||||
namespace: rook-noobaa
|
||||
finalizers:
|
||||
- finalizer.noobaa.rook.io
|
||||
spec:
|
||||
...
|
||||
status:
|
||||
health: WARNING
|
||||
buckets: 3
|
||||
issues:
|
||||
- title: Bucket-class "noobaa-default-class" - Cannot remove `finalizer.noobaa.rook.io` to complete deletion while it has buckets
|
||||
buckets:
|
||||
- bucket1
|
||||
- bucket2
|
||||
- bucket3
|
||||
createTime: "2019-06-04T13:05:35.473Z"
|
||||
lastTime: "2019-06-04T13:05:35.473Z"
|
||||
troubleshooting: "https://github.com/noobaa/noobaa-core/wiki/Bucket-class-finalizer-troubleshooting"
|
||||
```
|
||||
@@ -1,56 +0,0 @@
|
||||
♜ [Rook NooBaa Design](README.md) /
|
||||
# OBC Provisioner
|
||||
|
||||
Kubernetes natively supports dynamic provisioning for many types of file and block storage, but lacks support for object bucket provisioning.
|
||||
In order to provide a native provisioning of object storage buckets, the concept of Object Bucket Claim (OBC/OB) was introduced in a similar manner to Persistent Volume Claim (PVC/PV)
|
||||
|
||||
The `lib-bucket-provisioner` repo provides a library implementation and design to unify the implementations:
|
||||
|
||||
[OBC Design Document](https://github.com/yard-turkey/lib-bucket-provisioner/blob/master/doc/design/object-bucket-lib.md)
|
||||
|
||||
|
||||
# StorageClass
|
||||
|
||||
The administrator will create StorageClasses and control its visibility to app-owners using RBAC rules.
|
||||
|
||||
See https://github.com/yard-turkey/lib-bucket-provisioner/blob/master/deploy/storageClass.yaml
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
apiVersion: storage.k8s.io/v1
|
||||
kind: StorageClass
|
||||
metadata:
|
||||
name: noobaa-default-class
|
||||
provisioner: noobaa.rook.io/bucket
|
||||
parameters:
|
||||
reclaimPolicy: Delete
|
||||
backingStore: noobaa-aws-resource
|
||||
```
|
||||
|
||||
# OBC
|
||||
|
||||
Applications that require a bucket will create an OBC and refer to a storage class name.
|
||||
|
||||
See https://github.com/yard-turkey/lib-bucket-provisioner/blob/master/deploy/example-claim.yaml
|
||||
|
||||
The operator will watch for OBC's and fulfill the claims by create/find existing bucket in NooBaa, and will share a config map and a secret with the application in order to give it all the needed details to work with the bucket.
|
||||
|
||||
Example:
|
||||
|
||||
```yaml
|
||||
apiVersion: objectbucket.io/v1alpha1
|
||||
kind: ObjectBucketClaim
|
||||
metadata:
|
||||
name: my-bucket-claim
|
||||
spec:
|
||||
generateBucketName: "my-bucket-"
|
||||
storageClassName: noobaa-default-class
|
||||
SSL: false
|
||||
```
|
||||
|
||||
# Bucket Permissions and Sharing
|
||||
|
||||
The scope of bucket permissions will be at the namespace scope - this means that all the OBC's from the same namespace will receive S3 credentials that has permission to use any other bucket provisioned by that namespace. Notice that also listing buckets with these S3 credentials will return only the subset of buckets claimed by that namespace.
|
||||
|
||||
While there are cases that this namespace scope is not enough, it provides a simple model for sharing and privacy for the initial release.
|
||||
@@ -1,61 +0,0 @@
|
||||
♜ [Rook NooBaa Design](README.md) /
|
||||
# S3 Account
|
||||
|
||||
Native S3 applications need credentials (access-key, secret-key) and an endpoint in order to use the S3 API from the application.
|
||||
|
||||
In order to provide applications these credentials the application will set annotations `s3-account.noobaa.rook.io/*` as shown in the examples below, to request that the operator provide a secret with S3 credentials.
|
||||
|
||||
The annotations will be added to the service-account in order to create a matching identity in NooBaa and provide a secret with S3 credentials in the application namespace. The operator will fulfill the request by creating a NooBaa account and put the access-key, secret-key, and endpoint in a secret for the application.
|
||||
|
||||
Once the service-account is removed or the annotations are removed, the operator will delete the NooBaa account and the secret.
|
||||
|
||||
# Example
|
||||
|
||||
The annotations on the service-account:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: app-account
|
||||
namespace: app-namespace
|
||||
# These annotations are requesting the operator to create a secret called `s3-credentials`
|
||||
# such that `noobaa-default-class` is used for new buckets.
|
||||
annotations:
|
||||
s3-account.noobaa.rook.io/secret-name: s3-credentials
|
||||
s3-account.noobaa.rook.io/bucket-class: noobaa-default-class
|
||||
```
|
||||
|
||||
The operator will create a secret like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: s3-credentials
|
||||
namespace: app-namespace
|
||||
type: Opaque
|
||||
data:
|
||||
AWS_ACCESS_KEY_ID: XXXXX
|
||||
AWS_SECRET_ACCESS_KEY: YYYYY
|
||||
S3_ENDPOINT: ZZZZZ
|
||||
```
|
||||
|
||||
And the application deployments can load it to pods like this:
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: app-deployment
|
||||
spec:
|
||||
replicas: 42
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: app-container
|
||||
# Here we map the secret to the container env
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: s3-credentials
|
||||
```
|
||||
@@ -1,121 +0,0 @@
|
||||
♜ [Rook NooBaa Design](README.md) /
|
||||
# NooBaaSystem
|
||||
|
||||
NooBaaSystem is a CRD that represents a single installation of NooBaa that includes a set of sub-resources (backing-stores, bucket-classes, and buckets) and has a lifecycle as a single integrated system.
|
||||
|
||||
|
||||
# Definitions
|
||||
|
||||
- CRD: [noobaa-system-crd.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-system-crd.yaml)
|
||||
- Example: [noobaa-system-example.yaml](../../cluster/examples/kubernetes/noobaa/noobaa-system-example.yaml)
|
||||
|
||||
|
||||
# Reconcile
|
||||
|
||||
The operator watches for NooBaaSystem changes and reconcile them to apply the following deployment:
|
||||
|
||||
- Kubernetes Resources
|
||||
- General:
|
||||
- ServiceAccount
|
||||
- Role
|
||||
- RoleBinding
|
||||
- Core server:
|
||||
- StatefulSet
|
||||
- Service (mgmt)
|
||||
- Service (s3)
|
||||
- Route/Ingress
|
||||
- Endpoints:
|
||||
- Deployment
|
||||
- HorizontalPodAutoscaler
|
||||
- NooBaa Setup
|
||||
- Admin Account
|
||||
- Once the server is up and running the operator will call an API to setup a new system in the server which returns a secret token:
|
||||
- `noobaa-admin-mgmt-secret` will be created to contain the admin account token. It will be used for next API calls on this system.
|
||||
- `noobaa-admin-s3-secret` will be created with the admin S3 credentials to allow easy onboarding for the admin.
|
||||
- Internal backing-store
|
||||
- In order to reduce the friction of setting up backing-stores right after deployment, we create an internal backing-store with the following characteristics:
|
||||
- The internal store has limited size
|
||||
- It is using the core server PV for storage
|
||||
- It should not be used for production workloads.
|
||||
- Default bucket-class
|
||||
- This is a simple class that uses just the internal backing-store.
|
||||
- Once backing-stores are added the default class should be updated and existing data will automatically move from internal store to the new stores.
|
||||
- first.bucket
|
||||
- The operator will create a `first.bucket` using the default bucket-class.
|
||||
|
||||
|
||||
# Read Status
|
||||
|
||||
The operator will set the status of the NooBaaSystem to represent the current state of reconciling to the desired state.\
|
||||
Here is the example status structure as would be returned by a `kubectl get noobaa/noobaa-system -n rook-noobaa -o yaml`:
|
||||
|
||||
```yaml
|
||||
apiVersion: noobaa.rook.io/v1alpha1
|
||||
kind: NooBaaSystem
|
||||
metadata:
|
||||
name: noobaa-system
|
||||
namespace: rook-noobaa
|
||||
spec:
|
||||
# ...
|
||||
status:
|
||||
|
||||
health:
|
||||
backingStores: OK
|
||||
bucketClasses: OK
|
||||
buckets: OK
|
||||
accounts: OK
|
||||
issues: []
|
||||
|
||||
counters:
|
||||
backingStores: 2
|
||||
bucketClasses: 3
|
||||
buckets: 33
|
||||
accounts: 3
|
||||
|
||||
readme: |
|
||||
|
||||
Welcome to NooBaa
|
||||
|
||||
S3 Endpoint
|
||||
-----------
|
||||
- Access key : export AWS_ACCESS_KEY_ID=$(kubectl get secret noobaa-admin-s3-secret -n rook-noobaa -o json | jq -r '.data.AWS_ACCESS_KEY_ID|@base64d')
|
||||
- Secret key : export AWS_SECRET_ACCESS_KEY=$(kubectl get secret noobaa-admin-s3-secret -n rook-noobaa -o json | jq -r '.data.AWS_SECRET_ACCESS_KEY|@base64d')
|
||||
- External address : https://222.222.222.222:8443
|
||||
- ClusterIP address : https://s3.noobaa
|
||||
- NodePort address : http://192.168.99.100:30361
|
||||
- Port forwarding : kubectl port-forward -n noobaa service/s3 10443:443 # then open https://localhost:10443
|
||||
- aws-cli : alias s3="aws --endpoint https://localhost:10443 s3"
|
||||
|
||||
Management
|
||||
-------------
|
||||
- Username/password : kubectl get secret noobaa-admin-mgmt-secret -n rook-noobaa -o json | jq '.data|map_values(@base64d)'
|
||||
- External address : https://111.111.111.111:8443
|
||||
- ClusterIP address : https://noobaa-mgmt.noobaa:8443
|
||||
- Node port address : http://192.168.99.100:30785
|
||||
- Port forwarding : kubectl port-forward -n noobaa service/noobaa-mgmt 11443:8443 # then open https://localhost:11443
|
||||
```
|
||||
|
||||
Example health status when there is an issue with the availability of a backing-store:
|
||||
```yaml
|
||||
status:
|
||||
health:
|
||||
backingStores: WARNING
|
||||
bucketClasses: OK
|
||||
buckets: OK
|
||||
accounts: OK
|
||||
issues:
|
||||
- title: backingStore "aws" is not accessible
|
||||
createTime: "2019-06-04T13:05:35.473Z"
|
||||
lastTime: "2019-06-04T13:05:35.473Z"
|
||||
```
|
||||
|
||||
|
||||
# Delete
|
||||
|
||||
The operator will detect deletion of a system CR, and will followup by deleting all the owned resources.
|
||||
|
||||
In general we can leave this to Garbage Collection as described here:
|
||||
|
||||
https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/
|
||||
|
||||
While this works, and there are no known dependencies on the order to deletions, we will prefer to call it explicitly to handle finalizers faster, and leave the GC to edge cases.
|
||||
Reference in New Issue
Block a user