diff --git a/Documentation/ceph-object-multisite-crd.md b/Documentation/ceph-object-multisite-crd.md new file mode 100644 index 000000000..653dd012b --- /dev/null +++ b/Documentation/ceph-object-multisite-crd.md @@ -0,0 +1,84 @@ +--- +title: Object Multisite CRDs +weight: 2825 +indent: true +--- + +# Ceph Object Multisite CRDs + +The following CRDs enable Ceph object stores to isolate or replicate data via multisite. For more information on multisite, visit the [ceph-object-multisite](/Documentation/ceph-object-multisite.md) documentation. + +## Ceph Object Realm CRD + +Rook allows creation of a realm in a ceph cluster for object stores through the custom resource definitions (CRDs). The following settings are available for Ceph object store realms. + +### Sample + +```yaml +apiVersion: ceph.rook.io/v1 +kind: CephObjectRealm +metadata: + name: realm-a + namespace: rook-ceph +``` + +### Object Realm Settings + +#### Metadata + +* `name`: The name of the object realm to create +* `namespace`: The namespace of the Rook cluster where the object realm is created. + +## Ceph Object Zone Group CRD + +Rook allows creation of zone groups in a ceph cluster for object stores through the custom resource definitions (CRDs). The following settings are available for Ceph object store zone groups. + +### Sample + +```yaml +apiVersion: ceph.rook.io/v1 +kind: CephObjectZoneGroup +metadata: + name: zonegroup-a + namespace: rook-ceph +spec: + realm: realm-a +``` + +### Object Zone Group Settings + +#### Metadata + +* `name`: The name of the object zone group to create +* `namespace`: The namespace of the Rook cluster where the object zone group is created. + +#### Spec + +* `realm`: The object realm in which the zone group will be created. This matches the name of the object realm CRD. + +## Ceph Object Zone CRD + +Rook allows creation of zones in a ceph cluster for object stores through the custom resource definitions (CRDs). The following settings are available for Ceph object store zone. + +### Sample + +```yaml +apiVersion: ceph.rook.io/v1 +kind: CephObjectZone +metadata: + name: zone-a + namespace: rook-ceph +spec: + zonegroup: zonegroup-a +``` + +### Object Zone Settings + +#### Metadata + +* `name`: The name of the object zone to create +* `namespace`: The namespace of the Rook cluster where the object zone is created. + +#### Spec + +* `zonegroup`: The object zonegroup in which the zone will be created. This matches the name of the object zone group CRD. diff --git a/Documentation/ceph-object-multisite.md b/Documentation/ceph-object-multisite.md new file mode 100644 index 000000000..2467de251 --- /dev/null +++ b/Documentation/ceph-object-multisite.md @@ -0,0 +1,126 @@ +--- +title: Object Multisite +weight: 2250 +indent: true +--- +# NOTE + +At the moment there is no realm pull feature implemented for the ceph-object-realm CR at the moment. Thus syncing objects between Ceph clusters is not working yet. Completing this feature is going to be completed by the Rook 1.4 release. + +# Object Multisite + +Multisite is a feature of Ceph that allows object stores to replicate its data over multiple Ceph clusters. + +Multisite also allows object stores to be independent and isolated from other object stores in a cluster. + +When a ceph-object-store is created without the `zone` section; a realm, zone group, and zone is created with the same name as the ceph-object-store. + +Since it is the only ceph-object-store in the realm, the data in the ceph-object-store remain independent and isolated from others on the same cluster. + +When a ceph-object-store is created with the `zone` section, the ceph-object-store will join a custom created zone, zone group, and realm with a different than its own. + +This allows the ceph-object-store to replicate its data over multiple Ceph clusters. + +To review core multisite concepts please read the [ceph-multisite design overview](/design/ceph/object/ceph-multisite-overview.md). + +## Prerequisites + +This guide assumes a Rook cluster as explained in the [Quickstart](ceph-quickstart.md). + +# Creating Object Multisite + +If an admin wants to set up multisite on a Rook Ceph cluster, the admin should create: + + - A [realm](/Documentation/ceph-object-multisite-crd.md) + - A [zonegroup](/Documentation/ceph-object-multisite-crd.md) + - A [zone](/Documentation/ceph-object-multisite-crd.md) + - An [object-store](/Documentation/ceph-object-store.md) with the zone section + +object-multisite.yaml in the [examples](/cluster/examples/kubernetes/ceph/) directory can be used to create the multisite CRDs. +```console +kubectl create -f object-multisite.yaml +``` + +The first zone group created in a realm is the master zone group. The first zone created in a zone group is the master zone. + +When one of the multisite CRs (realm, zone group, zone) is deleted the underlying ceph realm/zone group/zone is not deleted. This must be done manually (see next section). + +For more information on the multisite CRDs please read [ceph-object-multisite-crd](ceph-object-multisite-crd.md). + +# Multisite Cleanup + +## Realm Deletion + +Changes made to the resource's configuration or deletion of the resource are not reflected on the Ceph cluster. + +When the ceph-object-realm resource is deleted or modified, the realm is not deleted from the Ceph cluster. Realm deletion must be done via the toolbox. + +### Deleting a Realm + +The Rook toolbox can modify the Ceph Multisite state via the radosgw-admin command. + +The following command, run via the toolbox, deletes the realm. + +```console +radosgw-admin realm rm --rgw-realm=realm-a +``` + +## Zone Group Deletion + +Changes made to the resource's configuration or deletion of the resource are not reflected on the Ceph cluster. + +When the ceph-object-zone group resource is deleted or modified, the zone group is not deleted from the Ceph cluster. Zone Group deletion must be done through the toolbox. + +### Deleting a Zone Group + +The Rook toolbox can modify the Ceph Multisite state via the radosgw-admin command. + +The following command, run via the toolbox, deletes the zone group. + +```console +radosgw-admin zonegroup delete --rgw-realm=realm-a --rgw-zonegroup=zone-group-a +radosgw-admin period update --commit --rgw-realm=realm-a --rgw-zone-group=zone-group-a +``` + +## Deleting and Reconfiguring the Ceph Object Zone + +Changes made to the resource's configuration or deletion of the resource are not reflected on the Ceph cluster. + +When the ceph-object-zone resource is deleted or modified, the zone is not deleted from the Ceph cluster. Zone deletion must be done through the toolbox. + +### Changing the Master Zone + +The Rook toolbox can change the master zone in a zone group. + +```console +radosgw-admin zone modify --rgw-realm=realm-a --rgw-zonegroup=zone-group-a --rgw-zone=zone-a --master +radosgw-admin zonegroup modify --rgw-realm=realm-a --rgw-zonegroup=zone-group-a --master +radosgw-admin period update --commit --rgw-realm=realm-a --rgw-zonegroup=zone-group-a --rgw-zone=zone-a +``` + +### Deleting Zone + +The Rook toolbox can modify the Ceph Multisite state via the radosgw-admin command. + +There are two scenarios possible when deleting a zone. +The following commands, run via the toolbox, deletes the zone if there is only one zone in the zone group. + +```console +radosgw-admin zone rm --rgw-realm=realm-a --rgw-zone-group=zone-group-a --rgw-zone=zone-a +radosgw-admin period update --commit --rgw-realm=realm-a --rgw-zone-group=zone-group-a --rgw-zone=zone-a +``` + +In the other scenario, there are more than one zones in a zone group. + +Care must be taken when changing which zone is the master zone. + +Please read the following [documentation](https://docs.ceph.com/docs/master/radosgw/multisite/#changing-the-metadata-master-zone) before running the below commands: + +The following commands, run via toolboxes, remove the zone from the zone group first, then delete the zone. + +```console +radosgw-admin zonegroup rm --rgw-realm=realm-a --rgw-zone-group=zone-group-a --rgw-zone=zone-a +radosgw-admin period update --commit --rgw-realm=realm-a --rgw-zone-group=zone-group-a --rgw-zone=zone-a +radosgw-admin zone rm --rgw-realm=realm-a --rgw-zone-group=zone-group-a --rgw-zone=zone-a +radosgw-admin period update --commit --rgw-realm=realm-a --rgw-zone-group=zone-group-a --rgw-zone=zone-a +``` diff --git a/Documentation/ceph-object-store-crd.md b/Documentation/ceph-object-store-crd.md index 4f93a9e9e..8b7b14459 100644 --- a/Documentation/ceph-object-store-crd.md +++ b/Documentation/ceph-object-store-crd.md @@ -66,6 +66,8 @@ spec: # requests: # cpu: "500m" # memory: "1024Mi" + #zone: + #name: zone-a ``` ## Object Store Settings @@ -97,6 +99,12 @@ The gateway settings correspond to the RGW daemon settings. * `resources`: Set resource requests/limits for the Gateway Pod(s), see [Resource Requirements/Limits](ceph-cluster-crd.md#resource-requirementslimits). * `priorityClassName`: Set priority class name for the Gateway Pod(s) +## Zone Settings + +The [zone](ceph-object-multisite.md) settings allow the object store to join custom created [ceph-object-zone](ceph-object-multisite-crd.md). + +* `name`: the name of the ceph-object-zone the object stores should be in. + ## Runtime settings ### MIME types diff --git a/Documentation/ceph-object.md b/Documentation/ceph-object.md index 9a7af19c1..b53cd8f52 100644 --- a/Documentation/ceph-object.md +++ b/Documentation/ceph-object.md @@ -279,3 +279,11 @@ To directly retrieve the secrets: kubectl -n rook-ceph get secret rook-ceph-object-user-my-store-my-user -o yaml | grep AccessKey | awk '{print $2}' | base64 --decode kubectl -n rook-ceph get secret rook-ceph-object-user-my-store-my-user -o yaml | grep SecretKey | awk '{print $2}' | base64 --decode ``` + +## Object Multisite + +Multisite is a feature of Ceph that allows object stores to replicate its data over multiple Ceph clusters. + +Multisite also allows object stores to be independent and isloated from other object stores in a cluster. + +For more information on multisite please read the [ceph multisite overview](ceph-object-multisite.md) for how to run it. diff --git a/cluster/charts/rook-ceph/templates/resources.yaml b/cluster/charts/rook-ceph/templates/resources.yaml index 41e50c737..4ae24fbe0 100644 --- a/cluster/charts/rook-ceph/templates/resources.yaml +++ b/cluster/charts/rook-ceph/templates/resources.yaml @@ -431,6 +431,57 @@ spec: --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition +metadata: + name: cephobjectrealms.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectRealm + listKind: CephObjectRealmList + plural: cephobjectrealms + singular: cephobjectrealm + shortNames: + - realm + - realms + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzonegroups.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZoneGroup + listKind: CephObjectZoneGroupList + plural: cephobjectzonegroups + singular: cephobjectzonegroup + shortNames: + - zonegroup + - zonegroups + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzones.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZone + listKind: CephObjectZoneList + plural: cephobjectzones + singular: cephobjectzone + shortNames: + - zone + - zones + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition metadata: name: cephblockpools.ceph.rook.io spec: diff --git a/cluster/examples/kubernetes/ceph/common.yaml b/cluster/examples/kubernetes/ceph/common.yaml index ce126c581..a9b7fc281 100644 --- a/cluster/examples/kubernetes/ceph/common.yaml +++ b/cluster/examples/kubernetes/ceph/common.yaml @@ -508,6 +508,60 @@ spec: subresources: status: {} # OLM: END CEPH OBJECT STORE USERS CRD +# OLM: BEGIN CEPH OBJECT REALM CRD +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectrealms.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectRealm + listKind: CephObjectRealmList + plural: cephobjectrealms + singular: cephobjectrealm + scope: Namespaced + version: v1 + subresources: + status: {} +# OLM: END CEPH OBJECT REALM CRD +# OLM: BEGIN CEPH OBJECT ZONEGROUP CRD +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzonegroups.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZoneGroup + listKind: CephObjectZoneGroupList + plural: cephobjectzonegroups + singular: cephobjectzonegroup + scope: Namespaced + version: v1 + subresources: + status: {} +# OLM: END CEPH OBJECT ZONEGROUP CRD +# OLM: BEGIN CEPH OBJECT ZONE CRD +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzones.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZone + listKind: CephObjectZoneList + plural: cephobjectzones + singular: cephobjectzone + scope: Namespaced + version: v1 + subresources: + status: {} +# OLM: END CEPH OBJECT ZONE CRD # OLM: BEGIN CEPH BLOCK POOL CRD --- apiVersion: apiextensions.k8s.io/v1beta1 diff --git a/cluster/examples/kubernetes/ceph/object-multisite.yaml b/cluster/examples/kubernetes/ceph/object-multisite.yaml new file mode 100644 index 000000000..213d21076 --- /dev/null +++ b/cluster/examples/kubernetes/ceph/object-multisite.yaml @@ -0,0 +1,51 @@ +################################################################################################################# +# Create an object store with settings for a test environment. Only a single OSD is required in this example. +# kubectl create -f object-multisite.yaml +################################################################################################################# +apiVersion: ceph.rook.io/v1 +kind: CephObjectRealm +metadata: + name: realm-a + namespace: rook-ceph +--- +apiVersion: ceph.rook.io/v1 +kind: CephObjectZoneGroup +metadata: + name: zonegroup-a + namespace: rook-ceph +spec: + realm: realm-a +--- +apiVersion: ceph.rook.io/v1 +kind: CephObjectZone +metadata: + name: zone-a + namespace: rook-ceph +spec: + zoneGroup: zonegroup-a +--- +apiVersion: ceph.rook.io/v1 +kind: CephObjectStore +metadata: + name: multisite-store + namespace: rook-ceph +spec: + metadataPool: + failureDomain: host + replicated: + size: 1 + requireSafeReplicaSize: true + dataPool: + failureDomain: host + replicated: + size: 1 + requireSafeReplicaSize: true + compressionMode: none + preservePoolsOnDelete: false + gateway: + type: s3 + port: 80 + securePort: + instances: 1 + zone: + name: zone-a diff --git a/cluster/examples/kubernetes/ceph/object-realm.yaml b/cluster/examples/kubernetes/ceph/object-realm.yaml new file mode 100644 index 000000000..cbfdc4b4c --- /dev/null +++ b/cluster/examples/kubernetes/ceph/object-realm.yaml @@ -0,0 +1,10 @@ +################################################################################################################# +# Create a multisite realm for a collection of object stores +# kubectl create -f object-realm.yaml +################################################################################################################# + +apiVersion: ceph.rook.io/v1 +kind: CephObjectRealm +metadata: + name: realm-a + namespace: rook-ceph diff --git a/cluster/examples/kubernetes/ceph/object-zone.yaml b/cluster/examples/kubernetes/ceph/object-zone.yaml new file mode 100644 index 000000000..57b9f53ba --- /dev/null +++ b/cluster/examples/kubernetes/ceph/object-zone.yaml @@ -0,0 +1,12 @@ +################################################################################################################# +# Create a multisite zone for a collection of object stores +# kubectl create -f object-zone.yaml +################################################################################################################# + +apiVersion: ceph.rook.io/v1 +kind: CephObjectZone +metadata: + name: zone-a + namespace: rook-ceph +spec: + zoneGroup: zonegroup-a diff --git a/cluster/examples/kubernetes/ceph/object-zonegroup.yaml b/cluster/examples/kubernetes/ceph/object-zonegroup.yaml new file mode 100644 index 000000000..fdb43ed45 --- /dev/null +++ b/cluster/examples/kubernetes/ceph/object-zonegroup.yaml @@ -0,0 +1,12 @@ +################################################################################################################# +# Create a multisite zonegroup for a collection of object stores +# kubectl create -f object-realm.yaml +################################################################################################################# + +apiVersion: ceph.rook.io/v1 +kind: CephObjectZoneGroup +metadata: + name: zonegroup-a + namespace: rook-ceph +spec: + realm: realm-a diff --git a/cluster/examples/kubernetes/ceph/object.yaml b/cluster/examples/kubernetes/ceph/object.yaml index 9249f52db..a3ef991f2 100644 --- a/cluster/examples/kubernetes/ceph/object.yaml +++ b/cluster/examples/kubernetes/ceph/object.yaml @@ -77,3 +77,5 @@ spec: # cpu: "500m" # memory: "1024Mi" # priorityClassName: my-priority-class + #zone: + #name: zone-a diff --git a/cluster/olm/ceph/generate-rook-csv.sh b/cluster/olm/ceph/generate-rook-csv.sh index 5ab127ed8..870a58e21 100755 --- a/cluster/olm/ceph/generate-rook-csv.sh +++ b/cluster/olm/ceph/generate-rook-csv.sh @@ -89,6 +89,9 @@ CEPH_CRD_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephclusters.ceph.rook.io.crd.y CEPH_BLOCK_POOLS_CRD_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephblockpools.ceph.rook.io.crd.yaml" CEPH_OBJECT_STORE_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephobjectstores.ceph.rook.io.crd.yaml" CEPH_OBJECT_STORE_USERS_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephobjectstoreusers.ceph.rook.io.crd.yaml" +CEPH_OBJECT_REALM_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephobjectrealms.ceph.rook.io.crd.yaml" +CEPH_OBJECT_ZONEGROUP_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephobjectzonegroups.ceph.rook.io.crd.yaml" +CEPH_OBJECT_ZONE_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephobjectzones.ceph.rook.io.crd.yaml" CEPH_FILESYSTEMS_CRD_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephfilesystems.ceph.rook.io.crd.yaml" CEPH_NFS_CRD_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephnfses.ceph.rook.io.crd.yaml" CEPH_CLIENT_CRD_YAML_FILE="$OLM_CATALOG_DIR/deploy/crds/cephclients.ceph.rook.io.crd.yaml" @@ -202,6 +205,9 @@ function generate_crds_yaml() { sed -n '/^# OLM: BEGIN CEPH CRD$/,/# OLM: END CEPH CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_CRD_YAML_FILE" sed -n '/^# OLM: BEGIN CEPH OBJECT STORE CRD$/,/# OLM: END CEPH OBJECT STORE CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_OBJECT_STORE_YAML_FILE" sed -n '/^# OLM: BEGIN CEPH OBJECT STORE USERS CRD$/,/# OLM: END CEPH OBJECT STORE USERS CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_OBJECT_STORE_USERS_YAML_FILE" + sed -n '/^# OLM: BEGIN CEPH OBJECT REALM CRD$/,/# OLM: END CEPH OBJECT REALM CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_OBJECT_REALM_YAML_FILE" + sed -n '/^# OLM: BEGIN CEPH OBJECT ZONEGROUP CRD$/,/# OLM: END CEPH OBJECT ZONEGROUP CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_OBJECT_ZONEGROUP_YAML_FILE" + sed -n '/^# OLM: BEGIN CEPH OBJECT ZONE CRD$/,/# OLM: END CEPH OBJECT ZONE CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_OBJECT_ZONE_YAML_FILE" sed -n '/^# OLM: BEGIN CEPH BLOCK POOL CRD$/,/# OLM: END CEPH BLOCK POOL CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_BLOCK_POOLS_CRD_YAML_FILE" sed -n '/^# OLM: BEGIN CEPH NFS CRD$/,/# OLM: END CEPH NFS CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_NFS_CRD_YAML_FILE" sed -n '/^# OLM: BEGIN CEPH CLIENT CRD$/,/# OLM: END CEPH CLIENT CRD$/p' "$COMMON_YAML_FILE" | grep -v '^#' > "$CEPH_CLIENT_CRD_YAML_FILE" diff --git a/pkg/apis/ceph.rook.io/v1/object.go b/pkg/apis/ceph.rook.io/v1/object.go new file mode 100644 index 000000000..f83b9697a --- /dev/null +++ b/pkg/apis/ceph.rook.io/v1/object.go @@ -0,0 +1,21 @@ +/* +Copyright 2018 The Rook Authors. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + +    http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package v1 + +func (s *ObjectStoreSpec) IsMultisite() bool { + return s.Zone.Name != "" +} diff --git a/pkg/apis/ceph.rook.io/v1/register.go b/pkg/apis/ceph.rook.io/v1/register.go index 80e25ea90..1ee141e88 100644 --- a/pkg/apis/ceph.rook.io/v1/register.go +++ b/pkg/apis/ceph.rook.io/v1/register.go @@ -68,6 +68,12 @@ func addKnownTypes(scheme *runtime.Scheme) error { &CephObjectStoreList{}, &CephObjectStoreUser{}, &CephObjectStoreUserList{}, + &CephObjectRealm{}, + &CephObjectRealmList{}, + &CephObjectZoneGroup{}, + &CephObjectZoneGroupList{}, + &CephObjectZone{}, + &CephObjectZoneList{}, &CephRBDMirror{}, &CephRBDMirrorList{}, ) diff --git a/pkg/apis/ceph.rook.io/v1/types.go b/pkg/apis/ceph.rook.io/v1/types.go index 124ecd145..fabcf0331 100755 --- a/pkg/apis/ceph.rook.io/v1/types.go +++ b/pkg/apis/ceph.rook.io/v1/types.go @@ -386,33 +386,9 @@ type ObjectStoreSpec struct { // The rgw pod info Gateway GatewaySpec `json:"gateway"` -} -// +genclient -// +genclient:noStatus -// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object - -type CephObjectStoreUser struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata"` - Spec ObjectStoreUserSpec `json:"spec"` - Status *Status `json:"status"` -} - -// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object - -type CephObjectStoreUserList struct { - metav1.TypeMeta `json:",inline"` - metav1.ListMeta `json:"metadata"` - Items []CephObjectStoreUser `json:"items"` -} - -// ObjectStoreUserSpec represent the spec of an Objectstoreuser -type ObjectStoreUserSpec struct { - //The store the user will be created in - Store string `json:"store,omitempty"` - //The display name for the ceph users - DisplayName string `json:"displayName,omitempty"` + // The multisite info + Zone ZoneSpec `json:"zone"` } type GatewaySpec struct { @@ -444,6 +420,111 @@ type GatewaySpec struct { PriorityClassName string `json:"priorityClassName,omitempty"` } +type ZoneSpec struct { + // RGW Zone the Object Store is in + Name string `json:"name"` +} + +// +genclient +// +genclient:noStatus +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectStoreUser struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata"` + Spec ObjectStoreUserSpec `json:"spec"` + Status *Status `json:"status"` +} + +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectStoreUserList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata"` + Items []CephObjectStoreUser `json:"items"` +} + +// ObjectStoreUserSpec represent the spec of an Objectstoreuser +type ObjectStoreUserSpec struct { + //The store the user will be created in + Store string `json:"store,omitempty"` + //The display name for the ceph users + DisplayName string `json:"displayName,omitempty"` +} + +// +genclient +// +genclient:noStatus +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectRealm struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata"` + Spec ObjectRealmSpec `json:"spec"` + Status *Status `json:"status"` +} + +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectRealmList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata"` + Items []CephObjectRealm `json:"items"` +} + +// ObjectRealmSpec represent the spec of an ObjectRealm +type ObjectRealmSpec struct { +} + +// +genclient +// +genclient:noStatus +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectZoneGroup struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata"` + Spec ObjectZoneGroupSpec `json:"spec"` + Status *Status `json:"status"` +} + +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectZoneGroupList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata"` + Items []CephObjectZoneGroup `json:"items"` +} + +// ObjectZoneGroupSpec represent the spec of an ObjecZoneGroup +type ObjectZoneGroupSpec struct { + //The display name for the ceph users + Realm string `json:"realm"` +} + +// +genclient +// +genclient:noStatus +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectZone struct { + metav1.TypeMeta `json:",inline"` + metav1.ObjectMeta `json:"metadata"` + Spec ObjectZoneSpec `json:"spec"` + Status *Status `json:"status"` +} + +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object + +type CephObjectZoneList struct { + metav1.TypeMeta `json:",inline"` + metav1.ListMeta `json:"metadata"` + Items []CephObjectZone `json:"items"` +} + +// ObjectZoneSpec represent the spec of an ObjectZone +type ObjectZoneSpec struct { + //The display name for the ceph users + ZoneGroup string `json:"zoneGroup"` +} + // +genclient // +genclient:noStatus // +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object diff --git a/pkg/apis/ceph.rook.io/v1/zz_generated.deepcopy.go b/pkg/apis/ceph.rook.io/v1/zz_generated.deepcopy.go index 2cbe4fa00..04e5de5cb 100644 --- a/pkg/apis/ceph.rook.io/v1/zz_generated.deepcopy.go +++ b/pkg/apis/ceph.rook.io/v1/zz_generated.deepcopy.go @@ -358,6 +358,71 @@ func (in *CephNFSList) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CephObjectRealm) DeepCopyInto(out *CephObjectRealm) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + out.Spec = in.Spec + if in.Status != nil { + in, out := &in.Status, &out.Status + *out = new(Status) + **out = **in + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CephObjectRealm. +func (in *CephObjectRealm) DeepCopy() *CephObjectRealm { + if in == nil { + return nil + } + out := new(CephObjectRealm) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CephObjectRealm) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CephObjectRealmList) DeepCopyInto(out *CephObjectRealmList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CephObjectRealm, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CephObjectRealmList. +func (in *CephObjectRealmList) DeepCopy() *CephObjectRealmList { + if in == nil { + return nil + } + out := new(CephObjectRealmList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CephObjectRealmList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CephObjectStore) DeepCopyInto(out *CephObjectStore) { *out = *in @@ -488,6 +553,136 @@ func (in *CephObjectStoreUserList) DeepCopyObject() runtime.Object { return nil } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CephObjectZone) DeepCopyInto(out *CephObjectZone) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + out.Spec = in.Spec + if in.Status != nil { + in, out := &in.Status, &out.Status + *out = new(Status) + **out = **in + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CephObjectZone. +func (in *CephObjectZone) DeepCopy() *CephObjectZone { + if in == nil { + return nil + } + out := new(CephObjectZone) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CephObjectZone) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CephObjectZoneGroup) DeepCopyInto(out *CephObjectZoneGroup) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + out.Spec = in.Spec + if in.Status != nil { + in, out := &in.Status, &out.Status + *out = new(Status) + **out = **in + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CephObjectZoneGroup. +func (in *CephObjectZoneGroup) DeepCopy() *CephObjectZoneGroup { + if in == nil { + return nil + } + out := new(CephObjectZoneGroup) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CephObjectZoneGroup) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CephObjectZoneGroupList) DeepCopyInto(out *CephObjectZoneGroupList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CephObjectZoneGroup, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CephObjectZoneGroupList. +func (in *CephObjectZoneGroupList) DeepCopy() *CephObjectZoneGroupList { + if in == nil { + return nil + } + out := new(CephObjectZoneGroupList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CephObjectZoneGroupList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *CephObjectZoneList) DeepCopyInto(out *CephObjectZoneList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]CephObjectZone, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new CephObjectZoneList. +func (in *CephObjectZoneList) DeepCopy() *CephObjectZoneList { + if in == nil { + return nil + } + out := new(CephObjectZoneList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *CephObjectZoneList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *CephRBDMirror) DeepCopyInto(out *CephRBDMirror) { *out = *in @@ -1066,12 +1261,29 @@ func (in *NetworkSpec) DeepCopy() *NetworkSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ObjectRealmSpec) DeepCopyInto(out *ObjectRealmSpec) { + *out = *in + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ObjectRealmSpec. +func (in *ObjectRealmSpec) DeepCopy() *ObjectRealmSpec { + if in == nil { + return nil + } + out := new(ObjectRealmSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *ObjectStoreSpec) DeepCopyInto(out *ObjectStoreSpec) { *out = *in out.MetadataPool = in.MetadataPool out.DataPool = in.DataPool in.Gateway.DeepCopyInto(&out.Gateway) + out.Zone = in.Zone return } @@ -1101,6 +1313,38 @@ func (in *ObjectStoreUserSpec) DeepCopy() *ObjectStoreUserSpec { return out } +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ObjectZoneGroupSpec) DeepCopyInto(out *ObjectZoneGroupSpec) { + *out = *in + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ObjectZoneGroupSpec. +func (in *ObjectZoneGroupSpec) DeepCopy() *ObjectZoneGroupSpec { + if in == nil { + return nil + } + out := new(ObjectZoneGroupSpec) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ObjectZoneSpec) DeepCopyInto(out *ObjectZoneSpec) { + *out = *in + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ObjectZoneSpec. +func (in *ObjectZoneSpec) DeepCopy() *ObjectZoneSpec { + if in == nil { + return nil + } + out := new(ObjectZoneSpec) + in.DeepCopyInto(out) + return out +} + // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. func (in *PoolSpec) DeepCopyInto(out *PoolSpec) { *out = *in @@ -1175,3 +1419,19 @@ func (in *Status) DeepCopy() *Status { in.DeepCopyInto(out) return out } + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *ZoneSpec) DeepCopyInto(out *ZoneSpec) { + *out = *in + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ZoneSpec. +func (in *ZoneSpec) DeepCopy() *ZoneSpec { + if in == nil { + return nil + } + out := new(ZoneSpec) + in.DeepCopyInto(out) + return out +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/ceph.rook.io_client.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/ceph.rook.io_client.go index e5c738c14..51dfb5ebd 100644 --- a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/ceph.rook.io_client.go +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/ceph.rook.io_client.go @@ -31,8 +31,11 @@ type CephV1Interface interface { CephClustersGetter CephFilesystemsGetter CephNFSesGetter + CephObjectRealmsGetter CephObjectStoresGetter CephObjectStoreUsersGetter + CephObjectZonesGetter + CephObjectZoneGroupsGetter CephRBDMirrorsGetter } @@ -61,6 +64,10 @@ func (c *CephV1Client) CephNFSes(namespace string) CephNFSInterface { return newCephNFSes(c, namespace) } +func (c *CephV1Client) CephObjectRealms(namespace string) CephObjectRealmInterface { + return newCephObjectRealms(c, namespace) +} + func (c *CephV1Client) CephObjectStores(namespace string) CephObjectStoreInterface { return newCephObjectStores(c, namespace) } @@ -69,6 +76,14 @@ func (c *CephV1Client) CephObjectStoreUsers(namespace string) CephObjectStoreUse return newCephObjectStoreUsers(c, namespace) } +func (c *CephV1Client) CephObjectZones(namespace string) CephObjectZoneInterface { + return newCephObjectZones(c, namespace) +} + +func (c *CephV1Client) CephObjectZoneGroups(namespace string) CephObjectZoneGroupInterface { + return newCephObjectZoneGroups(c, namespace) +} + func (c *CephV1Client) CephRBDMirrors(namespace string) CephRBDMirrorInterface { return newCephRBDMirrors(c, namespace) } diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectrealm.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectrealm.go new file mode 100644 index 000000000..bbcbaac0a --- /dev/null +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectrealm.go @@ -0,0 +1,174 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package v1 + +import ( + "time" + + v1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + scheme "github.com/rook/rook/pkg/client/clientset/versioned/scheme" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + rest "k8s.io/client-go/rest" +) + +// CephObjectRealmsGetter has a method to return a CephObjectRealmInterface. +// A group's client should implement this interface. +type CephObjectRealmsGetter interface { + CephObjectRealms(namespace string) CephObjectRealmInterface +} + +// CephObjectRealmInterface has methods to work with CephObjectRealm resources. +type CephObjectRealmInterface interface { + Create(*v1.CephObjectRealm) (*v1.CephObjectRealm, error) + Update(*v1.CephObjectRealm) (*v1.CephObjectRealm, error) + Delete(name string, options *metav1.DeleteOptions) error + DeleteCollection(options *metav1.DeleteOptions, listOptions metav1.ListOptions) error + Get(name string, options metav1.GetOptions) (*v1.CephObjectRealm, error) + List(opts metav1.ListOptions) (*v1.CephObjectRealmList, error) + Watch(opts metav1.ListOptions) (watch.Interface, error) + Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *v1.CephObjectRealm, err error) + CephObjectRealmExpansion +} + +// cephObjectRealms implements CephObjectRealmInterface +type cephObjectRealms struct { + client rest.Interface + ns string +} + +// newCephObjectRealms returns a CephObjectRealms +func newCephObjectRealms(c *CephV1Client, namespace string) *cephObjectRealms { + return &cephObjectRealms{ + client: c.RESTClient(), + ns: namespace, + } +} + +// Get takes name of the cephObjectRealm, and returns the corresponding cephObjectRealm object, and an error if there is any. +func (c *cephObjectRealms) Get(name string, options metav1.GetOptions) (result *v1.CephObjectRealm, err error) { + result = &v1.CephObjectRealm{} + err = c.client.Get(). + Namespace(c.ns). + Resource("cephobjectrealms"). + Name(name). + VersionedParams(&options, scheme.ParameterCodec). + Do(). + Into(result) + return +} + +// List takes label and field selectors, and returns the list of CephObjectRealms that match those selectors. +func (c *cephObjectRealms) List(opts metav1.ListOptions) (result *v1.CephObjectRealmList, err error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + result = &v1.CephObjectRealmList{} + err = c.client.Get(). + Namespace(c.ns). + Resource("cephobjectrealms"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Do(). + Into(result) + return +} + +// Watch returns a watch.Interface that watches the requested cephObjectRealms. +func (c *cephObjectRealms) Watch(opts metav1.ListOptions) (watch.Interface, error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + opts.Watch = true + return c.client.Get(). + Namespace(c.ns). + Resource("cephobjectrealms"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Watch() +} + +// Create takes the representation of a cephObjectRealm and creates it. Returns the server's representation of the cephObjectRealm, and an error, if there is any. +func (c *cephObjectRealms) Create(cephObjectRealm *v1.CephObjectRealm) (result *v1.CephObjectRealm, err error) { + result = &v1.CephObjectRealm{} + err = c.client.Post(). + Namespace(c.ns). + Resource("cephobjectrealms"). + Body(cephObjectRealm). + Do(). + Into(result) + return +} + +// Update takes the representation of a cephObjectRealm and updates it. Returns the server's representation of the cephObjectRealm, and an error, if there is any. +func (c *cephObjectRealms) Update(cephObjectRealm *v1.CephObjectRealm) (result *v1.CephObjectRealm, err error) { + result = &v1.CephObjectRealm{} + err = c.client.Put(). + Namespace(c.ns). + Resource("cephobjectrealms"). + Name(cephObjectRealm.Name). + Body(cephObjectRealm). + Do(). + Into(result) + return +} + +// Delete takes name of the cephObjectRealm and deletes it. Returns an error if one occurs. +func (c *cephObjectRealms) Delete(name string, options *metav1.DeleteOptions) error { + return c.client.Delete(). + Namespace(c.ns). + Resource("cephobjectrealms"). + Name(name). + Body(options). + Do(). + Error() +} + +// DeleteCollection deletes a collection of objects. +func (c *cephObjectRealms) DeleteCollection(options *metav1.DeleteOptions, listOptions metav1.ListOptions) error { + var timeout time.Duration + if listOptions.TimeoutSeconds != nil { + timeout = time.Duration(*listOptions.TimeoutSeconds) * time.Second + } + return c.client.Delete(). + Namespace(c.ns). + Resource("cephobjectrealms"). + VersionedParams(&listOptions, scheme.ParameterCodec). + Timeout(timeout). + Body(options). + Do(). + Error() +} + +// Patch applies the patch and returns the patched cephObjectRealm. +func (c *cephObjectRealms) Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *v1.CephObjectRealm, err error) { + result = &v1.CephObjectRealm{} + err = c.client.Patch(pt). + Namespace(c.ns). + Resource("cephobjectrealms"). + SubResource(subresources...). + Name(name). + Body(data). + Do(). + Into(result) + return +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectzone.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectzone.go new file mode 100644 index 000000000..0716a5dbb --- /dev/null +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectzone.go @@ -0,0 +1,174 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package v1 + +import ( + "time" + + v1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + scheme "github.com/rook/rook/pkg/client/clientset/versioned/scheme" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + rest "k8s.io/client-go/rest" +) + +// CephObjectZonesGetter has a method to return a CephObjectZoneInterface. +// A group's client should implement this interface. +type CephObjectZonesGetter interface { + CephObjectZones(namespace string) CephObjectZoneInterface +} + +// CephObjectZoneInterface has methods to work with CephObjectZone resources. +type CephObjectZoneInterface interface { + Create(*v1.CephObjectZone) (*v1.CephObjectZone, error) + Update(*v1.CephObjectZone) (*v1.CephObjectZone, error) + Delete(name string, options *metav1.DeleteOptions) error + DeleteCollection(options *metav1.DeleteOptions, listOptions metav1.ListOptions) error + Get(name string, options metav1.GetOptions) (*v1.CephObjectZone, error) + List(opts metav1.ListOptions) (*v1.CephObjectZoneList, error) + Watch(opts metav1.ListOptions) (watch.Interface, error) + Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *v1.CephObjectZone, err error) + CephObjectZoneExpansion +} + +// cephObjectZones implements CephObjectZoneInterface +type cephObjectZones struct { + client rest.Interface + ns string +} + +// newCephObjectZones returns a CephObjectZones +func newCephObjectZones(c *CephV1Client, namespace string) *cephObjectZones { + return &cephObjectZones{ + client: c.RESTClient(), + ns: namespace, + } +} + +// Get takes name of the cephObjectZone, and returns the corresponding cephObjectZone object, and an error if there is any. +func (c *cephObjectZones) Get(name string, options metav1.GetOptions) (result *v1.CephObjectZone, err error) { + result = &v1.CephObjectZone{} + err = c.client.Get(). + Namespace(c.ns). + Resource("cephobjectzones"). + Name(name). + VersionedParams(&options, scheme.ParameterCodec). + Do(). + Into(result) + return +} + +// List takes label and field selectors, and returns the list of CephObjectZones that match those selectors. +func (c *cephObjectZones) List(opts metav1.ListOptions) (result *v1.CephObjectZoneList, err error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + result = &v1.CephObjectZoneList{} + err = c.client.Get(). + Namespace(c.ns). + Resource("cephobjectzones"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Do(). + Into(result) + return +} + +// Watch returns a watch.Interface that watches the requested cephObjectZones. +func (c *cephObjectZones) Watch(opts metav1.ListOptions) (watch.Interface, error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + opts.Watch = true + return c.client.Get(). + Namespace(c.ns). + Resource("cephobjectzones"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Watch() +} + +// Create takes the representation of a cephObjectZone and creates it. Returns the server's representation of the cephObjectZone, and an error, if there is any. +func (c *cephObjectZones) Create(cephObjectZone *v1.CephObjectZone) (result *v1.CephObjectZone, err error) { + result = &v1.CephObjectZone{} + err = c.client.Post(). + Namespace(c.ns). + Resource("cephobjectzones"). + Body(cephObjectZone). + Do(). + Into(result) + return +} + +// Update takes the representation of a cephObjectZone and updates it. Returns the server's representation of the cephObjectZone, and an error, if there is any. +func (c *cephObjectZones) Update(cephObjectZone *v1.CephObjectZone) (result *v1.CephObjectZone, err error) { + result = &v1.CephObjectZone{} + err = c.client.Put(). + Namespace(c.ns). + Resource("cephobjectzones"). + Name(cephObjectZone.Name). + Body(cephObjectZone). + Do(). + Into(result) + return +} + +// Delete takes name of the cephObjectZone and deletes it. Returns an error if one occurs. +func (c *cephObjectZones) Delete(name string, options *metav1.DeleteOptions) error { + return c.client.Delete(). + Namespace(c.ns). + Resource("cephobjectzones"). + Name(name). + Body(options). + Do(). + Error() +} + +// DeleteCollection deletes a collection of objects. +func (c *cephObjectZones) DeleteCollection(options *metav1.DeleteOptions, listOptions metav1.ListOptions) error { + var timeout time.Duration + if listOptions.TimeoutSeconds != nil { + timeout = time.Duration(*listOptions.TimeoutSeconds) * time.Second + } + return c.client.Delete(). + Namespace(c.ns). + Resource("cephobjectzones"). + VersionedParams(&listOptions, scheme.ParameterCodec). + Timeout(timeout). + Body(options). + Do(). + Error() +} + +// Patch applies the patch and returns the patched cephObjectZone. +func (c *cephObjectZones) Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *v1.CephObjectZone, err error) { + result = &v1.CephObjectZone{} + err = c.client.Patch(pt). + Namespace(c.ns). + Resource("cephobjectzones"). + SubResource(subresources...). + Name(name). + Body(data). + Do(). + Into(result) + return +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectzonegroup.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectzonegroup.go new file mode 100644 index 000000000..b940cfb4f --- /dev/null +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/cephobjectzonegroup.go @@ -0,0 +1,174 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package v1 + +import ( + "time" + + v1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + scheme "github.com/rook/rook/pkg/client/clientset/versioned/scheme" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + rest "k8s.io/client-go/rest" +) + +// CephObjectZoneGroupsGetter has a method to return a CephObjectZoneGroupInterface. +// A group's client should implement this interface. +type CephObjectZoneGroupsGetter interface { + CephObjectZoneGroups(namespace string) CephObjectZoneGroupInterface +} + +// CephObjectZoneGroupInterface has methods to work with CephObjectZoneGroup resources. +type CephObjectZoneGroupInterface interface { + Create(*v1.CephObjectZoneGroup) (*v1.CephObjectZoneGroup, error) + Update(*v1.CephObjectZoneGroup) (*v1.CephObjectZoneGroup, error) + Delete(name string, options *metav1.DeleteOptions) error + DeleteCollection(options *metav1.DeleteOptions, listOptions metav1.ListOptions) error + Get(name string, options metav1.GetOptions) (*v1.CephObjectZoneGroup, error) + List(opts metav1.ListOptions) (*v1.CephObjectZoneGroupList, error) + Watch(opts metav1.ListOptions) (watch.Interface, error) + Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *v1.CephObjectZoneGroup, err error) + CephObjectZoneGroupExpansion +} + +// cephObjectZoneGroups implements CephObjectZoneGroupInterface +type cephObjectZoneGroups struct { + client rest.Interface + ns string +} + +// newCephObjectZoneGroups returns a CephObjectZoneGroups +func newCephObjectZoneGroups(c *CephV1Client, namespace string) *cephObjectZoneGroups { + return &cephObjectZoneGroups{ + client: c.RESTClient(), + ns: namespace, + } +} + +// Get takes name of the cephObjectZoneGroup, and returns the corresponding cephObjectZoneGroup object, and an error if there is any. +func (c *cephObjectZoneGroups) Get(name string, options metav1.GetOptions) (result *v1.CephObjectZoneGroup, err error) { + result = &v1.CephObjectZoneGroup{} + err = c.client.Get(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + Name(name). + VersionedParams(&options, scheme.ParameterCodec). + Do(). + Into(result) + return +} + +// List takes label and field selectors, and returns the list of CephObjectZoneGroups that match those selectors. +func (c *cephObjectZoneGroups) List(opts metav1.ListOptions) (result *v1.CephObjectZoneGroupList, err error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + result = &v1.CephObjectZoneGroupList{} + err = c.client.Get(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Do(). + Into(result) + return +} + +// Watch returns a watch.Interface that watches the requested cephObjectZoneGroups. +func (c *cephObjectZoneGroups) Watch(opts metav1.ListOptions) (watch.Interface, error) { + var timeout time.Duration + if opts.TimeoutSeconds != nil { + timeout = time.Duration(*opts.TimeoutSeconds) * time.Second + } + opts.Watch = true + return c.client.Get(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + VersionedParams(&opts, scheme.ParameterCodec). + Timeout(timeout). + Watch() +} + +// Create takes the representation of a cephObjectZoneGroup and creates it. Returns the server's representation of the cephObjectZoneGroup, and an error, if there is any. +func (c *cephObjectZoneGroups) Create(cephObjectZoneGroup *v1.CephObjectZoneGroup) (result *v1.CephObjectZoneGroup, err error) { + result = &v1.CephObjectZoneGroup{} + err = c.client.Post(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + Body(cephObjectZoneGroup). + Do(). + Into(result) + return +} + +// Update takes the representation of a cephObjectZoneGroup and updates it. Returns the server's representation of the cephObjectZoneGroup, and an error, if there is any. +func (c *cephObjectZoneGroups) Update(cephObjectZoneGroup *v1.CephObjectZoneGroup) (result *v1.CephObjectZoneGroup, err error) { + result = &v1.CephObjectZoneGroup{} + err = c.client.Put(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + Name(cephObjectZoneGroup.Name). + Body(cephObjectZoneGroup). + Do(). + Into(result) + return +} + +// Delete takes name of the cephObjectZoneGroup and deletes it. Returns an error if one occurs. +func (c *cephObjectZoneGroups) Delete(name string, options *metav1.DeleteOptions) error { + return c.client.Delete(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + Name(name). + Body(options). + Do(). + Error() +} + +// DeleteCollection deletes a collection of objects. +func (c *cephObjectZoneGroups) DeleteCollection(options *metav1.DeleteOptions, listOptions metav1.ListOptions) error { + var timeout time.Duration + if listOptions.TimeoutSeconds != nil { + timeout = time.Duration(*listOptions.TimeoutSeconds) * time.Second + } + return c.client.Delete(). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + VersionedParams(&listOptions, scheme.ParameterCodec). + Timeout(timeout). + Body(options). + Do(). + Error() +} + +// Patch applies the patch and returns the patched cephObjectZoneGroup. +func (c *cephObjectZoneGroups) Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *v1.CephObjectZoneGroup, err error) { + result = &v1.CephObjectZoneGroup{} + err = c.client.Patch(pt). + Namespace(c.ns). + Resource("cephobjectzonegroups"). + SubResource(subresources...). + Name(name). + Body(data). + Do(). + Into(result) + return +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_ceph.rook.io_client.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_ceph.rook.io_client.go index 6a34de765..3a81884e9 100644 --- a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_ceph.rook.io_client.go +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_ceph.rook.io_client.go @@ -48,6 +48,10 @@ func (c *FakeCephV1) CephNFSes(namespace string) v1.CephNFSInterface { return &FakeCephNFSes{c, namespace} } +func (c *FakeCephV1) CephObjectRealms(namespace string) v1.CephObjectRealmInterface { + return &FakeCephObjectRealms{c, namespace} +} + func (c *FakeCephV1) CephObjectStores(namespace string) v1.CephObjectStoreInterface { return &FakeCephObjectStores{c, namespace} } @@ -56,6 +60,14 @@ func (c *FakeCephV1) CephObjectStoreUsers(namespace string) v1.CephObjectStoreUs return &FakeCephObjectStoreUsers{c, namespace} } +func (c *FakeCephV1) CephObjectZones(namespace string) v1.CephObjectZoneInterface { + return &FakeCephObjectZones{c, namespace} +} + +func (c *FakeCephV1) CephObjectZoneGroups(namespace string) v1.CephObjectZoneGroupInterface { + return &FakeCephObjectZoneGroups{c, namespace} +} + func (c *FakeCephV1) CephRBDMirrors(namespace string) v1.CephRBDMirrorInterface { return &FakeCephRBDMirrors{c, namespace} } diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectrealm.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectrealm.go new file mode 100644 index 000000000..c630a8815 --- /dev/null +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectrealm.go @@ -0,0 +1,128 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + cephrookiov1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + labels "k8s.io/apimachinery/pkg/labels" + schema "k8s.io/apimachinery/pkg/runtime/schema" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + testing "k8s.io/client-go/testing" +) + +// FakeCephObjectRealms implements CephObjectRealmInterface +type FakeCephObjectRealms struct { + Fake *FakeCephV1 + ns string +} + +var cephobjectrealmsResource = schema.GroupVersionResource{Group: "ceph.rook.io", Version: "v1", Resource: "cephobjectrealms"} + +var cephobjectrealmsKind = schema.GroupVersionKind{Group: "ceph.rook.io", Version: "v1", Kind: "CephObjectRealm"} + +// Get takes name of the cephObjectRealm, and returns the corresponding cephObjectRealm object, and an error if there is any. +func (c *FakeCephObjectRealms) Get(name string, options v1.GetOptions) (result *cephrookiov1.CephObjectRealm, err error) { + obj, err := c.Fake. + Invokes(testing.NewGetAction(cephobjectrealmsResource, c.ns, name), &cephrookiov1.CephObjectRealm{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectRealm), err +} + +// List takes label and field selectors, and returns the list of CephObjectRealms that match those selectors. +func (c *FakeCephObjectRealms) List(opts v1.ListOptions) (result *cephrookiov1.CephObjectRealmList, err error) { + obj, err := c.Fake. + Invokes(testing.NewListAction(cephobjectrealmsResource, cephobjectrealmsKind, c.ns, opts), &cephrookiov1.CephObjectRealmList{}) + + if obj == nil { + return nil, err + } + + label, _, _ := testing.ExtractFromListOptions(opts) + if label == nil { + label = labels.Everything() + } + list := &cephrookiov1.CephObjectRealmList{ListMeta: obj.(*cephrookiov1.CephObjectRealmList).ListMeta} + for _, item := range obj.(*cephrookiov1.CephObjectRealmList).Items { + if label.Matches(labels.Set(item.Labels)) { + list.Items = append(list.Items, item) + } + } + return list, err +} + +// Watch returns a watch.Interface that watches the requested cephObjectRealms. +func (c *FakeCephObjectRealms) Watch(opts v1.ListOptions) (watch.Interface, error) { + return c.Fake. + InvokesWatch(testing.NewWatchAction(cephobjectrealmsResource, c.ns, opts)) + +} + +// Create takes the representation of a cephObjectRealm and creates it. Returns the server's representation of the cephObjectRealm, and an error, if there is any. +func (c *FakeCephObjectRealms) Create(cephObjectRealm *cephrookiov1.CephObjectRealm) (result *cephrookiov1.CephObjectRealm, err error) { + obj, err := c.Fake. + Invokes(testing.NewCreateAction(cephobjectrealmsResource, c.ns, cephObjectRealm), &cephrookiov1.CephObjectRealm{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectRealm), err +} + +// Update takes the representation of a cephObjectRealm and updates it. Returns the server's representation of the cephObjectRealm, and an error, if there is any. +func (c *FakeCephObjectRealms) Update(cephObjectRealm *cephrookiov1.CephObjectRealm) (result *cephrookiov1.CephObjectRealm, err error) { + obj, err := c.Fake. + Invokes(testing.NewUpdateAction(cephobjectrealmsResource, c.ns, cephObjectRealm), &cephrookiov1.CephObjectRealm{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectRealm), err +} + +// Delete takes name of the cephObjectRealm and deletes it. Returns an error if one occurs. +func (c *FakeCephObjectRealms) Delete(name string, options *v1.DeleteOptions) error { + _, err := c.Fake. + Invokes(testing.NewDeleteAction(cephobjectrealmsResource, c.ns, name), &cephrookiov1.CephObjectRealm{}) + + return err +} + +// DeleteCollection deletes a collection of objects. +func (c *FakeCephObjectRealms) DeleteCollection(options *v1.DeleteOptions, listOptions v1.ListOptions) error { + action := testing.NewDeleteCollectionAction(cephobjectrealmsResource, c.ns, listOptions) + + _, err := c.Fake.Invokes(action, &cephrookiov1.CephObjectRealmList{}) + return err +} + +// Patch applies the patch and returns the patched cephObjectRealm. +func (c *FakeCephObjectRealms) Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *cephrookiov1.CephObjectRealm, err error) { + obj, err := c.Fake. + Invokes(testing.NewPatchSubresourceAction(cephobjectrealmsResource, c.ns, name, pt, data, subresources...), &cephrookiov1.CephObjectRealm{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectRealm), err +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectzone.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectzone.go new file mode 100644 index 000000000..bbedc43a0 --- /dev/null +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectzone.go @@ -0,0 +1,128 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + cephrookiov1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + labels "k8s.io/apimachinery/pkg/labels" + schema "k8s.io/apimachinery/pkg/runtime/schema" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + testing "k8s.io/client-go/testing" +) + +// FakeCephObjectZones implements CephObjectZoneInterface +type FakeCephObjectZones struct { + Fake *FakeCephV1 + ns string +} + +var cephobjectzonesResource = schema.GroupVersionResource{Group: "ceph.rook.io", Version: "v1", Resource: "cephobjectzones"} + +var cephobjectzonesKind = schema.GroupVersionKind{Group: "ceph.rook.io", Version: "v1", Kind: "CephObjectZone"} + +// Get takes name of the cephObjectZone, and returns the corresponding cephObjectZone object, and an error if there is any. +func (c *FakeCephObjectZones) Get(name string, options v1.GetOptions) (result *cephrookiov1.CephObjectZone, err error) { + obj, err := c.Fake. + Invokes(testing.NewGetAction(cephobjectzonesResource, c.ns, name), &cephrookiov1.CephObjectZone{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZone), err +} + +// List takes label and field selectors, and returns the list of CephObjectZones that match those selectors. +func (c *FakeCephObjectZones) List(opts v1.ListOptions) (result *cephrookiov1.CephObjectZoneList, err error) { + obj, err := c.Fake. + Invokes(testing.NewListAction(cephobjectzonesResource, cephobjectzonesKind, c.ns, opts), &cephrookiov1.CephObjectZoneList{}) + + if obj == nil { + return nil, err + } + + label, _, _ := testing.ExtractFromListOptions(opts) + if label == nil { + label = labels.Everything() + } + list := &cephrookiov1.CephObjectZoneList{ListMeta: obj.(*cephrookiov1.CephObjectZoneList).ListMeta} + for _, item := range obj.(*cephrookiov1.CephObjectZoneList).Items { + if label.Matches(labels.Set(item.Labels)) { + list.Items = append(list.Items, item) + } + } + return list, err +} + +// Watch returns a watch.Interface that watches the requested cephObjectZones. +func (c *FakeCephObjectZones) Watch(opts v1.ListOptions) (watch.Interface, error) { + return c.Fake. + InvokesWatch(testing.NewWatchAction(cephobjectzonesResource, c.ns, opts)) + +} + +// Create takes the representation of a cephObjectZone and creates it. Returns the server's representation of the cephObjectZone, and an error, if there is any. +func (c *FakeCephObjectZones) Create(cephObjectZone *cephrookiov1.CephObjectZone) (result *cephrookiov1.CephObjectZone, err error) { + obj, err := c.Fake. + Invokes(testing.NewCreateAction(cephobjectzonesResource, c.ns, cephObjectZone), &cephrookiov1.CephObjectZone{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZone), err +} + +// Update takes the representation of a cephObjectZone and updates it. Returns the server's representation of the cephObjectZone, and an error, if there is any. +func (c *FakeCephObjectZones) Update(cephObjectZone *cephrookiov1.CephObjectZone) (result *cephrookiov1.CephObjectZone, err error) { + obj, err := c.Fake. + Invokes(testing.NewUpdateAction(cephobjectzonesResource, c.ns, cephObjectZone), &cephrookiov1.CephObjectZone{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZone), err +} + +// Delete takes name of the cephObjectZone and deletes it. Returns an error if one occurs. +func (c *FakeCephObjectZones) Delete(name string, options *v1.DeleteOptions) error { + _, err := c.Fake. + Invokes(testing.NewDeleteAction(cephobjectzonesResource, c.ns, name), &cephrookiov1.CephObjectZone{}) + + return err +} + +// DeleteCollection deletes a collection of objects. +func (c *FakeCephObjectZones) DeleteCollection(options *v1.DeleteOptions, listOptions v1.ListOptions) error { + action := testing.NewDeleteCollectionAction(cephobjectzonesResource, c.ns, listOptions) + + _, err := c.Fake.Invokes(action, &cephrookiov1.CephObjectZoneList{}) + return err +} + +// Patch applies the patch and returns the patched cephObjectZone. +func (c *FakeCephObjectZones) Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *cephrookiov1.CephObjectZone, err error) { + obj, err := c.Fake. + Invokes(testing.NewPatchSubresourceAction(cephobjectzonesResource, c.ns, name, pt, data, subresources...), &cephrookiov1.CephObjectZone{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZone), err +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectzonegroup.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectzonegroup.go new file mode 100644 index 000000000..563191e47 --- /dev/null +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/fake/fake_cephobjectzonegroup.go @@ -0,0 +1,128 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by client-gen. DO NOT EDIT. + +package fake + +import ( + cephrookiov1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + labels "k8s.io/apimachinery/pkg/labels" + schema "k8s.io/apimachinery/pkg/runtime/schema" + types "k8s.io/apimachinery/pkg/types" + watch "k8s.io/apimachinery/pkg/watch" + testing "k8s.io/client-go/testing" +) + +// FakeCephObjectZoneGroups implements CephObjectZoneGroupInterface +type FakeCephObjectZoneGroups struct { + Fake *FakeCephV1 + ns string +} + +var cephobjectzonegroupsResource = schema.GroupVersionResource{Group: "ceph.rook.io", Version: "v1", Resource: "cephobjectzonegroups"} + +var cephobjectzonegroupsKind = schema.GroupVersionKind{Group: "ceph.rook.io", Version: "v1", Kind: "CephObjectZoneGroup"} + +// Get takes name of the cephObjectZoneGroup, and returns the corresponding cephObjectZoneGroup object, and an error if there is any. +func (c *FakeCephObjectZoneGroups) Get(name string, options v1.GetOptions) (result *cephrookiov1.CephObjectZoneGroup, err error) { + obj, err := c.Fake. + Invokes(testing.NewGetAction(cephobjectzonegroupsResource, c.ns, name), &cephrookiov1.CephObjectZoneGroup{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZoneGroup), err +} + +// List takes label and field selectors, and returns the list of CephObjectZoneGroups that match those selectors. +func (c *FakeCephObjectZoneGroups) List(opts v1.ListOptions) (result *cephrookiov1.CephObjectZoneGroupList, err error) { + obj, err := c.Fake. + Invokes(testing.NewListAction(cephobjectzonegroupsResource, cephobjectzonegroupsKind, c.ns, opts), &cephrookiov1.CephObjectZoneGroupList{}) + + if obj == nil { + return nil, err + } + + label, _, _ := testing.ExtractFromListOptions(opts) + if label == nil { + label = labels.Everything() + } + list := &cephrookiov1.CephObjectZoneGroupList{ListMeta: obj.(*cephrookiov1.CephObjectZoneGroupList).ListMeta} + for _, item := range obj.(*cephrookiov1.CephObjectZoneGroupList).Items { + if label.Matches(labels.Set(item.Labels)) { + list.Items = append(list.Items, item) + } + } + return list, err +} + +// Watch returns a watch.Interface that watches the requested cephObjectZoneGroups. +func (c *FakeCephObjectZoneGroups) Watch(opts v1.ListOptions) (watch.Interface, error) { + return c.Fake. + InvokesWatch(testing.NewWatchAction(cephobjectzonegroupsResource, c.ns, opts)) + +} + +// Create takes the representation of a cephObjectZoneGroup and creates it. Returns the server's representation of the cephObjectZoneGroup, and an error, if there is any. +func (c *FakeCephObjectZoneGroups) Create(cephObjectZoneGroup *cephrookiov1.CephObjectZoneGroup) (result *cephrookiov1.CephObjectZoneGroup, err error) { + obj, err := c.Fake. + Invokes(testing.NewCreateAction(cephobjectzonegroupsResource, c.ns, cephObjectZoneGroup), &cephrookiov1.CephObjectZoneGroup{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZoneGroup), err +} + +// Update takes the representation of a cephObjectZoneGroup and updates it. Returns the server's representation of the cephObjectZoneGroup, and an error, if there is any. +func (c *FakeCephObjectZoneGroups) Update(cephObjectZoneGroup *cephrookiov1.CephObjectZoneGroup) (result *cephrookiov1.CephObjectZoneGroup, err error) { + obj, err := c.Fake. + Invokes(testing.NewUpdateAction(cephobjectzonegroupsResource, c.ns, cephObjectZoneGroup), &cephrookiov1.CephObjectZoneGroup{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZoneGroup), err +} + +// Delete takes name of the cephObjectZoneGroup and deletes it. Returns an error if one occurs. +func (c *FakeCephObjectZoneGroups) Delete(name string, options *v1.DeleteOptions) error { + _, err := c.Fake. + Invokes(testing.NewDeleteAction(cephobjectzonegroupsResource, c.ns, name), &cephrookiov1.CephObjectZoneGroup{}) + + return err +} + +// DeleteCollection deletes a collection of objects. +func (c *FakeCephObjectZoneGroups) DeleteCollection(options *v1.DeleteOptions, listOptions v1.ListOptions) error { + action := testing.NewDeleteCollectionAction(cephobjectzonegroupsResource, c.ns, listOptions) + + _, err := c.Fake.Invokes(action, &cephrookiov1.CephObjectZoneGroupList{}) + return err +} + +// Patch applies the patch and returns the patched cephObjectZoneGroup. +func (c *FakeCephObjectZoneGroups) Patch(name string, pt types.PatchType, data []byte, subresources ...string) (result *cephrookiov1.CephObjectZoneGroup, err error) { + obj, err := c.Fake. + Invokes(testing.NewPatchSubresourceAction(cephobjectzonegroupsResource, c.ns, name, pt, data, subresources...), &cephrookiov1.CephObjectZoneGroup{}) + + if obj == nil { + return nil, err + } + return obj.(*cephrookiov1.CephObjectZoneGroup), err +} diff --git a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/generated_expansion.go b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/generated_expansion.go index 4f076c122..8086a7534 100644 --- a/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/generated_expansion.go +++ b/pkg/client/clientset/versioned/typed/ceph.rook.io/v1/generated_expansion.go @@ -28,8 +28,14 @@ type CephFilesystemExpansion interface{} type CephNFSExpansion interface{} +type CephObjectRealmExpansion interface{} + type CephObjectStoreExpansion interface{} type CephObjectStoreUserExpansion interface{} +type CephObjectZoneExpansion interface{} + +type CephObjectZoneGroupExpansion interface{} + type CephRBDMirrorExpansion interface{} diff --git a/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectrealm.go b/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectrealm.go new file mode 100644 index 000000000..b706beb85 --- /dev/null +++ b/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectrealm.go @@ -0,0 +1,89 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package v1 + +import ( + time "time" + + cephrookiov1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + versioned "github.com/rook/rook/pkg/client/clientset/versioned" + internalinterfaces "github.com/rook/rook/pkg/client/informers/externalversions/internalinterfaces" + v1 "github.com/rook/rook/pkg/client/listers/ceph.rook.io/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + watch "k8s.io/apimachinery/pkg/watch" + cache "k8s.io/client-go/tools/cache" +) + +// CephObjectRealmInformer provides access to a shared informer and lister for +// CephObjectRealms. +type CephObjectRealmInformer interface { + Informer() cache.SharedIndexInformer + Lister() v1.CephObjectRealmLister +} + +type cephObjectRealmInformer struct { + factory internalinterfaces.SharedInformerFactory + tweakListOptions internalinterfaces.TweakListOptionsFunc + namespace string +} + +// NewCephObjectRealmInformer constructs a new informer for CephObjectRealm type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewCephObjectRealmInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers) cache.SharedIndexInformer { + return NewFilteredCephObjectRealmInformer(client, namespace, resyncPeriod, indexers, nil) +} + +// NewFilteredCephObjectRealmInformer constructs a new informer for CephObjectRealm type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewFilteredCephObjectRealmInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers, tweakListOptions internalinterfaces.TweakListOptionsFunc) cache.SharedIndexInformer { + return cache.NewSharedIndexInformer( + &cache.ListWatch{ + ListFunc: func(options metav1.ListOptions) (runtime.Object, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CephV1().CephObjectRealms(namespace).List(options) + }, + WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CephV1().CephObjectRealms(namespace).Watch(options) + }, + }, + &cephrookiov1.CephObjectRealm{}, + resyncPeriod, + indexers, + ) +} + +func (f *cephObjectRealmInformer) defaultInformer(client versioned.Interface, resyncPeriod time.Duration) cache.SharedIndexInformer { + return NewFilteredCephObjectRealmInformer(client, f.namespace, resyncPeriod, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, f.tweakListOptions) +} + +func (f *cephObjectRealmInformer) Informer() cache.SharedIndexInformer { + return f.factory.InformerFor(&cephrookiov1.CephObjectRealm{}, f.defaultInformer) +} + +func (f *cephObjectRealmInformer) Lister() v1.CephObjectRealmLister { + return v1.NewCephObjectRealmLister(f.Informer().GetIndexer()) +} diff --git a/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectzone.go b/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectzone.go new file mode 100644 index 000000000..a72c8b6c8 --- /dev/null +++ b/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectzone.go @@ -0,0 +1,89 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package v1 + +import ( + time "time" + + cephrookiov1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + versioned "github.com/rook/rook/pkg/client/clientset/versioned" + internalinterfaces "github.com/rook/rook/pkg/client/informers/externalversions/internalinterfaces" + v1 "github.com/rook/rook/pkg/client/listers/ceph.rook.io/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + watch "k8s.io/apimachinery/pkg/watch" + cache "k8s.io/client-go/tools/cache" +) + +// CephObjectZoneInformer provides access to a shared informer and lister for +// CephObjectZones. +type CephObjectZoneInformer interface { + Informer() cache.SharedIndexInformer + Lister() v1.CephObjectZoneLister +} + +type cephObjectZoneInformer struct { + factory internalinterfaces.SharedInformerFactory + tweakListOptions internalinterfaces.TweakListOptionsFunc + namespace string +} + +// NewCephObjectZoneInformer constructs a new informer for CephObjectZone type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewCephObjectZoneInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers) cache.SharedIndexInformer { + return NewFilteredCephObjectZoneInformer(client, namespace, resyncPeriod, indexers, nil) +} + +// NewFilteredCephObjectZoneInformer constructs a new informer for CephObjectZone type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewFilteredCephObjectZoneInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers, tweakListOptions internalinterfaces.TweakListOptionsFunc) cache.SharedIndexInformer { + return cache.NewSharedIndexInformer( + &cache.ListWatch{ + ListFunc: func(options metav1.ListOptions) (runtime.Object, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CephV1().CephObjectZones(namespace).List(options) + }, + WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CephV1().CephObjectZones(namespace).Watch(options) + }, + }, + &cephrookiov1.CephObjectZone{}, + resyncPeriod, + indexers, + ) +} + +func (f *cephObjectZoneInformer) defaultInformer(client versioned.Interface, resyncPeriod time.Duration) cache.SharedIndexInformer { + return NewFilteredCephObjectZoneInformer(client, f.namespace, resyncPeriod, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, f.tweakListOptions) +} + +func (f *cephObjectZoneInformer) Informer() cache.SharedIndexInformer { + return f.factory.InformerFor(&cephrookiov1.CephObjectZone{}, f.defaultInformer) +} + +func (f *cephObjectZoneInformer) Lister() v1.CephObjectZoneLister { + return v1.NewCephObjectZoneLister(f.Informer().GetIndexer()) +} diff --git a/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectzonegroup.go b/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectzonegroup.go new file mode 100644 index 000000000..2cfe6ff66 --- /dev/null +++ b/pkg/client/informers/externalversions/ceph.rook.io/v1/cephobjectzonegroup.go @@ -0,0 +1,89 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by informer-gen. DO NOT EDIT. + +package v1 + +import ( + time "time" + + cephrookiov1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + versioned "github.com/rook/rook/pkg/client/clientset/versioned" + internalinterfaces "github.com/rook/rook/pkg/client/informers/externalversions/internalinterfaces" + v1 "github.com/rook/rook/pkg/client/listers/ceph.rook.io/v1" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" + watch "k8s.io/apimachinery/pkg/watch" + cache "k8s.io/client-go/tools/cache" +) + +// CephObjectZoneGroupInformer provides access to a shared informer and lister for +// CephObjectZoneGroups. +type CephObjectZoneGroupInformer interface { + Informer() cache.SharedIndexInformer + Lister() v1.CephObjectZoneGroupLister +} + +type cephObjectZoneGroupInformer struct { + factory internalinterfaces.SharedInformerFactory + tweakListOptions internalinterfaces.TweakListOptionsFunc + namespace string +} + +// NewCephObjectZoneGroupInformer constructs a new informer for CephObjectZoneGroup type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewCephObjectZoneGroupInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers) cache.SharedIndexInformer { + return NewFilteredCephObjectZoneGroupInformer(client, namespace, resyncPeriod, indexers, nil) +} + +// NewFilteredCephObjectZoneGroupInformer constructs a new informer for CephObjectZoneGroup type. +// Always prefer using an informer factory to get a shared informer instead of getting an independent +// one. This reduces memory footprint and number of connections to the server. +func NewFilteredCephObjectZoneGroupInformer(client versioned.Interface, namespace string, resyncPeriod time.Duration, indexers cache.Indexers, tweakListOptions internalinterfaces.TweakListOptionsFunc) cache.SharedIndexInformer { + return cache.NewSharedIndexInformer( + &cache.ListWatch{ + ListFunc: func(options metav1.ListOptions) (runtime.Object, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CephV1().CephObjectZoneGroups(namespace).List(options) + }, + WatchFunc: func(options metav1.ListOptions) (watch.Interface, error) { + if tweakListOptions != nil { + tweakListOptions(&options) + } + return client.CephV1().CephObjectZoneGroups(namespace).Watch(options) + }, + }, + &cephrookiov1.CephObjectZoneGroup{}, + resyncPeriod, + indexers, + ) +} + +func (f *cephObjectZoneGroupInformer) defaultInformer(client versioned.Interface, resyncPeriod time.Duration) cache.SharedIndexInformer { + return NewFilteredCephObjectZoneGroupInformer(client, f.namespace, resyncPeriod, cache.Indexers{cache.NamespaceIndex: cache.MetaNamespaceIndexFunc}, f.tweakListOptions) +} + +func (f *cephObjectZoneGroupInformer) Informer() cache.SharedIndexInformer { + return f.factory.InformerFor(&cephrookiov1.CephObjectZoneGroup{}, f.defaultInformer) +} + +func (f *cephObjectZoneGroupInformer) Lister() v1.CephObjectZoneGroupLister { + return v1.NewCephObjectZoneGroupLister(f.Informer().GetIndexer()) +} diff --git a/pkg/client/informers/externalversions/ceph.rook.io/v1/interface.go b/pkg/client/informers/externalversions/ceph.rook.io/v1/interface.go index 7bcc52f42..e764bd022 100644 --- a/pkg/client/informers/externalversions/ceph.rook.io/v1/interface.go +++ b/pkg/client/informers/externalversions/ceph.rook.io/v1/interface.go @@ -34,10 +34,16 @@ type Interface interface { CephFilesystems() CephFilesystemInformer // CephNFSes returns a CephNFSInformer. CephNFSes() CephNFSInformer + // CephObjectRealms returns a CephObjectRealmInformer. + CephObjectRealms() CephObjectRealmInformer // CephObjectStores returns a CephObjectStoreInformer. CephObjectStores() CephObjectStoreInformer // CephObjectStoreUsers returns a CephObjectStoreUserInformer. CephObjectStoreUsers() CephObjectStoreUserInformer + // CephObjectZones returns a CephObjectZoneInformer. + CephObjectZones() CephObjectZoneInformer + // CephObjectZoneGroups returns a CephObjectZoneGroupInformer. + CephObjectZoneGroups() CephObjectZoneGroupInformer // CephRBDMirrors returns a CephRBDMirrorInformer. CephRBDMirrors() CephRBDMirrorInformer } @@ -78,6 +84,11 @@ func (v *version) CephNFSes() CephNFSInformer { return &cephNFSInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} } +// CephObjectRealms returns a CephObjectRealmInformer. +func (v *version) CephObjectRealms() CephObjectRealmInformer { + return &cephObjectRealmInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} +} + // CephObjectStores returns a CephObjectStoreInformer. func (v *version) CephObjectStores() CephObjectStoreInformer { return &cephObjectStoreInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} @@ -88,6 +99,16 @@ func (v *version) CephObjectStoreUsers() CephObjectStoreUserInformer { return &cephObjectStoreUserInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} } +// CephObjectZones returns a CephObjectZoneInformer. +func (v *version) CephObjectZones() CephObjectZoneInformer { + return &cephObjectZoneInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} +} + +// CephObjectZoneGroups returns a CephObjectZoneGroupInformer. +func (v *version) CephObjectZoneGroups() CephObjectZoneGroupInformer { + return &cephObjectZoneGroupInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} +} + // CephRBDMirrors returns a CephRBDMirrorInformer. func (v *version) CephRBDMirrors() CephRBDMirrorInformer { return &cephRBDMirrorInformer{factory: v.factory, namespace: v.namespace, tweakListOptions: v.tweakListOptions} diff --git a/pkg/client/informers/externalversions/generic.go b/pkg/client/informers/externalversions/generic.go index 59b25b207..b6699bb71 100644 --- a/pkg/client/informers/externalversions/generic.go +++ b/pkg/client/informers/externalversions/generic.go @@ -73,10 +73,16 @@ func (f *sharedInformerFactory) ForResource(resource schema.GroupVersionResource return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephFilesystems().Informer()}, nil case v1.SchemeGroupVersion.WithResource("cephnfses"): return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephNFSes().Informer()}, nil + case v1.SchemeGroupVersion.WithResource("cephobjectrealms"): + return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephObjectRealms().Informer()}, nil case v1.SchemeGroupVersion.WithResource("cephobjectstores"): return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephObjectStores().Informer()}, nil case v1.SchemeGroupVersion.WithResource("cephobjectstoreusers"): return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephObjectStoreUsers().Informer()}, nil + case v1.SchemeGroupVersion.WithResource("cephobjectzones"): + return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephObjectZones().Informer()}, nil + case v1.SchemeGroupVersion.WithResource("cephobjectzonegroups"): + return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephObjectZoneGroups().Informer()}, nil case v1.SchemeGroupVersion.WithResource("cephrbdmirrors"): return &genericInformer{resource: resource.GroupResource(), informer: f.Ceph().V1().CephRBDMirrors().Informer()}, nil diff --git a/pkg/client/listers/ceph.rook.io/v1/cephobjectrealm.go b/pkg/client/listers/ceph.rook.io/v1/cephobjectrealm.go new file mode 100644 index 000000000..9bd2b11b1 --- /dev/null +++ b/pkg/client/listers/ceph.rook.io/v1/cephobjectrealm.go @@ -0,0 +1,94 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by lister-gen. DO NOT EDIT. + +package v1 + +import ( + v1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/client-go/tools/cache" +) + +// CephObjectRealmLister helps list CephObjectRealms. +type CephObjectRealmLister interface { + // List lists all CephObjectRealms in the indexer. + List(selector labels.Selector) (ret []*v1.CephObjectRealm, err error) + // CephObjectRealms returns an object that can list and get CephObjectRealms. + CephObjectRealms(namespace string) CephObjectRealmNamespaceLister + CephObjectRealmListerExpansion +} + +// cephObjectRealmLister implements the CephObjectRealmLister interface. +type cephObjectRealmLister struct { + indexer cache.Indexer +} + +// NewCephObjectRealmLister returns a new CephObjectRealmLister. +func NewCephObjectRealmLister(indexer cache.Indexer) CephObjectRealmLister { + return &cephObjectRealmLister{indexer: indexer} +} + +// List lists all CephObjectRealms in the indexer. +func (s *cephObjectRealmLister) List(selector labels.Selector) (ret []*v1.CephObjectRealm, err error) { + err = cache.ListAll(s.indexer, selector, func(m interface{}) { + ret = append(ret, m.(*v1.CephObjectRealm)) + }) + return ret, err +} + +// CephObjectRealms returns an object that can list and get CephObjectRealms. +func (s *cephObjectRealmLister) CephObjectRealms(namespace string) CephObjectRealmNamespaceLister { + return cephObjectRealmNamespaceLister{indexer: s.indexer, namespace: namespace} +} + +// CephObjectRealmNamespaceLister helps list and get CephObjectRealms. +type CephObjectRealmNamespaceLister interface { + // List lists all CephObjectRealms in the indexer for a given namespace. + List(selector labels.Selector) (ret []*v1.CephObjectRealm, err error) + // Get retrieves the CephObjectRealm from the indexer for a given namespace and name. + Get(name string) (*v1.CephObjectRealm, error) + CephObjectRealmNamespaceListerExpansion +} + +// cephObjectRealmNamespaceLister implements the CephObjectRealmNamespaceLister +// interface. +type cephObjectRealmNamespaceLister struct { + indexer cache.Indexer + namespace string +} + +// List lists all CephObjectRealms in the indexer for a given namespace. +func (s cephObjectRealmNamespaceLister) List(selector labels.Selector) (ret []*v1.CephObjectRealm, err error) { + err = cache.ListAllByNamespace(s.indexer, s.namespace, selector, func(m interface{}) { + ret = append(ret, m.(*v1.CephObjectRealm)) + }) + return ret, err +} + +// Get retrieves the CephObjectRealm from the indexer for a given namespace and name. +func (s cephObjectRealmNamespaceLister) Get(name string) (*v1.CephObjectRealm, error) { + obj, exists, err := s.indexer.GetByKey(s.namespace + "/" + name) + if err != nil { + return nil, err + } + if !exists { + return nil, errors.NewNotFound(v1.Resource("cephobjectrealm"), name) + } + return obj.(*v1.CephObjectRealm), nil +} diff --git a/pkg/client/listers/ceph.rook.io/v1/cephobjectzone.go b/pkg/client/listers/ceph.rook.io/v1/cephobjectzone.go new file mode 100644 index 000000000..e9b3f56ad --- /dev/null +++ b/pkg/client/listers/ceph.rook.io/v1/cephobjectzone.go @@ -0,0 +1,94 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by lister-gen. DO NOT EDIT. + +package v1 + +import ( + v1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/client-go/tools/cache" +) + +// CephObjectZoneLister helps list CephObjectZones. +type CephObjectZoneLister interface { + // List lists all CephObjectZones in the indexer. + List(selector labels.Selector) (ret []*v1.CephObjectZone, err error) + // CephObjectZones returns an object that can list and get CephObjectZones. + CephObjectZones(namespace string) CephObjectZoneNamespaceLister + CephObjectZoneListerExpansion +} + +// cephObjectZoneLister implements the CephObjectZoneLister interface. +type cephObjectZoneLister struct { + indexer cache.Indexer +} + +// NewCephObjectZoneLister returns a new CephObjectZoneLister. +func NewCephObjectZoneLister(indexer cache.Indexer) CephObjectZoneLister { + return &cephObjectZoneLister{indexer: indexer} +} + +// List lists all CephObjectZones in the indexer. +func (s *cephObjectZoneLister) List(selector labels.Selector) (ret []*v1.CephObjectZone, err error) { + err = cache.ListAll(s.indexer, selector, func(m interface{}) { + ret = append(ret, m.(*v1.CephObjectZone)) + }) + return ret, err +} + +// CephObjectZones returns an object that can list and get CephObjectZones. +func (s *cephObjectZoneLister) CephObjectZones(namespace string) CephObjectZoneNamespaceLister { + return cephObjectZoneNamespaceLister{indexer: s.indexer, namespace: namespace} +} + +// CephObjectZoneNamespaceLister helps list and get CephObjectZones. +type CephObjectZoneNamespaceLister interface { + // List lists all CephObjectZones in the indexer for a given namespace. + List(selector labels.Selector) (ret []*v1.CephObjectZone, err error) + // Get retrieves the CephObjectZone from the indexer for a given namespace and name. + Get(name string) (*v1.CephObjectZone, error) + CephObjectZoneNamespaceListerExpansion +} + +// cephObjectZoneNamespaceLister implements the CephObjectZoneNamespaceLister +// interface. +type cephObjectZoneNamespaceLister struct { + indexer cache.Indexer + namespace string +} + +// List lists all CephObjectZones in the indexer for a given namespace. +func (s cephObjectZoneNamespaceLister) List(selector labels.Selector) (ret []*v1.CephObjectZone, err error) { + err = cache.ListAllByNamespace(s.indexer, s.namespace, selector, func(m interface{}) { + ret = append(ret, m.(*v1.CephObjectZone)) + }) + return ret, err +} + +// Get retrieves the CephObjectZone from the indexer for a given namespace and name. +func (s cephObjectZoneNamespaceLister) Get(name string) (*v1.CephObjectZone, error) { + obj, exists, err := s.indexer.GetByKey(s.namespace + "/" + name) + if err != nil { + return nil, err + } + if !exists { + return nil, errors.NewNotFound(v1.Resource("cephobjectzone"), name) + } + return obj.(*v1.CephObjectZone), nil +} diff --git a/pkg/client/listers/ceph.rook.io/v1/cephobjectzonegroup.go b/pkg/client/listers/ceph.rook.io/v1/cephobjectzonegroup.go new file mode 100644 index 000000000..d1399dcea --- /dev/null +++ b/pkg/client/listers/ceph.rook.io/v1/cephobjectzonegroup.go @@ -0,0 +1,94 @@ +/* +Copyright The Kubernetes Authors. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Code generated by lister-gen. DO NOT EDIT. + +package v1 + +import ( + v1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + "k8s.io/apimachinery/pkg/api/errors" + "k8s.io/apimachinery/pkg/labels" + "k8s.io/client-go/tools/cache" +) + +// CephObjectZoneGroupLister helps list CephObjectZoneGroups. +type CephObjectZoneGroupLister interface { + // List lists all CephObjectZoneGroups in the indexer. + List(selector labels.Selector) (ret []*v1.CephObjectZoneGroup, err error) + // CephObjectZoneGroups returns an object that can list and get CephObjectZoneGroups. + CephObjectZoneGroups(namespace string) CephObjectZoneGroupNamespaceLister + CephObjectZoneGroupListerExpansion +} + +// cephObjectZoneGroupLister implements the CephObjectZoneGroupLister interface. +type cephObjectZoneGroupLister struct { + indexer cache.Indexer +} + +// NewCephObjectZoneGroupLister returns a new CephObjectZoneGroupLister. +func NewCephObjectZoneGroupLister(indexer cache.Indexer) CephObjectZoneGroupLister { + return &cephObjectZoneGroupLister{indexer: indexer} +} + +// List lists all CephObjectZoneGroups in the indexer. +func (s *cephObjectZoneGroupLister) List(selector labels.Selector) (ret []*v1.CephObjectZoneGroup, err error) { + err = cache.ListAll(s.indexer, selector, func(m interface{}) { + ret = append(ret, m.(*v1.CephObjectZoneGroup)) + }) + return ret, err +} + +// CephObjectZoneGroups returns an object that can list and get CephObjectZoneGroups. +func (s *cephObjectZoneGroupLister) CephObjectZoneGroups(namespace string) CephObjectZoneGroupNamespaceLister { + return cephObjectZoneGroupNamespaceLister{indexer: s.indexer, namespace: namespace} +} + +// CephObjectZoneGroupNamespaceLister helps list and get CephObjectZoneGroups. +type CephObjectZoneGroupNamespaceLister interface { + // List lists all CephObjectZoneGroups in the indexer for a given namespace. + List(selector labels.Selector) (ret []*v1.CephObjectZoneGroup, err error) + // Get retrieves the CephObjectZoneGroup from the indexer for a given namespace and name. + Get(name string) (*v1.CephObjectZoneGroup, error) + CephObjectZoneGroupNamespaceListerExpansion +} + +// cephObjectZoneGroupNamespaceLister implements the CephObjectZoneGroupNamespaceLister +// interface. +type cephObjectZoneGroupNamespaceLister struct { + indexer cache.Indexer + namespace string +} + +// List lists all CephObjectZoneGroups in the indexer for a given namespace. +func (s cephObjectZoneGroupNamespaceLister) List(selector labels.Selector) (ret []*v1.CephObjectZoneGroup, err error) { + err = cache.ListAllByNamespace(s.indexer, s.namespace, selector, func(m interface{}) { + ret = append(ret, m.(*v1.CephObjectZoneGroup)) + }) + return ret, err +} + +// Get retrieves the CephObjectZoneGroup from the indexer for a given namespace and name. +func (s cephObjectZoneGroupNamespaceLister) Get(name string) (*v1.CephObjectZoneGroup, error) { + obj, exists, err := s.indexer.GetByKey(s.namespace + "/" + name) + if err != nil { + return nil, err + } + if !exists { + return nil, errors.NewNotFound(v1.Resource("cephobjectzonegroup"), name) + } + return obj.(*v1.CephObjectZoneGroup), nil +} diff --git a/pkg/client/listers/ceph.rook.io/v1/expansion_generated.go b/pkg/client/listers/ceph.rook.io/v1/expansion_generated.go index 5f9a16f59..36069d13f 100644 --- a/pkg/client/listers/ceph.rook.io/v1/expansion_generated.go +++ b/pkg/client/listers/ceph.rook.io/v1/expansion_generated.go @@ -58,6 +58,14 @@ type CephNFSListerExpansion interface{} // CephNFSNamespaceLister. type CephNFSNamespaceListerExpansion interface{} +// CephObjectRealmListerExpansion allows custom methods to be added to +// CephObjectRealmLister. +type CephObjectRealmListerExpansion interface{} + +// CephObjectRealmNamespaceListerExpansion allows custom methods to be added to +// CephObjectRealmNamespaceLister. +type CephObjectRealmNamespaceListerExpansion interface{} + // CephObjectStoreListerExpansion allows custom methods to be added to // CephObjectStoreLister. type CephObjectStoreListerExpansion interface{} @@ -74,6 +82,22 @@ type CephObjectStoreUserListerExpansion interface{} // CephObjectStoreUserNamespaceLister. type CephObjectStoreUserNamespaceListerExpansion interface{} +// CephObjectZoneListerExpansion allows custom methods to be added to +// CephObjectZoneLister. +type CephObjectZoneListerExpansion interface{} + +// CephObjectZoneNamespaceListerExpansion allows custom methods to be added to +// CephObjectZoneNamespaceLister. +type CephObjectZoneNamespaceListerExpansion interface{} + +// CephObjectZoneGroupListerExpansion allows custom methods to be added to +// CephObjectZoneGroupLister. +type CephObjectZoneGroupListerExpansion interface{} + +// CephObjectZoneGroupNamespaceListerExpansion allows custom methods to be added to +// CephObjectZoneGroupNamespaceLister. +type CephObjectZoneGroupNamespaceListerExpansion interface{} + // CephRBDMirrorListerExpansion allows custom methods to be added to // CephRBDMirrorLister. type CephRBDMirrorListerExpansion interface{} diff --git a/pkg/operator/ceph/cluster/register_controllers.go b/pkg/operator/ceph/cluster/register_controllers.go index 39766fa78..c44237754 100644 --- a/pkg/operator/ceph/cluster/register_controllers.go +++ b/pkg/operator/ceph/cluster/register_controllers.go @@ -29,7 +29,10 @@ import ( "github.com/rook/rook/pkg/operator/ceph/file" "github.com/rook/rook/pkg/operator/ceph/nfs" "github.com/rook/rook/pkg/operator/ceph/object" + "github.com/rook/rook/pkg/operator/ceph/object/realm" objectuser "github.com/rook/rook/pkg/operator/ceph/object/user" + "github.com/rook/rook/pkg/operator/ceph/object/zone" + "github.com/rook/rook/pkg/operator/ceph/object/zonegroup" "github.com/rook/rook/pkg/operator/ceph/pool" "sigs.k8s.io/controller-runtime/pkg/manager" @@ -57,6 +60,9 @@ var AddToManagerFuncs = []func(manager.Manager, *clusterd.Context) error{ crash.Add, pool.Add, objectuser.Add, + realm.Add, + zonegroup.Add, + zone.Add, object.Add, file.Add, nfs.Add, diff --git a/pkg/operator/ceph/controller/predicate.go b/pkg/operator/ceph/controller/predicate.go index 2d1c80bb7..3c89e3563 100644 --- a/pkg/operator/ceph/controller/predicate.go +++ b/pkg/operator/ceph/controller/predicate.go @@ -106,6 +106,63 @@ func WatchControllerPredicate() predicate.Funcs { logger.Debugf("skipping resource %q update with unchanged spec", objNew.Name) } + case *cephv1.CephObjectRealm: + objNew := e.ObjectNew.(*cephv1.CephObjectRealm) + logger.Debug("update event on CephObjectRealm") + // If the labels "do_not_reconcile" is set on the object, let's not reconcile that request + isDoNotReconcile := isDoNotReconcile(objNew.GetLabels()) + if isDoNotReconcile { + logger.Debugf("object %q matched on update but %q label is set, doing nothing", doNotReconcileLabelName, objNew.Name) + return false + } + diff := cmp.Diff(objOld.Spec, objNew.Spec, resourceQtyComparer) + if diff != "" { + logger.Infof("CR has changed for %q. diff=%s", objNew.Name, diff) + } else if objOld.GetDeletionTimestamp() != objNew.GetDeletionTimestamp() { + logger.Debugf("CR %q is going be deleted", objNew.Name) + return true + } else if objOld.GetGeneration() != objNew.GetGeneration() { + logger.Debugf("skipping resource %q update with unchanged spec", objNew.Name) + } + + case *cephv1.CephObjectZoneGroup: + objNew := e.ObjectNew.(*cephv1.CephObjectZoneGroup) + logger.Debug("update event on CephObjectZoneGroup") + // If the labels "do_not_reconcile" is set on the object, let's not reconcile that request + isDoNotReconcile := isDoNotReconcile(objNew.GetLabels()) + if isDoNotReconcile { + logger.Debugf("object %q matched on update but %q label is set, doing nothing", doNotReconcileLabelName, objNew.Name) + return false + } + diff := cmp.Diff(objOld.Spec, objNew.Spec, resourceQtyComparer) + if diff != "" { + logger.Infof("CR has changed for %q. diff=%s", objNew.Name, diff) + } else if objOld.GetDeletionTimestamp() != objNew.GetDeletionTimestamp() { + logger.Debugf("CR %q is going be deleted", objNew.Name) + return true + } else if objOld.GetGeneration() != objNew.GetGeneration() { + logger.Debugf("skipping resource %q update with unchanged spec", objNew.Name) + } + + case *cephv1.CephObjectZone: + objNew := e.ObjectNew.(*cephv1.CephObjectZone) + logger.Debug("update event on CephObjectZone") + // If the labels "do_not_reconcile" is set on the object, let's not reconcile that request + isDoNotReconcile := isDoNotReconcile(objNew.GetLabels()) + if isDoNotReconcile { + logger.Debugf("object %q matched on update but %q label is set, doing nothing", doNotReconcileLabelName, objNew.Name) + return false + } + diff := cmp.Diff(objOld.Spec, objNew.Spec, resourceQtyComparer) + if diff != "" { + logger.Infof("CR has changed for %q. diff=%s", objNew.Name, diff) + } else if objOld.GetDeletionTimestamp() != objNew.GetDeletionTimestamp() { + logger.Debugf("CR %q is going be deleted", objNew.Name) + return true + } else if objOld.GetGeneration() != objNew.GetGeneration() { + logger.Debugf("skipping resource %q update with unchanged spec", objNew.Name) + } + case *cephv1.CephBlockPool: objNew := e.ObjectNew.(*cephv1.CephBlockPool) logger.Debug("update event on CephBlockPool CR") diff --git a/pkg/operator/ceph/object/admin.go b/pkg/operator/ceph/object/admin.go index d7c0b2f81..c9e61e90b 100644 --- a/pkg/operator/ceph/object/admin.go +++ b/pkg/operator/ceph/object/admin.go @@ -19,7 +19,6 @@ package object import ( "fmt" - "github.com/pkg/errors" "github.com/rook/rook/pkg/clusterd" "github.com/rook/rook/pkg/daemon/ceph/client" ) @@ -37,13 +36,13 @@ func NewContext(context *clusterd.Context, name, clusterName string) *Context { return &Context{Context: context, Name: name, ClusterName: clusterName, RunAsUser: client.AdminUsername} } -func runAdminCommandNoRealm(c *Context, args ...string) (string, error) { +func RunAdminCommandNoRealm(c *Context, args ...string) (string, error) { command, args := client.FinalizeCephCommandArgs("radosgw-admin", args, c.Context.ConfigDir, c.ClusterName, c.RunAsUser) // start the rgw admin command output, err := c.Context.Executor.ExecuteCommandWithOutput(command, args...) if err != nil { - return output, errors.Wrap(err, "failed to run radosgw-admin") + return "", err } return output, nil @@ -59,8 +58,8 @@ func runAdminCommand(c *Context, args ...string) (string, error) { fmt.Sprintf("--rgw-zonegroup=%s", c.Name), } - return runAdminCommandNoRealm(c, append(args, options...)...) + return RunAdminCommandNoRealm(c, append(args, options...)...) } - return runAdminCommandNoRealm(c, args...) + return RunAdminCommandNoRealm(c, args...) } diff --git a/pkg/operator/ceph/object/controller.go b/pkg/operator/ceph/object/controller.go index acacb7227..9e0748cc1 100644 --- a/pkg/operator/ceph/object/controller.go +++ b/pkg/operator/ceph/object/controller.go @@ -21,6 +21,8 @@ import ( "fmt" "reflect" "strings" + "syscall" + "time" "github.com/coreos/pkg/capnslog" bktclient "github.com/kube-object-storage/lib-bucket-provisioner/pkg/client/clientset/versioned" @@ -33,6 +35,7 @@ import ( opconfig "github.com/rook/rook/pkg/operator/ceph/config" opcontroller "github.com/rook/rook/pkg/operator/ceph/controller" "github.com/rook/rook/pkg/operator/k8sutil" + "github.com/rook/rook/pkg/util/exec" appsv1 "k8s.io/api/apps/v1" corev1 "k8s.io/api/core/v1" v1 "k8s.io/api/core/v1" @@ -52,6 +55,8 @@ const ( controllerName = "ceph-object-controller" ) +var waitForRequeueIfObjectStoreNotReady = reconcile.Result{Requeue: true, RequeueAfter: 10 * time.Second} + var logger = capnslog.NewPackageLogger("github.com/rook/rook", controllerName) // List of object resources to watch by the controller @@ -273,6 +278,20 @@ func (r *ReconcileCephObjectStore) reconcileCreateObjectStore(cephObjectStore *c skipUpgradeChecks: r.cephClusterSpec.SkipUpgradeChecks, } + // Reconcile realm/zonegroup/zone CRs & update their names + realmName, zoneGroupName, zoneName, reconcileResponse, err := r.reconcileMultisiteCRs(cephObjectStore) + if err != nil { + return reconcileResponse, err + } + + // Reconcile Ceph Zone if Multisite + if cephObjectStore.Spec.IsMultisite() { + reconcileResponse, err := r.reconcileCephZone(cephObjectStore, zoneGroupName, realmName) + if err != nil { + return reconcileResponse, err + } + } + // RECONCILE SERVICE logger.Debug("reconciling object store service") serviceIP, err := cfg.reconcileService(cephObjectStore) @@ -290,13 +309,13 @@ func (r *ReconcileCephObjectStore) reconcileCreateObjectStore(cephObjectStore *c } // RECONCILE REALM - logger.Info("reconciling object store realms") - err = reconcileRealm(objContext, serviceIP, cephObjectStore.Spec.Gateway.Port) + logger.Infof("setting multisite settings for object store %q", cephObjectStore.Name) + err = setMultisite(objContext, serviceIP, cephObjectStore.Spec, realmName, zoneGroupName, zoneName) if err != nil { - return r.setFailedStatus(name, "failed to create object store realm", err) + return r.setFailedStatus(name, "failed to configure multisite for object store", err) } - err = cfg.createOrUpdateStore() + err = cfg.createOrUpdateStore(realmName, zoneGroupName, zoneName) if err != nil { return reconcile.Result{}, errors.Wrapf(err, "failed to create object store %q", cephObjectStore.Name) } @@ -304,6 +323,61 @@ func (r *ReconcileCephObjectStore) reconcileCreateObjectStore(cephObjectStore *c return reconcile.Result{}, nil } +func (r *ReconcileCephObjectStore) reconcileCephZone(store *cephv1.CephObjectStore, zoneGroupName string, realmName string) (reconcile.Result, error) { + realmArg := fmt.Sprintf("--rgw-realm=%s", realmName) + zoneGroupArg := fmt.Sprintf("--rgw-zonegroup=%s", zoneGroupName) + zoneArg := fmt.Sprintf("--rgw-zone=%s", store.Spec.Zone.Name) + objContext := NewContext(r.context, store.Name, store.Namespace) + + _, err := RunAdminCommandNoRealm(objContext, "zone", "get", realmArg, zoneGroupArg, zoneArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + return waitForRequeueIfObjectStoreNotReady, errors.Wrapf(err, "ceph zone %q not found", store.Spec.Zone.Name) + } else { + return waitForRequeueIfObjectStoreNotReady, errors.Wrapf(err, "radosgw-admin zone get failed with code %d", code) + } + } + + logger.Infof("Zone %q found in Ceph cluster will include object store %q", store.Spec.Zone.Name, store.Name) + return reconcile.Result{}, nil +} + +func (r *ReconcileCephObjectStore) reconcileMultisiteCRs(cephObjectStore *cephv1.CephObjectStore) (string, string, string, reconcile.Result, error) { + if cephObjectStore.Spec.IsMultisite() { + zoneName := cephObjectStore.Spec.Zone.Name + zone, err := r.context.RookClientset.CephV1().CephObjectZones(cephObjectStore.Namespace).Get(zoneName, metav1.GetOptions{}) + if err != nil { + if kerrors.IsNotFound(err) { + return "", "", "", waitForRequeueIfObjectStoreNotReady, err + } + return "", "", "", waitForRequeueIfObjectStoreNotReady, errors.Wrapf(err, "error getting CephObjectZone %q", cephObjectStore.Spec.Zone.Name) + } + logger.Infof("CephObjectZone resource %s found", zone.Name) + + zonegroup, err := r.context.RookClientset.CephV1().CephObjectZoneGroups(cephObjectStore.Namespace).Get(zone.Spec.ZoneGroup, metav1.GetOptions{}) + if err != nil { + if kerrors.IsNotFound(err) { + return "", "", "", waitForRequeueIfObjectStoreNotReady, err + } + return "", "", "", waitForRequeueIfObjectStoreNotReady, errors.Wrapf(err, "error getting CephObjectZoneGroup %q", zone.Spec.ZoneGroup) + } + logger.Infof("CephObjectZoneGroup resource %s found", zonegroup.Name) + + realm, err := r.context.RookClientset.CephV1().CephObjectRealms(cephObjectStore.Namespace).Get(zonegroup.Spec.Realm, metav1.GetOptions{}) + if err != nil { + if kerrors.IsNotFound(err) { + return "", "", "", waitForRequeueIfObjectStoreNotReady, err + } + return "", "", "", waitForRequeueIfObjectStoreNotReady, errors.Wrapf(err, "error getting CephObjectRealm %q", zonegroup.Spec.Realm) + } + logger.Infof("CephObjectRealm resource %s found", realm.Name) + + return realm.Name, zonegroup.Name, zone.Name, reconcile.Result{}, nil + } + + return cephObjectStore.Name, cephObjectStore.Name, cephObjectStore.Name, reconcile.Result{}, nil +} + func (r *ReconcileCephObjectStore) setFailedStatus(name types.NamespacedName, errMessage string, err error) (reconcile.Result, error) { updateStatus(r.client, name, k8sutil.ReconcileFailedStatus) return reconcile.Result{}, errors.Wrapf(err, "%s", errMessage) diff --git a/pkg/operator/ceph/object/objectstore.go b/pkg/operator/ceph/object/objectstore.go index 8725c4918..4250362ce 100644 --- a/pkg/operator/ceph/object/objectstore.go +++ b/pkg/operator/ceph/object/objectstore.go @@ -88,74 +88,81 @@ func deleteRealmAndPools(context *Context, spec cephv1.ObjectStoreSpec) error { return nil } -func reconcileRealm(context *Context, serviceIP string, port int32) error { - zoneArg := fmt.Sprintf("--rgw-zone=%s", context.Name) - endpointArg := fmt.Sprintf("--endpoints=%s:%d", serviceIP, port) +func setMultisite(context *Context, serviceIP string, spec cephv1.ObjectStoreSpec, realmName string, zoneGroupName string, zoneName string) error { + realmArg := fmt.Sprintf("--rgw-realm=%s", realmName) + zoneGroupArg := fmt.Sprintf("--rgw-zonegroup=%s", zoneGroupName) + zoneArg := fmt.Sprintf("--rgw-zone=%s", zoneName) + endpointArg := fmt.Sprintf("--endpoints=%s:%d", serviceIP, spec.Gateway.Port) updatePeriod := false - // The first realm must be marked as the default - defaultArg := "" - stores, err := getObjectStores(context) - if err != nil { - return errors.Wrap(err, "failed to get object stores") - } - if len(stores) == 0 { - defaultArg = "--default" - } - - // create the realm if it doesn't exist yet - output, err := runAdminCommand(context, "realm", "get") - if err != nil { + if spec.IsMultisite() { updatePeriod = true - output, err = runAdminCommand(context, "realm", "create", defaultArg) + _, err := RunAdminCommandNoRealm(context, "zonegroup", "modify", realmArg, zoneGroupArg, endpointArg) if err != nil { - return errors.Wrapf(err, "failed to create rgw realm %s", context.Name) + return errors.Wrapf(err, "failed to add object store %q in rgw zone group %q", context.Name, zoneGroupName) } - } - - realmID, err := decodeID(output) - if err != nil { - return errors.Wrap(err, "failed to parse realm id") - } - - // create the zonegroup if it doesn't exist yet - output, err = runAdminCommand(context, "zonegroup", "get") - if err != nil { - updatePeriod = true - output, err = runAdminCommand(context, "zonegroup", "create", "--master", endpointArg, defaultArg) + _, err = RunAdminCommandNoRealm(context, "zone", "modify", realmArg, zoneGroupArg, zoneArg, endpointArg) if err != nil { - return errors.Wrapf(err, "failed to create rgw zonegroup for %s", context.Name) + return errors.Wrapf(err, "failed to add object store %q in rgw zone %q", context.Name, zoneName) } - } - - zoneGroupID, err := decodeID(output) - if err != nil { - return errors.Wrap(err, "failed to parse zone group id") - } - - // create the zone if it doesn't exist yet - output, err = runAdminCommand(context, "zone", "get", zoneArg) - if err != nil { - updatePeriod = true - output, err = runAdminCommand(context, "zone", "create", "--master", endpointArg, zoneArg, defaultArg) + logger.Infof("Added object store %q to realm %q, zonegroup %q, zone %q", context.Name, realmName, zoneGroupName, zoneName) + } else { + // create the realm if it doesn't exist yet + output, err := runAdminCommand(context, "realm", "get") if err != nil { - return errors.Wrapf(err, "failed to create rgw zonegroup for %s", context.Name) + updatePeriod = true + output, err = runAdminCommand(context, "realm", "create") + if err != nil { + return errors.Wrapf(err, "failed to create rgw realm %q", context.Name) + } } - } - zoneID, err := decodeID(output) - if err != nil { - return errors.Wrap(err, "failed to parse zone id") + + realmID, err := decodeID(output) + if err != nil { + return errors.Wrap(err, "failed to parse realm id") + } + + // create the zonegroup if it doesn't exist yet + output, err = runAdminCommand(context, "zonegroup", "get") + if err != nil { + updatePeriod = true + output, err = runAdminCommand(context, "zonegroup", "create", "--master", endpointArg) + if err != nil { + return errors.Wrapf(err, "failed to create rgw zonegroup for %q", context.Name) + } + } + + zoneGroupID, err := decodeID(output) + if err != nil { + return errors.Wrap(err, "failed to parse zone group id") + } + + // create the zone if it doesn't exist yet + output, err = runAdminCommand(context, "zone", "get", zoneArg) + if err != nil { + updatePeriod = true + output, err = runAdminCommand(context, "zone", "create", "--master", endpointArg, zoneArg) + if err != nil { + return errors.Wrapf(err, "failed to create rgw zonegroup for %q", context.Name) + } + } + + zoneID, err := decodeID(output) + if err != nil { + return errors.Wrap(err, "failed to parse zone id") + } + + logger.Infof("RGW: realm=%s, zonegroup=%s, zone=%s", realmID, zoneGroupID, zoneID) } if updatePeriod { // the period will help notify other zones of changes if there are multi-zones - _, err := runAdminCommandNoRealm(context, "period", "update", "--commit") + _, err := RunAdminCommandNoRealm(context, "period", "update", "--commit", realmArg, zoneGroupArg, zoneArg) if err != nil { return errors.Wrap(err, "failed to update period") } } - logger.Infof("RGW: realm=%s, zonegroup=%s, zone=%s", realmID, zoneGroupID, zoneID) return nil } @@ -190,7 +197,7 @@ func decodeID(data string) (string, error) { } func getObjectStores(context *Context) ([]string, error) { - output, err := runAdminCommandNoRealm(context, "realm", "list") + output, err := RunAdminCommandNoRealm(context, "realm", "list") if err != nil { if strings.Index(err.Error(), "exit status 2") != 0 { return []string{}, nil diff --git a/pkg/operator/ceph/object/objectstore_test.go b/pkg/operator/ceph/object/objectstore_test.go index 9bf7969ee..2977f27e3 100644 --- a/pkg/operator/ceph/object/objectstore_test.go +++ b/pkg/operator/ceph/object/objectstore_test.go @@ -33,26 +33,9 @@ import ( ) func TestReconcileRealm(t *testing.T) { - defaultStore := true executorFunc := func(command string, args ...string) (string, error) { idResponse := `{"id":"test-id"}` logger.Infof("Execute: %s %v", command, args) - if args[1] == "get" { - return "", errors.New("induce a create") - } else if args[1] == "create" { - for _, arg := range args { - if arg == "--default" { - assert.True(t, defaultStore, "did not expect to find --default in %v", args) - return idResponse, nil - } - } - assert.False(t, defaultStore, "did not find --default flag in %v", args) - } else if args[0] == "realm" && args[1] == "list" { - if defaultStore { - return "", errors.New("failed to run radosgw-admin: Failed to complete : exit status 2") - } - return `{"realms": ["myobj"]}`, nil - } return idResponse, nil } executor := &exectest.MockExecutor{ @@ -64,12 +47,12 @@ func TestReconcileRealm(t *testing.T) { context := &clusterd.Context{Executor: executor} objContext := NewContext(context, storeName, "mycluster") // create the first realm, marked as default - err := reconcileRealm(objContext, "1.2.3.4", 80) + spec := cephv1.ObjectStoreSpec{} + err := setMultisite(objContext, "1.2.3.4", spec, storeName, storeName, storeName) assert.Nil(t, err) // create the second realm, not marked as default - defaultStore = false - err = reconcileRealm(objContext, "2.3.4.5", 80) + err = setMultisite(objContext, "2.3.4.5", spec, storeName, storeName, storeName) assert.Nil(t, err) } diff --git a/pkg/operator/ceph/object/realm/controller.go b/pkg/operator/ceph/object/realm/controller.go new file mode 100644 index 000000000..b2008b352 --- /dev/null +++ b/pkg/operator/ceph/object/realm/controller.go @@ -0,0 +1,238 @@ +/* +Copyright 2020 The Rook Authors. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package objectrealm to manage a rook object realm. +package realm + +import ( + "context" + "fmt" + "reflect" + "syscall" + + opcontroller "github.com/rook/rook/pkg/operator/ceph/controller" + "k8s.io/apimachinery/pkg/runtime" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/manager" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" + + "github.com/coreos/pkg/capnslog" + "github.com/pkg/errors" + cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + "github.com/rook/rook/pkg/clusterd" + "github.com/rook/rook/pkg/operator/ceph/object" + "github.com/rook/rook/pkg/operator/k8sutil" + "github.com/rook/rook/pkg/util/exec" + kerrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +const ( + controllerName = "ceph-object-realm-controller" +) + +var logger = capnslog.NewPackageLogger("github.com/rook/rook", controllerName) + +var cephObjectRealmKind = reflect.TypeOf(cephv1.CephObjectRealm{}).Name() + +// Sets the type meta for the controller main object +var controllerTypeMeta = metav1.TypeMeta{ + Kind: cephObjectRealmKind, + APIVersion: fmt.Sprintf("%s/%s", cephv1.CustomResourceGroup, cephv1.Version), +} + +// ReconcileObjectRealm reconciles a ObjectRealm object +type ReconcileObjectRealm struct { + client client.Client + scheme *runtime.Scheme + context *clusterd.Context +} + +// Add creates a new CephObjectRealm Controller and adds it to the Manager. The Manager will set fields on the Controller +// and Start it when the Manager is Started. +func Add(mgr manager.Manager, context *clusterd.Context) error { + return add(mgr, newReconciler(mgr, context)) +} + +// newReconciler returns a new reconcile.Reconciler +func newReconciler(mgr manager.Manager, context *clusterd.Context) reconcile.Reconciler { + // Add the cephv1 scheme to the manager scheme so that the controller knows about it + mgrScheme := mgr.GetScheme() + cephv1.AddToScheme(mgr.GetScheme()) + + return &ReconcileObjectRealm{ + client: mgr.GetClient(), + scheme: mgrScheme, + context: context, + } +} + +func add(mgr manager.Manager, r reconcile.Reconciler) error { + // Create a new controller + c, err := controller.New(controllerName, mgr, controller.Options{Reconciler: r}) + if err != nil { + return err + } + logger.Info("successfully started") + + // Watch for changes on the CephObjectRealm CRD object + err = c.Watch(&source.Kind{Type: &cephv1.CephObjectRealm{TypeMeta: controllerTypeMeta}}, &handler.EnqueueRequestForObject{}, opcontroller.WatchControllerPredicate()) + if err != nil { + return err + } + + return nil +} + +// Reconcile reads that state of the cluster for a CephObjectRealm object and makes changes based on the state read +// and what is in the CephObjectRealm.Spec +// The Controller will requeue the Request to be processed again if the returned error is non-nil or +// Result.Requeue is true, otherwise upon completion it will remove the work from the queue. +func (r *ReconcileObjectRealm) Reconcile(request reconcile.Request) (reconcile.Result, error) { + // workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface + reconcileResponse, err := r.reconcile(request) + if err != nil { + logger.Errorf("failed to reconcile: %v", err) + } + + return reconcileResponse, err +} + +func (r *ReconcileObjectRealm) reconcile(request reconcile.Request) (reconcile.Result, error) { + // Fetch the CephObjectRealm instance + cephObjectRealm := &cephv1.CephObjectRealm{} + err := r.client.Get(context.TODO(), request.NamespacedName, cephObjectRealm) + if err != nil { + if kerrors.IsNotFound(err) { + logger.Debug("CephObjectRealm resource not found. Ignoring since object must be deleted.") + return reconcile.Result{}, nil + } + // Error reading the object - requeue the request. + return reconcile.Result{}, errors.Wrap(err, "failed to get CephObjectRealm") + } + + // The CR was just created, initializing status fields + if cephObjectRealm.Status == nil { + updateStatus(r.client, request.NamespacedName, k8sutil.Created) + } + + // Make sure a CephCluster is present otherwise do nothing + _, isReadyToReconcile, cephClusterExists, reconcileResponse := opcontroller.IsReadyToReconcile(r.client, r.context, request.NamespacedName, controllerName) + if !isReadyToReconcile { + // This handles the case where the Ceph Cluster is gone and we want to delete that CR + if !cephObjectRealm.GetDeletionTimestamp().IsZero() && !cephClusterExists { + // Return and do not requeue. Successful deletion. + return reconcile.Result{}, nil + } + return reconcileResponse, nil + } + + // DELETE: the CR was deleted + if !cephObjectRealm.GetDeletionTimestamp().IsZero() { + logger.Debugf("deleting realm CR %q", cephObjectRealm.Name) + + // Return and do not requeue. Successful deletion. + return reconcile.Result{}, nil + } + + // validate the realm settings + err = validateRealmCR(cephObjectRealm) + if err != nil { + updateStatus(r.client, request.NamespacedName, k8sutil.ReconcileFailedStatus) + return reconcile.Result{}, errors.Wrapf(err, "invalid CephObjectRealm CR %q", cephObjectRealm.Name) + } + + // Start object reconciliation, updating status for this + updateStatus(r.client, request.NamespacedName, k8sutil.ReconcilingStatus) + + // CREATE/UPDATE CEPH REALM + reconcileResponse, err = r.createCephRealm(cephObjectRealm) + if err != nil { + return r.setFailedStatus(request.NamespacedName, "failed to create ceph realm", err) + } + + // Set Ready status, we are done reconciling + updateStatus(r.client, request.NamespacedName, k8sutil.ReadyStatus) + + // Return and do not requeue + logger.Debug("realm done reconciling") + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectRealm) createCephRealm(realm *cephv1.CephObjectRealm) (reconcile.Result, error) { + realmArg := fmt.Sprintf("--rgw-realm=%s", realm.Name) + objContext := object.NewContext(r.context, realm.Name, realm.Namespace) + + _, err := object.RunAdminCommandNoRealm(objContext, "realm", "get", realmArg) + + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + logger.Debugf("ceph realm %q not found, running `radosgw-admin realm create`", realm.Name) + _, err := object.RunAdminCommandNoRealm(objContext, "realm", "create", realmArg) + if err != nil { + return reconcile.Result{}, errors.Wrapf(err, "failed to create ceph realm %s", realm.Name) + } + logger.Debugf("created ceph realm %q", realm.Name) + } else { + return reconcile.Result{}, errors.Wrapf(err, "radosgw-admin realm get failed with code %d", code) + } + } + + return reconcile.Result{}, nil +} + +// validateRealmCR validates the realm arguments +func validateRealmCR(u *cephv1.CephObjectRealm) error { + if u.Name == "" { + return errors.New("missing name") + } + if u.Namespace == "" { + return errors.New("missing namespace") + } + return nil +} + +func (r *ReconcileObjectRealm) setFailedStatus(name types.NamespacedName, errMessage string, err error) (reconcile.Result, error) { + updateStatus(r.client, name, k8sutil.ReconcileFailedStatus) + return reconcile.Result{}, errors.Wrapf(err, "%s", errMessage) +} + +// updateStatus updates an realm with a given status +func updateStatus(client client.Client, name types.NamespacedName, status string) { + objectRealm := &cephv1.CephObjectRealm{} + if err := client.Get(context.TODO(), name, objectRealm); err != nil { + if kerrors.IsNotFound(err) { + logger.Debug("CephObjectRealm resource not found. Ignoring since object must be deleted.") + return + } + logger.Warningf("failed to retrieve object realm %q to update status to %q. %v", name, status, err) + return + } + if objectRealm.Status == nil { + objectRealm.Status = &cephv1.Status{} + } + + objectRealm.Status.Phase = status + if err := opcontroller.UpdateStatus(client, objectRealm); err != nil { + logger.Errorf("failed to set object realm %q status to %q. %v", name, status, err) + return + } + logger.Debugf("object realm %q status updated to %q", name, status) +} diff --git a/pkg/operator/ceph/object/rgw.go b/pkg/operator/ceph/object/rgw.go index 42851bc78..8d613676b 100644 --- a/pkg/operator/ceph/object/rgw.go +++ b/pkg/operator/ceph/object/rgw.go @@ -56,6 +56,9 @@ type clusterConfig struct { type rgwConfig struct { ResourceName string DaemonID string + Realm string + ZoneGroup string + Zone string } const ( @@ -64,10 +67,10 @@ const ( var updateDeploymentAndWait = mon.UpdateCephDeploymentAndWait -func (c *clusterConfig) createOrUpdateStore() error { +func (c *clusterConfig) createOrUpdateStore(realmName string, zoneGroupName string, zoneName string) error { logger.Infof("creating object store %q in namespace %q", c.store.Name, c.store.Namespace) - if err := c.startRGWPods(); err != nil { + if err := c.startRGWPods(realmName, zoneGroupName, zoneName); err != nil { return errors.Wrap(err, "failed to start rgw pods") } @@ -75,7 +78,7 @@ func (c *clusterConfig) createOrUpdateStore() error { return nil } -func (c *clusterConfig) startRGWPods() error { +func (c *clusterConfig) startRGWPods(realmName string, zoneGroupName string, zoneName string) error { // backward compatibility, triggered during updates if c.store.Spec.Gateway.AllNodes { // log we don't support that anymore @@ -111,6 +114,9 @@ func (c *clusterConfig) startRGWPods() error { rgwConfig := &rgwConfig{ ResourceName: resourceName, DaemonID: daemonName, + Realm: realmName, + ZoneGroup: zoneGroupName, + Zone: zoneName, } // We set the owner reference of the Secret to the Object controller instead of the replicaset diff --git a/pkg/operator/ceph/object/rgw_test.go b/pkg/operator/ceph/object/rgw_test.go index e664d5653..bed5a6d3f 100644 --- a/pkg/operator/ceph/object/rgw_test.go +++ b/pkg/operator/ceph/object/rgw_test.go @@ -62,7 +62,7 @@ func TestStartRGW(t *testing.T) { // start a basic cluster c := &clusterConfig{info, context, store, version, &cephv1.ClusterSpec{}, &metav1.OwnerReference{}, data, false, r.client, s, cephv1.NetworkSpec{}} - err := c.startRGWPods() + err := c.startRGWPods(store.Name, store.Name, store.Name) assert.Nil(t, err) validateStart(t, c, clientset) @@ -108,7 +108,7 @@ func TestCreateObjectStore(t *testing.T) { cl := fake.NewFakeClientWithScheme(s, object...) r := &ReconcileCephObjectStore{client: cl, scheme: s} c := &clusterConfig{info, context, store, "1.2.3.4", &cephv1.ClusterSpec{}, &metav1.OwnerReference{}, data, false, r.client, s, cephv1.NetworkSpec{}} - err := c.createOrUpdateStore() + err := c.createOrUpdateStore(store.Name, store.Name, store.Name) assert.Nil(t, err) } diff --git a/pkg/operator/ceph/object/spec.go b/pkg/operator/ceph/object/spec.go index f58960ed4..c67455b20 100644 --- a/pkg/operator/ceph/object/spec.go +++ b/pkg/operator/ceph/object/spec.go @@ -148,6 +148,9 @@ func (c *clusterConfig) makeDaemonContainer(rgwConfig *rgwConfig) v1.Container { cephconfig.NewFlag("rgw frontends", fmt.Sprintf("%s %s", rgwFrontendName, c.portString())), cephconfig.NewFlag("host", controller.ContainerEnvVarReference(k8sutil.PodNameEnvVar)), cephconfig.NewFlag("rgw-mime-types-file", mimeTypesMountPath()), + cephconfig.NewFlag("rgw realm", rgwConfig.Realm), + cephconfig.NewFlag("rgw zonegroup", rgwConfig.ZoneGroup), + cephconfig.NewFlag("rgw zone", rgwConfig.Zone), ), ), VolumeMounts: append( diff --git a/pkg/operator/ceph/object/zone/controller.go b/pkg/operator/ceph/object/zone/controller.go new file mode 100644 index 000000000..50e29cfaa --- /dev/null +++ b/pkg/operator/ceph/object/zone/controller.go @@ -0,0 +1,300 @@ +/* +Copyright 2020 The Rook Authors. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package objectzone to manage a rook object zone. +package zone + +import ( + "context" + "fmt" + "reflect" + "syscall" + "time" + + opcontroller "github.com/rook/rook/pkg/operator/ceph/controller" + "k8s.io/apimachinery/pkg/runtime" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/manager" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" + + "github.com/coreos/pkg/capnslog" + "github.com/pkg/errors" + cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + "github.com/rook/rook/pkg/clusterd" + "github.com/rook/rook/pkg/operator/ceph/object" + "github.com/rook/rook/pkg/operator/k8sutil" + "github.com/rook/rook/pkg/util/exec" + kerrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +const ( + controllerName = "ceph-object-zone-controller" +) + +var waitForRequeueIfObjectZoneNotReady = reconcile.Result{Requeue: true, RequeueAfter: 10 * time.Second} + +var logger = capnslog.NewPackageLogger("github.com/rook/rook", controllerName) + +var cephObjectZoneKind = reflect.TypeOf(cephv1.CephObjectZone{}).Name() + +// Sets the type meta for the controller main object +var controllerTypeMeta = metav1.TypeMeta{ + Kind: cephObjectZoneKind, + APIVersion: fmt.Sprintf("%s/%s", cephv1.CustomResourceGroup, cephv1.Version), +} + +// ReconcileObjectZone reconciles a ObjectZone object +type ReconcileObjectZone struct { + client client.Client + scheme *runtime.Scheme + context *clusterd.Context +} + +// Add creates a new CephObjectZone Controller and adds it to the Manager. The Manager will set fields on the Controller +// and Start it when the Manager is Started. +func Add(mgr manager.Manager, context *clusterd.Context) error { + return add(mgr, newReconciler(mgr, context)) +} + +// newReconciler returns a new reconcile.Reconciler +func newReconciler(mgr manager.Manager, context *clusterd.Context) reconcile.Reconciler { + // Add the cephv1 scheme to the manager scheme so that the controller knows about it + mgrScheme := mgr.GetScheme() + cephv1.AddToScheme(mgr.GetScheme()) + + return &ReconcileObjectZone{ + client: mgr.GetClient(), + scheme: mgrScheme, + context: context, + } +} + +func add(mgr manager.Manager, r reconcile.Reconciler) error { + // Create a new controller + c, err := controller.New(controllerName, mgr, controller.Options{Reconciler: r}) + if err != nil { + return err + } + logger.Info("successfully started") + + // Watch for changes on the CephObjectZone CRD object + err = c.Watch(&source.Kind{Type: &cephv1.CephObjectZone{TypeMeta: controllerTypeMeta}}, &handler.EnqueueRequestForObject{}, opcontroller.WatchControllerPredicate()) + if err != nil { + return err + } + + return nil +} + +// Reconcile reads that state of the cluster for a CephObjectZone object and makes changes based on the state read +// and what is in the CephObjectZone.Spec +// The Controller will requeue the Request to be processed again if the returned error is non-nil or +// Result.Requeue is true, otherwise upon completion it will remove the work from the queue. +func (r *ReconcileObjectZone) Reconcile(request reconcile.Request) (reconcile.Result, error) { + // workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface + reconcileResponse, err := r.reconcile(request) + if err != nil { + logger.Errorf("failed to reconcile: %v", err) + } + + return reconcileResponse, err +} + +func (r *ReconcileObjectZone) reconcile(request reconcile.Request) (reconcile.Result, error) { + // Fetch the CephObjectZone instance + cephObjectZone := &cephv1.CephObjectZone{} + err := r.client.Get(context.TODO(), request.NamespacedName, cephObjectZone) + if err != nil { + if kerrors.IsNotFound(err) { + logger.Debug("CephObjectZone resource not found. Ignoring since object must be deleted.") + return reconcile.Result{}, nil + } + // Error reading the object - requeue the request. + return reconcile.Result{}, errors.Wrap(err, "failed to get CephObjectZone") + } + + // The CR was just created, initializing status fields + if cephObjectZone.Status == nil { + updateStatus(r.client, request.NamespacedName, k8sutil.Created) + } + + // Make sure a CephCluster is present otherwise do nothing + _, isReadyToReconcile, cephClusterExists, reconcileResponse := opcontroller.IsReadyToReconcile(r.client, r.context, request.NamespacedName, controllerName) + if !isReadyToReconcile { + // This handles the case where the Ceph Cluster is gone and we want to delete that CR + // + if !cephObjectZone.GetDeletionTimestamp().IsZero() && !cephClusterExists { + // Return and do not requeue. Successful deletion. + return reconcile.Result{}, nil + } + return reconcileResponse, nil + } + + // DELETE: the CR was deleted + if !cephObjectZone.GetDeletionTimestamp().IsZero() { + logger.Debugf("deleting zone CR %q", cephObjectZone.Name) + + // Return and do not requeue. Successful deletion. + return reconcile.Result{}, nil + } + + // validate the zone settings + err = validateZoneCR(cephObjectZone) + if err != nil { + updateStatus(r.client, request.NamespacedName, k8sutil.ReconcileFailedStatus) + return reconcile.Result{}, errors.Wrapf(err, "invalid CephObjectZone CR %q", cephObjectZone.Name) + } + + // Start object reconciliation, updating status for this + updateStatus(r.client, request.NamespacedName, k8sutil.ReconcilingStatus) + + // Make sure an ObjectZoneGroup is present + realmName, reconcileResponse, err := r.reconcileObjectZoneGroup(cephObjectZone) + if err != nil { + return reconcileResponse, err + } + + // Make sure zone group has been created in Ceph Cluster + reconcileResponse, err = r.reconcileCephZoneGroup(cephObjectZone, realmName) + if err != nil { + return reconcileResponse, err + } + + // Create Ceph Zone + reconcileResponse, err = r.createCephZone(cephObjectZone, realmName) + if err != nil { + return r.setFailedStatus(request.NamespacedName, "failed to create ceph zone", err) + } + + // Set Ready status, we are done reconciling + updateStatus(r.client, request.NamespacedName, k8sutil.ReadyStatus) + + // Return and do not requeue + logger.Debug("zone done reconciling") + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZone) createCephZone(zone *cephv1.CephObjectZone, realmName string) (reconcile.Result, error) { + logger.Infof("creating object zone %q in zonegroup %q in realm %q", zone.Name, zone.Spec.ZoneGroup, realmName) + + realmArg := fmt.Sprintf("--rgw-realm=%s", realmName) + zoneGroupArg := fmt.Sprintf("--rgw-zonegroup=%s", zone.Spec.ZoneGroup) + zoneArg := fmt.Sprintf("--rgw-zone=%s", zone.Name) + objContext := object.NewContext(r.context, zone.Name, zone.Namespace) + + // get zone group to see if master zone exists yet + output, err := object.RunAdminCommandNoRealm(objContext, "zonegroup", "get", realmArg, zoneGroupArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + return reconcile.Result{}, errors.Wrapf(err, "ceph zone group %q not found", zone.Spec.ZoneGroup) + } else { + return reconcile.Result{}, errors.Wrapf(err, "radosgw-admin zonegroup get failed with code %d", code) + } + } + + // check if master zone group does not exist yet for period + masterZone, err := decodeMasterZone(output) + if err != nil { + return reconcile.Result{}, errors.Wrapf(err, "failed to parse `radosgw-admin zonegroup get` output") + } + + masterArg := "" + // master zone does not exist yet for zone group + if masterZone == "" { + masterArg = "--master" + } + + // create zone + _, err = object.RunAdminCommandNoRealm(objContext, "zone", "get", realmArg, zoneGroupArg, zoneArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + logger.Debugf("ceph zone %q not found, running `radosgw-admin zone create`", zone.Name) + _, err := object.RunAdminCommandNoRealm(objContext, "zone", "create", realmArg, zoneGroupArg, zoneArg, masterArg) + if err != nil { + return reconcile.Result{}, errors.Wrapf(err, "failed to create ceph zone %q", zone.Name) + } + } else { + return reconcile.Result{}, errors.Wrapf(err, "radosgw-admin zone get failed with code %d", code) + } + } + + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZone) reconcileObjectZoneGroup(zone *cephv1.CephObjectZone) (string, reconcile.Result, error) { + // Verify the object zone API object actually exists + zoneGroup, err := r.context.RookClientset.CephV1().CephObjectZoneGroups(zone.Namespace).Get(zone.Spec.ZoneGroup, metav1.GetOptions{}) + if err != nil { + if kerrors.IsNotFound(err) { + return "", waitForRequeueIfObjectZoneNotReady, err + } + return "", waitForRequeueIfObjectZoneNotReady, errors.Wrapf(err, "error getting cephObjectZoneGroup %v", zone.Spec.ZoneGroup) + } + + logger.Infof("CephObjectZoneGroup %v found", zoneGroup.Name) + return zoneGroup.Spec.Realm, reconcile.Result{}, nil +} + +func (r *ReconcileObjectZone) reconcileCephZoneGroup(zone *cephv1.CephObjectZone, realmName string) (reconcile.Result, error) { + realmArg := fmt.Sprintf("--rgw-realm=%s", realmName) + zoneGroupArg := fmt.Sprintf("--rgw-zonegroup=%s", zone.Spec.ZoneGroup) + objContext := object.NewContext(r.context, zone.Name, zone.Namespace) + + _, err := object.RunAdminCommandNoRealm(objContext, "zonegroup", "get", realmArg, zoneGroupArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + return waitForRequeueIfObjectZoneNotReady, errors.Wrapf(err, "ceph zone group %q not found", zone.Spec.ZoneGroup) + } else { + return waitForRequeueIfObjectZoneNotReady, errors.Wrapf(err, "radosgw-admin zonegroup get failed with code %d", code) + } + } + + logger.Infof("Zone group %q found in Ceph cluster to create ceph zone %q", zone.Spec.ZoneGroup, zone.Name) + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZone) setFailedStatus(name types.NamespacedName, errMessage string, err error) (reconcile.Result, error) { + updateStatus(r.client, name, k8sutil.ReconcileFailedStatus) + return reconcile.Result{}, errors.Wrapf(err, "%s", errMessage) +} + +// updateStatus updates an zone with a given status +func updateStatus(client client.Client, name types.NamespacedName, status string) { + objectZone := &cephv1.CephObjectZone{} + if err := client.Get(context.TODO(), name, objectZone); err != nil { + if kerrors.IsNotFound(err) { + logger.Debug("CephObjectZone resource not found. Ignoring since object must be deleted.") + return + } + logger.Warningf("failed to retrieve object zone %q to update status to %q. %v", name, status, err) + return + } + if objectZone.Status == nil { + objectZone.Status = &cephv1.Status{} + } + + objectZone.Status.Phase = status + if err := opcontroller.UpdateStatus(client, objectZone); err != nil { + logger.Errorf("failed to set object zone %q status to %q. %v", name, status, err) + return + } + logger.Debugf("object zone %q status updated to %q", name, status) +} diff --git a/pkg/operator/ceph/object/zone/zone.go b/pkg/operator/ceph/object/zone/zone.go new file mode 100644 index 000000000..0aa787c8f --- /dev/null +++ b/pkg/operator/ceph/object/zone/zone.go @@ -0,0 +1,51 @@ +/* +Copyright 2020 The Rook Authors. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package zone + +import ( + "encoding/json" + "github.com/pkg/errors" + cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" +) + +type masterZoneType struct { + MasterZone string `json:"master_zone"` +} + +func decodeMasterZone(data string) (string, error) { + var zoneGroupGet masterZoneType + err := json.Unmarshal([]byte(data), &zoneGroupGet) + if err != nil { + return "", errors.Wrap(err, "Failed to unmarshal json") + } + + return zoneGroupGet.MasterZone, err +} + +// validateZoneCR validates the zone arguments +func validateZoneCR(u *cephv1.CephObjectZone) error { + if u.Name == "" { + return errors.New("missing name") + } + if u.Namespace == "" { + return errors.New("missing namespace") + } + if u.Spec.ZoneGroup == "" { + return errors.New("missing zonegroup") + } + return nil +} diff --git a/pkg/operator/ceph/object/zonegroup/controller.go b/pkg/operator/ceph/object/zonegroup/controller.go new file mode 100644 index 000000000..e2f174410 --- /dev/null +++ b/pkg/operator/ceph/object/zonegroup/controller.go @@ -0,0 +1,296 @@ +/* +Copyright 2020 The Rook Authors. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +// Package objectzonegroup to manage a rook object zonegroup. +package zonegroup + +import ( + "context" + "fmt" + "reflect" + "syscall" + "time" + + opcontroller "github.com/rook/rook/pkg/operator/ceph/controller" + "k8s.io/apimachinery/pkg/runtime" + "sigs.k8s.io/controller-runtime/pkg/controller" + "sigs.k8s.io/controller-runtime/pkg/handler" + "sigs.k8s.io/controller-runtime/pkg/manager" + "sigs.k8s.io/controller-runtime/pkg/reconcile" + "sigs.k8s.io/controller-runtime/pkg/source" + + "github.com/coreos/pkg/capnslog" + "github.com/pkg/errors" + cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" + "github.com/rook/rook/pkg/clusterd" + "github.com/rook/rook/pkg/operator/ceph/object" + "github.com/rook/rook/pkg/operator/k8sutil" + "github.com/rook/rook/pkg/util/exec" + kerrors "k8s.io/apimachinery/pkg/api/errors" + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/types" + "sigs.k8s.io/controller-runtime/pkg/client" +) + +const ( + controllerName = "ceph-object-zonegroup-controller" +) + +var waitForRequeueIfObjectZoneNotReady = reconcile.Result{Requeue: true, RequeueAfter: 10 * time.Second} + +var logger = capnslog.NewPackageLogger("github.com/rook/rook", controllerName) + +var cephObjectZoneGroupKind = reflect.TypeOf(cephv1.CephObjectZoneGroup{}).Name() + +// Sets the type meta for the controller main object +var controllerTypeMeta = metav1.TypeMeta{ + Kind: cephObjectZoneGroupKind, + APIVersion: fmt.Sprintf("%s/%s", cephv1.CustomResourceGroup, cephv1.Version), +} + +// ReconcileObjectZoneGroup reconciles a ObjectZoneGroup object +type ReconcileObjectZoneGroup struct { + client client.Client + scheme *runtime.Scheme + context *clusterd.Context +} + +// Add creates a new CephObjectZoneGroup Controller and adds it to the Manager. The Manager will set fields on the Controller +// and Start it when the Manager is Started. +func Add(mgr manager.Manager, context *clusterd.Context) error { + return add(mgr, newReconciler(mgr, context)) +} + +// newReconciler returns a new reconcile.Reconciler +func newReconciler(mgr manager.Manager, context *clusterd.Context) reconcile.Reconciler { + // Add the cephv1 scheme to the manager scheme so that the controller knows about it + mgrScheme := mgr.GetScheme() + cephv1.AddToScheme(mgr.GetScheme()) + + return &ReconcileObjectZoneGroup{ + client: mgr.GetClient(), + scheme: mgrScheme, + context: context, + } +} + +func add(mgr manager.Manager, r reconcile.Reconciler) error { + // Create a new controller + c, err := controller.New(controllerName, mgr, controller.Options{Reconciler: r}) + if err != nil { + return err + } + logger.Info("successfully started") + + // Watch for changes on the CephObjectZoneGroup CRD object + err = c.Watch(&source.Kind{Type: &cephv1.CephObjectZoneGroup{TypeMeta: controllerTypeMeta}}, &handler.EnqueueRequestForObject{}, opcontroller.WatchControllerPredicate()) + if err != nil { + return err + } + + return nil +} + +// Reconcile reads that state of the cluster for a CephObjectZoneGroup object and makes changes based on the state read +// and what is in the CephObjectZoneGroup.Spec +// The Controller will requeue the Request to be processed again if the returned error is non-nil or +// Result.Requeue is true, otherwise upon completion it will remove the work from the queue. +func (r *ReconcileObjectZoneGroup) Reconcile(request reconcile.Request) (reconcile.Result, error) { + // workaround because the rook logging mechanism is not compatible with the controller-runtime loggin interface + reconcileResponse, err := r.reconcile(request) + if err != nil { + logger.Errorf("failed to reconcile: %v", err) + } + + return reconcileResponse, err +} + +func (r *ReconcileObjectZoneGroup) reconcile(request reconcile.Request) (reconcile.Result, error) { + // Fetch the CephObjectZoneGroup instance + cephObjectZoneGroup := &cephv1.CephObjectZoneGroup{} + err := r.client.Get(context.TODO(), request.NamespacedName, cephObjectZoneGroup) + if err != nil { + if kerrors.IsNotFound(err) { + logger.Debug("CephObjectZoneGroup resource not found. Ignoring since object must be deleted.") + return reconcile.Result{}, nil + } + // Error reading the object - requeue the request. + return reconcile.Result{}, errors.Wrap(err, "failed to get CephObjectZoneGroup") + } + + // The CR was just created, initializing status fields + if cephObjectZoneGroup.Status == nil { + updateStatus(r.client, request.NamespacedName, k8sutil.Created) + } + + // Make sure a CephCluster is present otherwise do nothing + _, isReadyToReconcile, cephClusterExists, reconcileResponse := opcontroller.IsReadyToReconcile(r.client, r.context, request.NamespacedName, controllerName) + if !isReadyToReconcile { + // This handles the case where the Ceph Cluster is gone and we want to delete that CR + if !cephObjectZoneGroup.GetDeletionTimestamp().IsZero() && !cephClusterExists { + // Return and do not requeue. Successful deletion. + return reconcile.Result{}, nil + } + return reconcileResponse, nil + } + + // DELETE: the CR was deleted + if !cephObjectZoneGroup.GetDeletionTimestamp().IsZero() { + logger.Debugf("deleting zone group CR %q", cephObjectZoneGroup.Name) + + // Return and do not requeue. Successful deletion. + return reconcile.Result{}, nil + } + + // validate the zone group settings + err = validateZoneGroup(cephObjectZoneGroup) + if err != nil { + updateStatus(r.client, request.NamespacedName, k8sutil.ReconcileFailedStatus) + return reconcile.Result{}, errors.Wrapf(err, "invalid CephObjectZoneGroup CR %q", cephObjectZoneGroup.Name) + } + + // Start object reconciliation, updating status for this + updateStatus(r.client, request.NamespacedName, k8sutil.ReconcilingStatus) + + // Make sure an ObjectRealm Resource is present + reconcileResponse, err = r.reconcileObjectRealm(cephObjectZoneGroup) + if err != nil { + return reconcileResponse, err + } + + // Make sure Realm has been created in Ceph Cluster + reconcileResponse, err = r.reconcileCephRealm(cephObjectZoneGroup) + if err != nil { + return reconcileResponse, err + } + + // Create/Update Ceph Zone Group + reconcileResponse, err = r.createCephZoneGroup(cephObjectZoneGroup) + if err != nil { + return r.setFailedStatus(request.NamespacedName, "failed to create ceph zone", err) + } + + // Set Ready status, we are done reconciling + updateStatus(r.client, request.NamespacedName, k8sutil.ReadyStatus) + + // Return and do not requeue + logger.Debug("zone group done reconciling") + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZoneGroup) createCephZoneGroup(zoneGroup *cephv1.CephObjectZoneGroup) (reconcile.Result, error) { + logger.Infof("creating object zone group %q in realm %q", zoneGroup.Name, zoneGroup.Spec.Realm) + + realmArg := fmt.Sprintf("--rgw-realm=%s", zoneGroup.Spec.Realm) + zoneGroupArg := fmt.Sprintf("--rgw-zonegroup=%s", zoneGroup.Name) + objContext := object.NewContext(r.context, zoneGroup.Name, zoneGroup.Namespace) + + // get period to see if master zone group exists yet + output, err := object.RunAdminCommandNoRealm(objContext, "period", "get", realmArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + return reconcile.Result{}, errors.Wrapf(err, "ceph period %q not found", zoneGroup.Spec.Realm) + } else { + return reconcile.Result{}, errors.Wrapf(err, "radosgw-admin period get failed with code %d", code) + } + } + + // check if master zone group does not exist yet for period + masterZoneGroup, err := decodeMasterZoneGroup(output) + if err != nil { + return reconcile.Result{}, errors.Wrapf(err, "failed to parse `radosgw-admin period get` output") + } + + masterArg := "" + if masterZoneGroup == "" { + masterArg = "--master" + } + + // create zone group + _, err = object.RunAdminCommandNoRealm(objContext, "zonegroup", "get", realmArg, zoneGroupArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + logger.Debugf("ceph zone group %q not found, running `radosgw-admin zonegroup create`", zoneGroup.Name) + _, err := object.RunAdminCommandNoRealm(objContext, "zonegroup", "create", realmArg, zoneGroupArg, masterArg) + if err != nil { + return reconcile.Result{}, errors.Wrapf(err, "failed to create ceph zone group %q", zoneGroup.Name) + } + } else { + return reconcile.Result{}, errors.Wrapf(err, "radosgw-admin zonegroup get failed with code %d", code) + } + } + + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZoneGroup) reconcileObjectRealm(zoneGroup *cephv1.CephObjectZoneGroup) (reconcile.Result, error) { + // Verify the object realm API object actually exists + _, err := r.context.RookClientset.CephV1().CephObjectRealms(zoneGroup.Namespace).Get(zoneGroup.Spec.Realm, metav1.GetOptions{}) + if err != nil { + if kerrors.IsNotFound(err) { + return waitForRequeueIfObjectZoneNotReady, err + } + return waitForRequeueIfObjectZoneNotReady, errors.Wrapf(err, "error finding CephObjectRealm %s", zoneGroup.Spec.Realm) + } + + logger.Infof("CephObjectRealm %q found for CephObjectZoneGroup %q", zoneGroup.Spec.Realm, zoneGroup.Name) + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZoneGroup) reconcileCephRealm(zoneGroup *cephv1.CephObjectZoneGroup) (reconcile.Result, error) { + realmArg := fmt.Sprintf("--rgw-realm=%s", zoneGroup.Spec.Realm) + objContext := object.NewContext(r.context, zoneGroup.Name, zoneGroup.Namespace) + + _, err := object.RunAdminCommandNoRealm(objContext, "realm", "get", realmArg) + if err != nil { + if code, ok := exec.ExitStatus(err); ok && code == int(syscall.ENOENT) { + return waitForRequeueIfObjectZoneNotReady, errors.Wrapf(err, "ceph realm %q not found", zoneGroup.Spec.Realm) + } else { + return waitForRequeueIfObjectZoneNotReady, errors.Wrapf(err, "radosgw-admin realm get failed with code %d", code) + } + } + + logger.Infof("Realm %q found in Ceph cluster to create ceph zone group %q", zoneGroup.Spec.Realm, zoneGroup.Name) + return reconcile.Result{}, nil +} + +func (r *ReconcileObjectZoneGroup) setFailedStatus(name types.NamespacedName, errMessage string, err error) (reconcile.Result, error) { + updateStatus(r.client, name, k8sutil.ReconcileFailedStatus) + return reconcile.Result{}, errors.Wrapf(err, "%s", errMessage) +} + +// updateStatus updates an zone group with a given status +func updateStatus(client client.Client, name types.NamespacedName, status string) { + objectZoneGroup := &cephv1.CephObjectZoneGroup{} + if err := client.Get(context.TODO(), name, objectZoneGroup); err != nil { + if kerrors.IsNotFound(err) { + logger.Debug("CephObjectZoneGroup resource not found. Ignoring since object must be deleted.") + return + } + logger.Warningf("failed to retrieve object zone group %q to update status to %q. %v", name, status, err) + return + } + if objectZoneGroup.Status == nil { + objectZoneGroup.Status = &cephv1.Status{} + } + + objectZoneGroup.Status.Phase = status + if err := opcontroller.UpdateStatus(client, objectZoneGroup); err != nil { + logger.Errorf("failed to set object zone group %q status to %q. %v", name, status, err) + return + } + logger.Debugf("object zone group %q status updated to %q", name, status) +} diff --git a/pkg/operator/ceph/object/zonegroup/zonegroup.go b/pkg/operator/ceph/object/zonegroup/zonegroup.go new file mode 100644 index 000000000..a6be59727 --- /dev/null +++ b/pkg/operator/ceph/object/zonegroup/zonegroup.go @@ -0,0 +1,51 @@ +/* +Copyright 2020 The Rook Authors. All rights reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +*/ + +package zonegroup + +import ( + "encoding/json" + "github.com/pkg/errors" + cephv1 "github.com/rook/rook/pkg/apis/ceph.rook.io/v1" +) + +type masterZoneGroupType struct { + MasterZoneGroup string `json:"master_zonegroup"` +} + +func decodeMasterZoneGroup(data string) (string, error) { + var periodGet masterZoneGroupType + err := json.Unmarshal([]byte(data), &periodGet) + if err != nil { + return "", errors.Wrap(err, "Failed to unmarshal json") + } + + return periodGet.MasterZoneGroup, err +} + +// validateZoneGroup validates the zonegroup arguments +func validateZoneGroup(u *cephv1.CephObjectZoneGroup) error { + if u.Name == "" { + return errors.New("missing name") + } + if u.Namespace == "" { + return errors.New("missing namespace") + } + if u.Spec.Realm == "" { + return errors.New("missing realm") + } + return nil +} diff --git a/tests/framework/installer/ceph_installer.go b/tests/framework/installer/ceph_installer.go index f2d645ef0..0cd6a0b87 100644 --- a/tests/framework/installer/ceph_installer.go +++ b/tests/framework/installer/ceph_installer.go @@ -524,6 +524,9 @@ func (h *CephInstaller) UninstallRookFromMultipleNS(systemNamespace string, name "cephblockpools.ceph.rook.io", "cephobjectstores.ceph.rook.io", "cephobjectstoreusers.ceph.rook.io", + "cephobjectrealms.ceph.rook.io", + "cephobjectzonegroups.ceph.rook.io", + "cephobjectzones.ceph.rook.io", "cephfilesystems.ceph.rook.io", "cephnfses.ceph.rook.io", "cephclients.ceph.rook.io", diff --git a/tests/framework/installer/ceph_manifests.go b/tests/framework/installer/ceph_manifests.go index afcfd4442..95103aeaf 100644 --- a/tests/framework/installer/ceph_manifests.go +++ b/tests/framework/installer/ceph_manifests.go @@ -512,6 +512,57 @@ spec: --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition +metadata: + name: cephobjectrealms.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectRealm + listKind: CephObjectRealmList + plural: cephobjectrealms + singular: cephobjectrealm + shortNames: + - realm + - realms + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzonegroups.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZoneGroup + listKind: CephObjectZoneGroupList + plural: cephobjectzonegroups + singular: cephobjectzonegroup + shortNames: + - zonegroup + - zonegroups + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzones.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZone + listKind: CephObjectZoneList + plural: cephobjectzones + singular: cephobjectzone + shortNames: + - zone + - zones + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition metadata: name: cephblockpools.ceph.rook.io spec: diff --git a/tests/integration/ceph_upgrade_test.go b/tests/integration/ceph_upgrade_test.go index cdf1f0d64..22fface06 100644 --- a/tests/integration/ceph_upgrade_test.go +++ b/tests/integration/ceph_upgrade_test.go @@ -365,7 +365,58 @@ spec: minimum: 1 maximum: 100 subresources: - status: {}` + status: {} +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectrealms.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectRealm + listKind: CephObjectRealmList + plural: cephobjectrealms + singular: cephobjectrealm + shortNames: + - realm + - realms + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzonegroups.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZoneGroup + listKind: CephObjectZoneGroupList + plural: cephobjectzonegroups + singular: cephobjectzonegroup + shortNames: + - zonegroup + - zonegroups + scope: Namespaced + version: v1 +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cephobjectzones.ceph.rook.io +spec: + group: ceph.rook.io + names: + kind: CephObjectZone + listKind: CephObjectZoneList + plural: cephobjectzones + singular: cephobjectzone + shortNames: + - zone + - zones + scope: Namespaced + version: v1` } func upgradeManifestTo1_3(namespace string) string {