- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.
Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
In order for the osd service account to reference the
cephcluster CR in the owner references, OpenShift requires
access to the CephCluster CRD. This adds the minimal changes
needed to appease OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
The CSI driver requires the ceph key for administrating
the cluster. The key is stored in a secret and specified in the
storage class. The operator will now automatically generate
a secret named rook-ceph-csi that can be specified by
default in the storage class instead of requiring the
admin to create it.
Signed-off-by: travisn <tnielsen@redhat.com>
The discovery daemon is only necessary to be run when
OSDs are being created on raw devices and detect when
new devices are added to the cluster. If OSDs do not need
to be configured on devices, the discovery daemonset
has no need to be started.
Signed-off-by: travisn <tnielsen@redhat.com>
The version of the csi driver to be installed that is officially
supported is the default baked into rook. If a different version
needs to be specified, it is officially unsupported. Therefore,
we leave the csi driver version settings unspecified in operator.yaml
so we can officially support the defaults.
Signed-off-by: travisn <tnielsen@redhat.com>
CSI is now the preferred storage driver for Rook.
By default both the CSI and flex drivers will be started
by the operator. In a future release the flex driver
will be deprecated, but for now is still supported.
The drivers can be disabled with an environment
variable in the operator deployment in operator.yaml.
If users are only using one driver or the other there is
no need to enable both drivers.
Signed-off-by: travisn <tnielsen@redhat.com>
In order for the osd service account to reference the
cephcluster CR in the owner references, OpenShift requires
access to the CephCluster CRD. This adds the minimal changes
needed to appease OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
this adds the access for rook to create monitoring resources
across namespaces and provides servicemonitor config to
monitor the current service being created
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
Ceph CSI now requires a tempfs mountpoint and will not function without
it. These changes are cloned from the ceph csi yaml.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Add Rook-Ceph PSP requirements to the `common.yaml` file. This is
commented for users to inspect and make informed decisions about their
own clusters just as with the rest of the file. For user clusters
without pod security policies enabled, this part of the config will be
a don't-care and not harm anything.
Because the PSP setup is now codified in `common.yaml`, it is no longer
necessary to have documentation for pod security policies, and the
`Documentation/ceph-psp.md` file is deleted.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Update the upgrade-from-*.yaml files to be appropriate for upgrade from
v1.0 to master. Currently no creations or deletions are necessary, but
the files are kept as a placeholder for the upcoming v1.1 upgrade docs.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Use cmd-reporter to detect Ceph version.
The job to detect Ceph version needs to run in a security account context
capable of getting/creating ConfigMaps, so a new security account, role,
and role binding are created for the cmd-runner jobs. In the future, the
Ceph mgr may create jobs in this service account for running
`ceph-volume` on nodes, and this also keeps Rook operators controlling
multiple Ceph clusters independent.
Fixes#2686 - ceph version job not working correctly with graylog
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
* This error is found in creating CephFS StorageClass using document `Documentation/ceph-csi-drivers.md`: "expect a pool named `cephfs_data` in your Ceph cluster"
* **`kubectl apply -f pool.yaml` using metadata.name "cephfs_data" produce error:** The CephBlockPool "cephfs_data" is invalid: metadata.name: Invalid value: "cephfs_data": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
* accoding to `Documentation/ceph-filesystem-crd.md` linked from `Documentation/ceph-csi-drivers.md` file and field test, it should be `myfs-data0`
Signed-off-by: Joel Huang <joelhy@gmail.com>
Previously, Rook Agent and Discovery DaemonSet deployment didn't allow
adding nodeAffinity. This commit adds nodeAffinity spec to daemonSet
deployment, which can be configured through environment variables in
operator deployment yaml.
+ Support multiple LabelKey, each with multiple LabelValue
+ Support multiple LabelKey with no value
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
Currently in cluster deploy, ceph image points to ceph/ceph:v14.2.1-20190430.
Changing this to use ceph/daemon-base:latest-nautilus-devel as its the latest nautilus.
Ceph nautilus has some changes in cephsfs mgr module, which is required by ceph-csi
canary image, without which the create/purge of RWX PVCs will fail. These changes
are not yet part of any released ceph version, i.e v14.2.1-20190430. Even in the
short future, we expect some changes to land in Nautilus after 14.2.2, hence
waiting for 14.2.2 release and updating to it, will not solve the issue.
Therefore using latest-nautilus-devel, which will always have the changes
required by ceph-csi. This is only a temperory change in the master, until
14.2.3 is released.
Signed-off-by: Poornima G <pgurusid@redhat.com>
Ceph-CSI repository had a recent PR [1] merged that contains
changes to the pod manifests.
This PR to Rook updates the pod manifests to catchup with the
changes to Ceph-CSI master.
Once Ceph-CSI has a 1.1.0 release, future PRs to Rook on manifest
changes would track that branch (at least for Rook 1.1 release).
[1] PR in ceph-csi changing the manifest:
https://github.com/ceph/ceph-csi/pull/443
Signed-off-by: ShyamsundarR <srangana@redhat.com>
The current doc was outdated and hardcoded, so removing the example from
the doc and add a proper psp.yaml file that people can use and
contribute too.
Closes: https://github.com/rook/rook/issues/3309
Signed-off-by: Sébastien Han <seb@redhat.com>
PR #3217 changed the pod manifest to drop some parameters to the
ceph-csi pods. This also resulted in a change to the operator with CSI
yaml for non-openshift case, but failed to update similar yaml's for
the openshift case.
This commit rectifies this problem.
Updates: #3312
Signed-off-by: ShyamsundarR <srangana@redhat.com>
This commit does multiple things:
* remove support for AllNodes where we would deploy one rgw per node on
all the nodes.
* a transition path is implemented in the code so that if someone has an
existing deployment, daemonsets will be removed and replaced by an
deployments.
* when using "instances", each rgw deployed has its own key which makes
Ceph reporting the exact number of rgw running, see:
```
[root@rook-ceph-operator-775cf575c5-bh4sr /]# ceph -s
cluster:
id: 611fcf39-0669-4864-9a12-debb35c0397a
health: HEALTH_OK
services:
mon: 3 daemons, quorum a,b,c (age 12h)
mgr: a(active, since 12h)
osd: 3 osds: 3 up (since 12h), 3 in (since 12h)
rgw: 3 daemons active (my.store.a, my.store.b, my.store.c)
data:
pools: 6 pools, 600 pgs
objects: 235 objects, 3.8 KiB
usage: 3.0 GiB used, 84 GiB / 87 GiB avail
pgs: 600 active+clean
```
Closes: https://github.com/rook/rook/issues/2474, https://github.com/rook/rook/issues/2957 and https://github.com/rook/rook/issues/3245
Signed-off-by: Sébastien Han <seb@redhat.com>
New versions of ceph csi rbd expect a clusterID that will be used
to index into the config map and determine what mons to use.
Also, remove mons from config example.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Create and maintain a config map that meets the requirements of
the ceph csi such that Rook can maintain the contents of config
map with up-to-date mon information to be used later by csi.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
This is a temporary change that fixes the templates so that they match
the so-called "canary" tag in csi. This version of the csi rbd driver
supports a external mon configuration (in a config map).
Signed-off-by: John Mulligan <jmulligan@redhat.com>
Temporary change to support the testing and development of new
integration between ceph csi and rook. This "canary" tag points
at new versions of csi that support taking mon config from a
config map.
Signed-off-by: John Mulligan <jmulligan@redhat.com>