Update the rbd and cephfs docs to include the new prefixed style
of name for the rook-managed csi instances. Document that this
prefix must match the namespace the rook operator is deployed in.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
- Improve readability of 'Replicated' and 'Erasure Coded' sections
- Clarify replicated sample requires 1 OSD located on 3 different nodes
- Add whitespace to 'Pool Settings' section
- Add link to the k8s 'StorageClass' concept
- Rename instances of 'object store' as literal
`CephObjectStore`
- Rename instances of 'object store user' as literal
`CephObjectStoreUser`
- Other minor verbiage and syntax clean-up
Signed-off-by: Michael Fritch <mfritch@suse.com>
The Nautilus v14.2.2 release was out last week. This is the new
default version for Rook in the documentation and manifests.
Signed-off-by: travisn <tnielsen@redhat.com>
With the 1.1 release we should not be creating bluestore OSDs
with rook's legacy partitioning scheme. All new bluestore OSDs
should be created with ceph-volume. Therefore, the miminum
version allowed is a version of mimic supporting ceph-volume.
This commit also adds consistent version checking between
the Add and Update methods for the operator handling the
CR events.
Signed-off-by: travisn <tnielsen@redhat.com>
- Updated Ceph Cluster CRD docuementation on how to use StorageClassDeviceSets to create OSD on PVC with examples CRD
- Updated the release notes with the new StorageClassDeviceSets functionality.
Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
CSI is now the preferred storage driver for Rook.
By default both the CSI and flex drivers will be started
by the operator. In a future release the flex driver
will be deprecated, but for now is still supported.
The drivers can be disabled with an environment
variable in the operator deployment in operator.yaml.
If users are only using one driver or the other there is
no need to enable both drivers.
Signed-off-by: travisn <tnielsen@redhat.com>
this adds the access for rook to create monitoring resources
across namespaces and provides servicemonitor config to
monitor the current service being created
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
creating custom access/secret keys without properly base64 encoding them resulted in failed access to minio dashboard
Signed-off-by: jbmcfar <jbmcfar@sandia.gov>
fail when metadataDevice is specified but ceph-volume batch does not
automatically use the metadataDevice as the block.db. (#3092)
Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
Add Rook-Ceph PSP requirements to the `common.yaml` file. This is
commented for users to inspect and make informed decisions about their
own clusters just as with the rest of the file. For user clusters
without pod security policies enabled, this part of the config will be
a don't-care and not harm anything.
Because the PSP setup is now codified in `common.yaml`, it is no longer
necessary to have documentation for pod security policies, and the
`Documentation/ceph-psp.md` file is deleted.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
In the 0.9 release rook converted the v1beta1 CRD resources
to v1 resources. This conversion code is no longer necessary
as we will be using v1 resources going forward.
The code paths will not be triggered anymore, thus
removing the dead code.
Signed-off-by: travisn <tnielsen@redhat.com>
Run the script like this:
./contrib/backport_to_stable_branch.sh STABLE_BRANCH_NAME COMMIT_SHA1 BACKPORT_BRANCH_NAME
Signed-off-by: Sébastien Han <seb@redhat.com>
this patch introduces failure-domain aware scheduling for monitor pods.
previously the only policy was to avoid scheduling monitors on the same
nodes. the new policy tries to spread monitors across zones and nodes.
NOTE: this patch only enforces the well-known k8s "zone"
failure-domain label (i.e. it does not look for region labels). this
decision is made for two reasons. first, eliminating explicit
scheduling is an important goal and will be done as part of upcoming
work. this will enable all supported failure-domain labels to be
handled. second, the "zone" domain is likely more appropriate for a
larger portion of users (as opposed to the geographical distinction
made with region), and supporting multiple failure domain types would
add significant complexity for a temporary resolution to failure
domain scheduling.
at a high-level this patch introduces a data structure `NodeUsage` that
is a pairing of a v1.Node and metadata relevant to scheduling: number of
monitor pods on the node, and its status w.r.t. to being scheduable for
new monitor pods.
each scheduling event computes a unified data structure that organizes
for each node a `NodeUsage` structure into per-failure-zone groups.
subsequent algorithms rely soley on this unified structure to make
scheduling decisions.
there are two entry points to the changes made:
1) during orchestration mon:Cluster:assignMons is invoked to make a
place new monitor pods onto nodes. the core scheduling logic is now
self-contained in mon:scheduleMonitor which implements node and zone
aware scheduling policies.
2) periodic health checks resolve any conflicts that may occur due to
changes to the cluster (e.g. new nodes, configuration changes, etc...).
the two existing checks: mons on invalid nodes, and overloaded nodes are
both handled.
fixes: #2603
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Add `--region :default-placement` in RadosGW tutorial to avoid the InvalidLocationConstraint error
Signed-off-by: Maxime Guyot <maxime.guyot@elits.com>
Use cmd-reporter to detect Ceph version.
The job to detect Ceph version needs to run in a security account context
capable of getting/creating ConfigMaps, so a new security account, role,
and role binding are created for the cmd-runner jobs. In the future, the
Ceph mgr may create jobs in this service account for running
`ceph-volume` on nodes, and this also keeps Rook operators controlling
multiple Ceph clusters independent.
Fixes#2686 - ceph version job not working correctly with graylog
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
* This error is found in creating CephFS StorageClass using document `Documentation/ceph-csi-drivers.md`: "expect a pool named `cephfs_data` in your Ceph cluster"
* **`kubectl apply -f pool.yaml` using metadata.name "cephfs_data" produce error:** The CephBlockPool "cephfs_data" is invalid: metadata.name: Invalid value: "cephfs_data": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
* accoding to `Documentation/ceph-filesystem-crd.md` linked from `Documentation/ceph-csi-drivers.md` file and field test, it should be `myfs-data0`
Signed-off-by: Joel Huang <joelhy@gmail.com>
Signed-off-by: Guy Margalit <guymguym@gmail.com>
Co-Authored-By: Sébastien Han <seb@redhat.com>
Co-Authored-By: Travis Nielsen <tnielsen@redhat.com>
This change is meant to allow running operators locally on a developer machine.
The idea is to allow faster development cycles by reducing the time and complexity of building -> deploying -> debugging on cluster.
For operators that rely only on kubernetes API this works easily - see cockroachdb and minio examples in development-flow doc.
The change includes:
- rook.NewContext() - Refactored to remove repeating initialization code that was copy-pasted in most of the operators in order to create the clusterd.Context and the Clientsets. Also it detects the mode of working in-cluster vs external and sets up the external mode with standard user config (~/.kube/config) and a job executor.
- rook.GetOperatorImage() - Refactor this repeating code in many operators to detect the operator pod image. Also added a global flag --operator-image that developers can use to override this when running locally.
- rook.TerminateOnError() - Added a convenient function.
The current doc was outdated and hardcoded, so removing the example from
the doc and add a proper psp.yaml file that people can use and
contribute too.
Closes: https://github.com/rook/rook/issues/3309
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit does multiple things:
* remove support for AllNodes where we would deploy one rgw per node on
all the nodes.
* a transition path is implemented in the code so that if someone has an
existing deployment, daemonsets will be removed and replaced by an
deployments.
* when using "instances", each rgw deployed has its own key which makes
Ceph reporting the exact number of rgw running, see:
```
[root@rook-ceph-operator-775cf575c5-bh4sr /]# ceph -s
cluster:
id: 611fcf39-0669-4864-9a12-debb35c0397a
health: HEALTH_OK
services:
mon: 3 daemons, quorum a,b,c (age 12h)
mgr: a(active, since 12h)
osd: 3 osds: 3 up (since 12h), 3 in (since 12h)
rgw: 3 daemons active (my.store.a, my.store.b, my.store.c)
data:
pools: 6 pools, 600 pgs
objects: 235 objects, 3.8 KiB
usage: 3.0 GiB used, 84 GiB / 87 GiB avail
pgs: 600 active+clean
```
Closes: https://github.com/rook/rook/issues/2474, https://github.com/rook/rook/issues/2957 and https://github.com/rook/rook/issues/3245
Signed-off-by: Sébastien Han <seb@redhat.com>
Update the csi document to assist the user in updating the new
`clusterID` field that is used to map to mons in the configmap
rook maintains.
Signed-off-by: John Mulligan <jmulligan@redhat.com>
John Mulligan
Travis Nielsen
Michael Fritch
Ashish Ranjanandrohan47
Kristoffer Grönlund
Noah Watkins
Umanga Chapagain
Anton Skriptsov
jbmcfar
Michael Vollman
Blaine Gardner
Mateusz Gozdek
Dmitry Yusupov
Sébastien Han
Maxime Guyot
Khanh Ngo
Alexander Trost
Joel Huang
Madhu Rajanna
Dmitry Yusupov
Guy Margalit