Commit Graph
2411 Commits
Author SHA1 Message Date
Sébastien Han c8ee5674dd Merge pull request #8103 from leseb/rm-jenkins
ci: remove jenkins from master branch
2021-07-23 18:20:10 +02:00
Sébastien Han 0811359d28 Merge pull request #8358 from leseb/move-to-quay
ceph: move all of our docker.io reference to quay.io
2021-07-23 18:03:53 +02:00
Travis Nielsen fd141af92c Merge pull request #8319 from BlaineEXE/disable-raw-mode-for-disks
ceph: disable raw mode for disks
2021-07-22 11:17:54 -06:00
Blaine Gardner c8a2db368a ceph: disable raw mode for disks
Even if we can use raw mode, do NOT use raw mode on disks. Ceph bluestore disks can
sometimes appear as though they have "phantom" Atari (AHDI) partitions created on them
when they don't in reality. This is due to a series of bugs in the Linux kernel when it
is built with Atari support enabled. This behavior does not appear for raw mode OSDs on
partitions, and we need the raw mode to create partition-based OSDs. We cannot merely
skip creating OSDs on "phantom" partitions due to a bug in `ceph-volume raw inventory`
which reports only the phantom partitions (and malformed OSD info) when they exist and
ignores the original (correct) OSDs created on the raw disk.

Resolves https://github.com/rook/rook/issues/7940

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-07-22 09:59:56 -06:00
Sébastien Han 6bce1ff3e9 ceph: move all of our docker.io reference to quay.io
Recently, the builds of `ceph/ceph` image moved to quay.io, see
https://github.com/ceph/ceph-build/pull/1883 for more details.
Current images will remain but new builds will happen on quay.io only.

This means that tags such as `v14.2`, `v15.2`,`v16.2` will need to
switch to quay.io to get updates.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-22 11:17:05 +02:00
Travis NielsenandNicolaj Græsholt f114db952b nfs: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for nfs for full fidelity of all settings.

Co-authored-by: Nicolaj Græsholt <figaw@hotmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
Travis Nielsen 093cf6dbbd cassandra: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for cassandra for full fidelity of all the
settings.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
Travis Nielsen a0b4b30101 ceph: example manifest for stretch cluster in aws
Show an example of configuring a stretch cluster in AWS.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-19 14:30:45 -06:00
Travis Nielsen 64df4cdbb6 ceph: update default ceph image to latest v16.2.5
The latest pacific release is 16.2.5 so we set this as the default
for Rook to deploy.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-16 10:21:48 -06:00
Satoru Takeuchi fe8efeb028 docs: add the example of pvc-based ceph cluster on top of bare metal
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-07-14 21:11:39 +00:00
Jiffin Tony Thottan 1665ad6ea7 ceph: add support for tls certs via k8s tls secrets for rgw
With this PR the RGW can accept TLS certs as K8s TLS secrets

Fixes: 2079
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-07-06 16:30:16 +05:30
Sébastien Han b578f916e7 ceph: add fs mirror config
Similarly to block volume replication, Ceph is capable of replicating the
content of a Ceph Filesystem from one cluster to another.
For this, during the 1.6 cycle, we introduced a new CRD called
CephFilesystemMirror which effectively deploys a cephfs-mirror daemon.
However, configuring peers to enable replication between two clusters
had to be done manually.
Also various bug fix made it in Ceph eventually and the minimum required
version for this to work is to run on Ceph Pacific 16.2.5 at least.

So the automatic configuration of Ceph Filesystem peers is now possible.

By editing the CephFilesystem CRD, you can now turn on mirroring:

```yaml
  mirroring:
    enabled: false
    # list of Kubernetes Secrets containing the peer token
    # for more details see: https://docs.ceph.com/en/latest/dev/cephfs-mirroring/#bootstrap-peers
    peers:
      secretNames:
        - secondary-cluster-peer
```

Also, the mirroring status is displayed in the CR status:

```
status:
  info:
    fsMirrorBootstrapPeerSecretName: fs-peer-token-myfs
  mirroringStatus:
    daemonsStatus:
    - daemon_id: 4186
      filesystems:
      - filesystem_id: 2
        name: myfs
    lastChecked: "2021-07-01T14:16:29Z"
  phase: Ready
  snapshotScheduleStatus:
    lastChecked: "2021-07-01T14:16:29Z"
    snapshotSchedules:
    - fs: myfs
      path: /
      rel_path: /
      retention: {}
      schedule: 24h
```

Closes: https://github.com/rook/rook/issues/7063
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-01 17:35:19 +02:00
Santosh Pillai 4041868e2e ceph: hybrid Storage Pools
Creates hybrid crush rule for choosing Primary OSD for
high performing SSD devices and remaining OSD for low performance HDD devices.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-07-01 11:16:54 +05:30
Blaine Gardner 037945aabc Merge pull request #8112 from BlaineEXE/dependencies-2-son-of-dependencies
ceph: block delete object store when buckets and/or users exist
2021-06-30 12:45:58 -06:00
Blaine Gardner 53459bbb44 Merge pull request #8195 from BlaineEXE/ignore-atari-partitions
ceph: ignore atari partitions when provisioning OSDs
2021-06-30 11:26:56 -06:00
Blaine Gardner 7f516b9e3d ceph: ignore atari partitions when scanning disks
Ceph bluestore raw disks can sometimes appear as though they have Atari
(AHDI) partitions on them. If a disk has Atari partitions, we just
ignore them as though they don't exist. This should be a safe assumption
since the hardware was last manufactured in 1992 and likely can't run
Kubernetes.

If we don't ignore the Atari partitions, Rook can create a new OSD on a
disk that is already running an OSD, corrupting the first and possibly
also the latest OSD. This can happen an arbitrary number of times per
disk.

More info: https://github.com/rook/rook/issues/7940

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-30 09:42:33 -06:00
Blaine Gardner c22f545ebf ceph: block delete object store when buckets exist
Block deletion of CephObjectStore resources when buckets exist in the
object store.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-29 14:31:39 -06:00
Rakshith R 2baf50771e ceph: update csi node-driver-registrar to latest release
This commit updates csi node-driver-registrar to latest
release v2.2.0.
https://github.com/kubernetes-csi/node-driver-registrar/releases

Signed-off-by: Rakshith R <rar@redhat.com>
2021-06-24 10:22:51 +05:30
Travis Nielsen 476ca7a792 Merge pull request #8145 from pavilalopes/update-docs
docs: clarify resource documentation
2021-06-18 07:57:39 -06:00
Pedro Lopes 9d4820ec56 docs: clarify resource documentation
Move MDS related information from the Cluster to the Filesystem document.
Document how resource requests influence Ceph configuration keys.

Signed-off-by: Pedro Lopes <paol@draic.org>
2021-06-18 09:25:54 +01:00
Travis Nielsen adf5027971 docs: cluster helm chart clarifies important settings
Some settings are critical to update in the helm values, so we document
them as clear as possible

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-17 15:03:31 -06:00
Travis Nielsen ec867d6a55 docs: helm chart command should directly create namespace
The helm doc example is more natural with the command to directly
create the namespace when needed instead of as a separate command.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-17 15:03:31 -06:00
Travis Nielsen d5031c969b docs: instructions for taking over cluster CR with helm chart
The CephCluster CR for an existing cluster can be taken over by
the new cluster helm chart if desired, for simpler management
of the RBAC and other resources.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-17 15:03:30 -06:00
Madhu Rajanna 08997434ac ceph: update csi sidecar images to latest release
This commit updates all the kubernetes sidecar
images to latest available release version.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-15 08:09:27 +05:30
Blaine Gardner 21e290e003 ceph: implement dependencies for CephCluster
Implement the first step of `design/ceph/resource-dependencies.md` to
add dependency checking when deleting a CephCluster.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-10 10:06:10 -06:00
Sébastien Han 18b1477352 ci: remove jenkins from master branch
Thank you Jenkins for all those years, but it's to retire.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-10 15:34:15 +02:00
Travis Nielsen 06fbf29b3c Merge pull request #8028 from degorenko/device-classes-resource-limits
ceph: add ability to set resource limit for OSDs based on device classes
2021-06-08 09:01:14 -06:00
Denis Egorenko 8a556fd847 ceph: add ability to set resource limit for OSDs based on device classes
Currently it is not possible to set resource limits based on device classes
for different OSDs. Now adding an ability to use predefined keys for main
cluster spec Resource section to reflect resource limits for different OSDs
with different device classes.

Closes: https://github.com/rook/rook/issues/8007
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
2021-06-08 17:46:41 +04:00
Travis Nielsen 5e86ef8b0c docs: clarify not to change crdsenabled in helm chart
Changing the crds.enabled setting in a running cluster will cause the
CRs to be deleted, which could be disastrous for a cluster. Add
and clarify the warning in the docs to prevent it and link
to recovery steps if it does happen.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-03 14:04:47 -06:00
Travis Nielsen abdaab1a75 docs: disaster recovery guide for deleted cephcluster
If the CephCluster CR or other CRs are mistakenly deleted,
the cluster can remain intact with steps to remove the owner
references to the cluster and re-creating the CRs.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-03 12:00:59 -06:00
Travis Nielsen b059cb9c75 Merge pull request #8017 from sp98/update-ceph-teardown-doc
docs: update ceph-teardown doc
2021-06-03 11:48:22 -06:00
Santosh Pillai 2ed122bc57 docs: update ceph-teardown doc
update ceph teardown doc to include cleanupPolicy which enables automatic clean up of the dataDirHostPath and the drives

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-06-03 11:32:14 +05:30
Travis Nielsen 4cf43bef95 Merge pull request #8006 from Rakshith-R/feature/options-toleration-affinity
ceph: add support for adding rbd and cephfs tolerations and affinities separately
2021-06-02 07:55:23 -06:00
Rakshith R 87cf668b28 ceph: add option for adding csi pods' tolerations & affinity separately
This commit adds support for adding tolerations and affinities for
cephfs and rbd (provisioner & nodeplugin) pods separately.

Closes: #7060

Signed-off-by: Rakshith R <rar@redhat.com>
2021-06-02 11:30:38 +05:30
rohan47 312854c4cb ceph: updated multus documentation with examples
updated multus documentation with ceph-cluster example and multiple
examples to show the network selector settings.

Signed-off-by: rohan47 <rohgupta@redhat.com>
2021-06-01 23:19:39 +05:30
Travis Nielsen 1af84e34c0 Merge pull request #7778 from henryzhangsta/ceph-cluster-helm-chart
ceph: implement CephCluster CRD Helm Chart
2021-05-27 12:41:01 -06:00
Henry Zhang b59858af38 ceph: implement CephCluster CRD Helm Chart
This manages a CephCluster CRD, and is designed to be deployed into its own namespace to manage a single Ceph cluster independent of the main Rook operator.

Additional functions:
- Ability to setup ServiceMonitor for the Prometheus operator

Signed-off-by: Henry Zhang <me@henry.dev>
2021-05-27 01:08:27 -07:00
Blaine Gardner 285dad339b ceph: update references to v15.2.11 to v16.2.4
Ceph v16 will be the default in Rook v1.7. Update references in the
master branch from v15 (currently v15.2.11) to v16 (currently v16.2.4).

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-05-26 12:30:18 -06:00
Shachar Sharon 4509958bab docs: call partprobe after zapping device
Play safe: after zapping devices, issue a call to 'partprobe' to inform
the OS on partition table changes. This may assists users who face
voodoo partition creation, as described in those ROOK issues:

https://github.com/rook/rook/issues/7734
https://github.com/rook/rook/issues/7940

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2021-05-20 10:00:01 +03:00
Travis Nielsen 142c3fd891 ceph: updated dashboard ingress example for v1
The ingress kind is found in apiversion networking.k8s.io/v1 since K8s
1.19. We now update the examples to use v1 instead of v1beta1.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-18 11:20:13 -06:00
Travis Nielsen bbf29dbe70 Merge pull request #7922 from leseb/doc-fix
doc: fix patch command
2021-05-18 07:41:48 -06:00
Sébastien Han bcdc7f8616 docs: fix patch command
The JSON was invalid.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-18 10:26:55 +02:00
Satoru Takeuchi c2adbdd8a8 ceph: remove an missing field in crd
`CephObjectStore->gateway->type` is not used. Rook has only supported s3-like
interface and hasn't had no code which handles `type` field.

In addition, this field was removed from CRD in the following commit.

ceph: auto-gen crds
31db03fece

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-05-17 07:51:13 +00:00
Sébastien Han 0bdbb944bf ceph: simplify external mode setup
The preparation script does not require any manual inputs anymore. The
cluster/examples/kubernetes/ceph/create-external-cluster-resources.sh
will create and set the env variable needed for the import script.
Hopefully this will reduce the friction in some installation, especially
for the mon ports.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-05-11 16:08:46 +02:00
Shachar Sharon 2a3cfc693b ceph: allow setting primary-affinity to osd
A user with unbalanced system (e.g., some OSDs use a partition which
resides on a device along-side OS partition) may want to have a
fine-grained control on the overall load of specific OSD. While osd
weight affects write-load, primary-affinity affects read-load.

This patch allow user to define osd's primary-affinity (similar to
'DeviceClass' and 'InitialWeight'), which is sent to the osd (via mon)
as part of deployment sequence, using 'ceph osd primary-affinity' command.

Valid values are within the range [0.0, 1.0). If not set, default value
is 1.

ROOK issue: https://github.com/rook/rook/issues/7773

Signed-off-by: Shachar Sharon <ssharon@redhat.com>
2021-05-06 17:46:15 +03:00
Travis Nielsen 583c94be38 Merge pull request #7829 from BlaineEXE/ceph-image-to-16.2.2
ceph: update images to v16.2.2
2021-05-05 12:57:49 -06:00
Travis Nielsen af1d728ba6 Merge pull request #7801 from sp98/mon-failover-on-drain
ceph: retry once before mon failover if mon pod is unscheduled
2021-05-05 12:39:05 -06:00
Santosh Pillai f38829b57d ceph: retry once before mon failover if mon pod is unscheduled
Events like node drain can take more than 10-15 minutes. If the node is not update the default monTimeOut of 10 minutes, then rook will attempt to fail over the mon. This failover won't work as the node is still down.
This PR retries once before the mon failover if the mon pod is not scheduled

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-05-05 23:32:27 +05:30
Blaine Gardner 5245ceaf06 ceph: update images to v16.2.2
Ceph v16.2.2 is released with dated tag v16.2.2-20210505. Update Ceph
images to match with or without date as appropriate.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-05-05 11:35:34 -06:00
Rakshith R 7987e1b8a2 ceph: update snapshot APIs from v1beta1 to v1
This commit updates external-snapshotter version to
v4.0.0 which supports snapshots v1.
Rook now defaults to enabling RBD and CephFS snapshotter
for K8s >= v1.17 and disabling it for K8s <= v1.16.
Supporting changes in documents and examples yaml files
are made.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-05 21:04:10 +05:30