Commit Graph
7961 Commits
Author SHA1 Message Date
Sébastien Han baaea4a1ea Merge pull request #7604 from leseb/cephfs-mirror-peer-config
ceph: add filesystem mirror peers configuration
2021-07-05 10:36:30 +02:00
Sébastien Han 6c408e04dc Merge pull request #8253 from travisn/stable-helm-tests
ceph: Run helm tests without admission controller
2021-07-05 10:30:19 +02:00
Sébastien Han 6be28011a4 Merge pull request #8208 from thotz/enabledebugforadminOps
ceph: enable debug for adminops client if the rook loglevel <= debug
2021-07-05 10:22:50 +02:00
Sébastien Han d59ae25d2b Merge pull request #8215 from sp98/disable-mirroring
ceph: ability to disable pool mirroring
2021-07-05 10:22:24 +02:00
Jiffin Tony Thottan e4a9b4bac1 ceph: fetch rgw port from cephobjectstore crd for obc
Currently `setObjectStorePort()` queries port from RGW service and
picks up the first port from the list, other places of the code
prefers for TLS, if it is enabled so for OBC's we end up having
`https` endpoint with non-secure port.

Fixes: #8141
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-07-05 10:55:05 +05:30
Henry Zhang a5227bf126 ceph: network configuration should be optional in Helm chart
adds a default empty dict for the network when deciding whether toolbox
should also be on the host network to catch the case when the entire
`network` block is empty

Signed-off-by: Henry Zhang <me@henry.dev>
2021-07-04 01:05:08 -07:00
Travis Nielsen 7cec49b2d8 ceph: run helm tests without admission controller
The helm tests frequently hit an issue with the admission controller.
The admission controller is also enabled in other test suites, so
it seems safe enough to disable for the helm test suite.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-02 15:28:57 -06:00
Travis Nielsen 4b1cf51ef1 ceph: scaling down nfs deployment was failing
When scaling down the nfs deployment gracefully, the wrong error code was checked,
resulting in a failed scaling down even though the condition was expected.
Now we check properly if the deployment was removed properly.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-02 11:52:03 -06:00
Travis Nielsen 0a09d844ad Merge pull request #8237 from travisn/retry-after-cancel
ceph: Retry reconcile immediately after cancellation
2021-07-02 10:11:24 -06:00
Travis Nielsen 8778e3cab5 Merge pull request #8238 from subhamkrai/oom-killed
ceph: operator pod killed due to resource limit
2021-07-02 10:11:04 -06:00
Travis Nielsen fc5402b3e5 Merge pull request #8236 from travisn/debug-log-k8s-version
ceph: Debug log the k8s version instead of info
2021-07-02 09:23:39 -06:00
subhamkraiandSébastien Han 4241ce7e66 ceph: operator pod killed due to resource limit
when operator pod resources are set(which are
default in case of helm), operator pod are
killed due to using concurrency while creating
object store.

this commit checks if operator resources are set
or not. if set then we'll *not* create object store
in concurrency or if not set then we'll create in
concurrency.

Closes: https://github.com/rook/rook/issues/8149
Co-authored-by: Sébastien Han <seb@redhat.com>
Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-02 20:18:58 +05:30
Travis Nielsen c39c1c7ddf ceph: retry reconcile immediately after cancellation
If the reconcile is cancelled due to a CR update, we want to retry the next
reconcile immediately rather than wait for the exponential backoff timeout
if the reconcile was already failing. The wait can easily be minutes
if the reconcile was in this state, which makes it appear the operator
is ignoring the request to start a new reconcile.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-02 08:23:25 -06:00
Travis Nielsen 889a082734 Merge pull request #8029 from Yuggupta27/update-csidriver
ceph: update CSIDriver object from betav1 to v1
2021-07-01 23:46:36 -06:00
Santosh Pillai 05b0ae09bb ceph: disable mirroring
The PR disables mirroring on a pool when PoolSpec.Mirroring.Enabled
is set to false
- disable mirroring on the pool for `Mirroring.Mode == pool`
- Add warning for the user to disable mirroring manually for `Mirroring.Mode == image`
- Stop the mirroring health checker goroutine
- Reset the mirroring health check status in the CR.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-07-02 10:35:38 +05:30
Yug d513dc4bc5 ceph: update CSIDriver object from betav1 to v1
Update the CSIDriver object to v1 from betav1 as it will be
deprecated in v1.19+ and become unavailable in v1.22+

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-02 10:19:53 +05:30
Travis Nielsen aaa939f686 ceph: debug log the k8s version instead of info
The info log is rather verbose with all the filtering of the K8s version
like the following:
returning version v1.19.0 instead of v1.19.0+43983cd

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-01 17:09:20 -06:00
Travis Nielsen af18727359 Merge pull request #8227 from leseb/silence-errors
ceph: silence harmless errors
2021-07-01 10:12:32 -06:00
Sébastien Han b578f916e7 ceph: add fs mirror config
Similarly to block volume replication, Ceph is capable of replicating the
content of a Ceph Filesystem from one cluster to another.
For this, during the 1.6 cycle, we introduced a new CRD called
CephFilesystemMirror which effectively deploys a cephfs-mirror daemon.
However, configuring peers to enable replication between two clusters
had to be done manually.
Also various bug fix made it in Ceph eventually and the minimum required
version for this to work is to run on Ceph Pacific 16.2.5 at least.

So the automatic configuration of Ceph Filesystem peers is now possible.

By editing the CephFilesystem CRD, you can now turn on mirroring:

```yaml
  mirroring:
    enabled: false
    # list of Kubernetes Secrets containing the peer token
    # for more details see: https://docs.ceph.com/en/latest/dev/cephfs-mirroring/#bootstrap-peers
    peers:
      secretNames:
        - secondary-cluster-peer
```

Also, the mirroring status is displayed in the CR status:

```
status:
  info:
    fsMirrorBootstrapPeerSecretName: fs-peer-token-myfs
  mirroringStatus:
    daemonsStatus:
    - daemon_id: 4186
      filesystems:
      - filesystem_id: 2
        name: myfs
    lastChecked: "2021-07-01T14:16:29Z"
  phase: Ready
  snapshotScheduleStatus:
    lastChecked: "2021-07-01T14:16:29Z"
    snapshotSchedules:
    - fs: myfs
      path: /
      rel_path: /
      retention: {}
      schedule: 24h
```

Closes: https://github.com/rook/rook/issues/7063
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-01 17:35:19 +02:00
Travis Nielsen 5dde16fcf7 Merge pull request #8180 from subhamkrai/mds-resource-limit
ceph: give resource request preference for mds_cache_memory_limit
2021-07-01 09:08:17 -06:00
subhamkrai d41878cb9e ceph: give resource request preference for mds_cache_memory_limit
earlier, for mds, when resource limit is not defined it has
default limit of `"mds_cache_memory_limit": "4294967296"`
even if resouce.request is defined.

Now, resource request will be given preference when both
request and limit is defined and if only limit is defined
it will be applied or otherwise.

Closes: https://github.com/rook/rook/issues/8143
Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-01 19:54:07 +05:30
Sébastien Han 8c589a3785 ceph: silence harmless errors
Let's not look at misleading errors if the operator is still
initializing.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-01 15:37:08 +02:00
Jiffin Tony Thottan 9e3cf68d04 test: ci test for TLS objectstore
Extend the object store smoke test to include TLS configurations.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-07-01 18:50:27 +05:30
Sébastien Han b68cac34cb Merge pull request #8222 from sp98/update-block-pool-channels
ceph: update blockPoolChannel before starting the mirror monitoring
2021-07-01 14:10:17 +02:00
Santosh Pillai 8d48a9e10a ceph: update blockPoolChannel before starting the mirror monitoring
update blockPoolChannel monitoringRunning field to `true` before
starting the goroutine to monitor mirroring

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-07-01 15:27:37 +05:30
Sébastien Han 048b75c254 Merge pull request #8226 from leseb/fix-secetion-pending-note
docs: add news feature in the features section
2021-07-01 11:38:27 +02:00
Sébastien Han 647cccf1d0 docs: add news feature in the features section
New features were in the "breaking changes" section.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-01 10:57:48 +02:00
Sébastien Han 2baac8372f Merge pull request #8172 from sp98/hybrid-storage-pools
ceph: add support for hybrid Storage Pools
2021-07-01 09:15:16 +02:00
Jiffin Tony Thottan 560f44b07e ceph: enable debug for adminops client if the rook loglevel <= debug
The debug for adminOps client can be enabled by setting `Debug` flag.
In this PR, it is enabled for OBC and cephobjectstore not healthchecker.
Also add similar changes while calling `NewS3Agent` in the bucket
provisioner code path.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-07-01 11:36:51 +05:30
Santosh Pillai 4041868e2e ceph: hybrid Storage Pools
Creates hybrid crush rule for choosing Primary OSD for
high performing SSD devices and remaining OSD for low performance HDD devices.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2021-07-01 11:16:54 +05:30
Blaine Gardner 037945aabc Merge pull request #8112 from BlaineEXE/dependencies-2-son-of-dependencies
ceph: block delete object store when buckets and/or users exist
2021-06-30 12:45:58 -06:00
Travis Nielsen 03f6dddcfd Merge pull request #8170 from subhamkrai/update-helm-version
ci: remove read access from kube config
2021-06-30 11:29:35 -06:00
Blaine Gardner 53459bbb44 Merge pull request #8195 from BlaineEXE/ignore-atari-partitions
ceph: ignore atari partitions when provisioning OSDs
2021-06-30 11:26:56 -06:00
Blaine Gardner 7f516b9e3d ceph: ignore atari partitions when scanning disks
Ceph bluestore raw disks can sometimes appear as though they have Atari
(AHDI) partitions on them. If a disk has Atari partitions, we just
ignore them as though they don't exist. This should be a safe assumption
since the hardware was last manufactured in 1992 and likely can't run
Kubernetes.

If we don't ignore the Atari partitions, Rook can create a new OSD on a
disk that is already running an OSD, corrupting the first and possibly
also the latest OSD. This can happen an arbitrary number of times per
disk.

More info: https://github.com/rook/rook/issues/7940

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-30 09:42:33 -06:00
Travis Nielsen c77a9a9aa8 Merge pull request #8219 from leseb/bz-1964467-upstream
ceph: do not require rgw 'instances' spec
2021-06-30 07:24:00 -06:00
Sébastien Han e25e0e8103 ceph: do not require rgw 'instances' spec
When the cluster is external we don't have any instances of rgw running
so let's not have this as a required field in the spec.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-30 11:24:09 +02:00
Sébastien Han d3ff3b4c50 Merge pull request #8092 from leseb/external-mode-rgw-ssl
ceph: add tls support to external rgw endpoint
2021-06-30 09:30:45 +02:00
Satoru Takeuchi 644039bb9a Merge pull request #8213 from leseb/retry-build-on-network-failure
ci: retry build on network failures
2021-06-30 09:12:50 +09:00
Blaine Gardner c22f545ebf ceph: block delete object store when buckets exist
Block deletion of CephObjectStore resources when buckets exist in the
object store.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-29 14:31:39 -06:00
Sébastien Han d645152743 ceph: ability pass rgw certificate to external mode
We now have a new option called --rgw-tls-cert-path to specify a
certificate path for the rgw external mode.

Example output:

```
[root@rook-ceph-tools-78cdfd976c-25942 /]# python3 /etc/ceph/create-external-cluster-resources.py --rbd-data-pool-name device_health_metrics --monitoring-endpoint 172.17.0.11 --rgw-tls-cert-path my-cert --rgw-endpoint 10.108.182.47:80
[{"name": "rook-ceph-mon-endpoints", "kind": "ConfigMap", "data": {"data": "a=10.100.223.211:6789", "maxMonId": "0", "mapping": "{}"}}, {"name": "rook-ceph-mon", "kind": "Secret", "data": {"admin-secret": "admin-secret", "fsid": "5cd5684c-8b8b-4d59-a98a-525f63ae69a3", "mon-secret": "mon-secret"}}, {"name": "rook-ceph-operator-creds", "kind": "Secret", "data": {"userID": "client.healthchecker", "userKey": "AQBnx9BgnH9kDxAAb6T0nmMpetfmwNw5gKgA2w=="}}, {"name": "rook-csi-rbd-node", "kind": "Secret", "data": {"userID": "csi-rbd-node", "userKey": "AQAe0bhgVvahJxAAGJLuP/Tajh2fz6ExqxYT/A=="}}, {"name": "ceph-rbd", "kind": "StorageClass", "data": {"pool": "device_health_metrics"}}, {"name": "monitoring-endpoint", "kind": "CephCluster", "data": {"MonitoringEndpoint": "172.17.0.11", "MonitoringPort": "9283"}}, {"name": "rook-csi-rbd-provisioner", "kind": "Secret", "data": {"userID": "csi-rbd-provisioner", "userKey": "AQAe0bhgkpHWGhAAJNPrBBD5OfbQvGRAYUrbCg=="}}, {"name": "ceph-rgw-tls-cert", "kind": "Secret", "data": {"cert": "-----BEGIN CERTIFICATE-----\nMIIFODCCAyCgAwIBAgIRAJW+FqD3LkbxezmCcvqLzZYwDQYJKoZIhvcNAQEFBQAw\nNzEUMBIGA1UECgwLVGVsaWFTb25lcmExHzAdBgNVBAMMFlRlbGlhU29uZXJhIFJv\nb3QgQ0EgdjEwHhcNMDcxMDE4MTIwMDUwWhcNMzIxMDE4MTIwMDUwWjA3MRQwEgYD\nVQQKDAtUZWxpYVNvbmVyYTEfMB0GA1UEAwwWVGVsaWFTb25lcmEgUm9vdCBDQSB2\nMTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMK+6yfwIaPzaSZVfp3F\nVRaRXP3vIb9TgHot0pGMYzHw7CTww6XScnwQbfQ3t+XmfHnqjLWCi65ItqwA3GV1\n7CpNX8GH9SBlK4GoRz6JI5UwFpB/6FcHSOcZrr9FZ7E3GwYq/t75rH2D+1665I+X\nZ75Ljo1kB1c4VWk0Nj0TSO9P4tNmHqTPGrdeNjPUtAa9GAH9d4RQAEX1jF3oI7x+\n/jXh7VB7qTCNGdMJjmhnXb88lxhTuylixcpecsHHltTbLaC0H2kD7OriUPEMPPCs\n81Mt8Bz17Ww5OXOAFshSsCPN4D7c3TxHoLs1iuKYaIu+5b9y7tL6pe0S7fyYGKkm\ndtwoSxAgHNN/Fnct7W+A90m7UwW7XWjH1Mh1Fj+JWov3F0fUTPHSiXk+TT2YqGHe\nOh7S+F4D4MHJHIzTjU3TlTazN19jY5szFPAtJmtTfImMMsJu7D0hADnJoWjiUIMu\nsDor8zagrC/kb2HCUQk5PotTubtn2txTuXZZNp1D5SDgPTJghSJRt8czu90VL6R4\npgd7gUY2BIbdeTXHlSw7sKMXNeVzH7RcWe/a6hBle3rQf5+ztCo3O3CLm1u5K7fs\nslESl1MpWtTwEhDcTwK7EpIvYtQ/aUN8Ddb8WHUBiJ1YFkveupD/RwGJBmr2X7KQ\narMCpgKIv7NHfirZ1fpoeDVNAgMBAAGjPzA9MA8GA1UdEwEB/wQFMAMBAf8wCwYD\nVR0PBAQDAgEGMB0GA1UdDgQWBBTwj1k4ALP1j5qWDNXr+nuqF+gTEjANBgkqhkiG\n9w0BAQUFAAOCAgEAvuRcYk4k9AwI//DTDGjkk0kiP0Qnb7tt3oNmzqjMDfz1mgbl\ndxSR651Be5kqhOX//CHBXfDkH1e3damhXwIm/9fH907eT/j3HEbAek9ALCI18Bmx\n0GtnLLCo4MBANzX2hFxc469CeP6nyQ1Q6g2EdvZR74NTxnr/DlZJLo961gzmJ1Tj\nTQpgcmLNkQfWpb/ImWvtxBnmq0wROMVvMeJuScg/doAmAyYp4Db29iBT4xdwNBed\nY2gea+zDTYa4EzAvXUYNR0PVG6pZDrlcjQZIrXSHX8f8MVRBE+LHIQ6e4B4N4cB7\nQ4WQxYpYxmUKeFfyxiMPAdkgS94P+5KFdSpcc41teyWRyu5FrgZLAMzTsVlQ2jqI\nOylDRl6XK1TOU2+NSueW+r9xDkKLfP0ooNBIytrEgUy7onOTJsjrDNYmiLbAJM+7\nvVvrdX3pCI6GMyx5dwlppYn8s3CQh3aP0yK7Qs69cwsgJirQmz1wHiRszYd2qReW\nt88NkvuOGKmYSdGe/mBEciG5Ge3C9THxOUiIkCR1VBatzvT4aRRkOfujuLpwQMcn\nHL/EVlP6Y2XQ8xwOFvVrhlhNGNTkDY6lnVuR3HYkUD/GKvvZt5y11ubQ2egZixVx\nSK236thZiNSQvxaz2emsWWFUyBy6ysHK4bkgTI86k4mloMy/0/Z1pHWWbVY=\n-----END CERTIFICATE-----"}}]
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-29 09:46:50 +02:00
Sébastien Han 9738131010 ci: retry on network build failures
We have seen cases where the build might fail due to a small network
hiccup so let's retry up to 3 times if this is the case.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-29 09:18:56 +02:00
Sébastien Han e6734304cd ci: format helper using standard 2 spaces indent
Bash common indent is 2 spaces so let's use it.
See: https://google.github.io/styleguide/shellguide.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-29 09:18:56 +02:00
subhamkrai d3190c3755 ci: remove read access from kube config
this commit remove read access from kube
config file for group and others.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-06-28 21:37:11 +05:30
Travis Nielsen e1a6333f79 Merge pull request #8207 from subhamkrai/revert-helm-changes
ci: downgrade minikube version from helm suite
2021-06-28 07:55:55 -06:00
Satoru Takeuchi c37f992830 Merge pull request #8206 from cybozu-go/ceph-fix-livenessprobe-when-config
ceph: fix overwriting shared livenessProbe
2021-06-28 20:46:36 +09:00
Yuji Ito d7c46d70d3 ceph: fix overwriting shared livenessProbe
When the Operator set the livenessProbe of the POD, it overwrote the contents of the shared
desiredProbe. Therefore, the wrongly configured livenessProbe was applied to other PODs,
resulting in a CrashLoop. In this PR, I created another instance to avoid overwriting the
desiredProbe.

Closes: https://github.com/rook/rook/issues/8196
Signed-off-by: Yuji Ito <llamerada.jp@gmail.com>
2021-06-28 09:54:33 +00:00
subhamkrai 279fbb0b93 ci: downgrade minikube version from helm suite
this commit downgrades minikube version
from release test for helm suite as
test are failing.

closes: https://github.com/rook/rook/issues/8202
Signed-off-by: subhamkrai <srai@redhat.com>
2021-06-28 10:51:56 +05:30
Satoru Takeuchi 66609f01b4 Merge pull request #8203 from sebastian-philipp/stale-disable-forks
ci: disable stale workflow on forks
2021-06-28 09:25:05 +09:00
Sebastian Wagner a411aa015f ci: disable stale workflow on forks
Running the stale workflow is a waste of resources.

Signed-off-by: Sebastian Wagner <sewagner@redhat.com>
2021-06-27 16:59:47 +02:00
Travis Nielsen 06a889fd46 Merge pull request #8199 from travisn/topology-spread-osd-prepare
ceph: Update cluster-on-pvc example for proper OSD scheduling
2021-06-25 13:33:30 -06:00