Commit Graph
13292 Commits
Author SHA1 Message Date
Travis Nielsen 9655e3a21e Merge pull request #8927 from synarete/aarch64-graviton
build: support build of rook on aarch64 (AWS/graviton)
2021-10-20 07:55:20 -06:00
Travis Nielsen 8f639c61de Merge pull request #8938 from subhamkrai/update-minikube
ci: update minikube and kubernetes version
2021-10-20 07:52:06 -06:00
Sébastien Han de0ba669fe Merge pull request #9009 from PixelJonas/patch-quotation-marks
ceph: use normal quotation marks for comment
2021-10-20 13:52:55 +02:00
subhamkrai 0150966024 ceph: remove ceph nautilus, ceph octopus to default
since rook 1.8, ceph nautilus no longer supported,
ceph octopus will be the minimum ceph version.

Closes: https://github.com/rook/rook/issues/7908
Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 14:45:12 +05:30
subhamkrai 6be9071a22 ci: update minikube.sh script to use minikube command
using `docker-env` command to copy image giving error
`X Exiting due to ENV_DRIVER_CONFLICT: 'none' driver does not support 'minikube docker-env' command`
so, using `minikube image load <image>` commmand to copy
image.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 14:26:50 +05:30
PixelJonas 8733bf6272 ceph: use normal quotation marks for comment
replaces all occurences of lduo/rduo quotation marks to make
sure that using the snippets in a k8s manifest will work

This fixes an issue with ArgoCD not being able
to apply `common.yaml` because of an encoding issue

Signed-off-by: PixelJonas <jonas@janz.digital>
2021-10-20 10:09:22 +02:00
Sébastien Han d7213c086b Merge pull request #9008 from subhamkrai/fix-ci
ci: clean validate_cluster.sh script
2021-10-20 09:55:44 +02:00
Sébastien Han c512e5f335 Merge pull request #9006 from BlaineEXE/osd-purge-in-cluster-namespace
docs: move purge osd to cluster namespace
2021-10-20 09:52:50 +02:00
subhamkrai e49cdf0a8d ci: clean validate_cluster.sh script
removing `trap display_status SIGINT ERR` command
from the file as `display_status` func has been removed.

Closes: https://github.com/rook/rook/issues/9004
Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 10:43:50 +05:30
subhamkrai 6d0a06365f ci: update minikube and kubernetes version
Updating minikube version to `v1.23.2` and
k8s version to `v1.22.2`

Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 10:04:31 +05:30
Yug Gupta e584efa966 docs: add a document to set-up rbd mirroring
The document tracks the steps which are required
to set-up rbd mirroring on clusters.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-10-20 09:11:33 +05:30
Blaine Gardner 01e2feaef5 ceph: get rid of dynamic clientset
Stop using the dynamic clientset in favor of the controller-runtime
clientset.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-19 17:05:38 -06:00
Blaine Gardner d43327de2a docs: move purge osd to cluster namespace
The rook-ceph-purge-osd job should be run for a particular CephCluster
and should therefore be namespaced to the cluster.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-19 13:23:42 -06:00
Travis Nielsen d01ea3a0b3 Merge pull request #8900 from BlaineEXE/create-library-chart
helm: create shared library chart
2021-10-19 11:22:23 -06:00
Sébastien Han 4bd179a973 Merge pull request #8767 from leseb/secret-cmd
osd: use rook binary to fetch key
2021-10-19 18:41:12 +02:00
Sébastien Han 4017f94464 osd: use rook binary to fetch key encryption key
When deploying a cluster-wde encrypted cluster we now use the rook
binary to execute some code to fetch the key encryption key.

Using cURL all the time to fetch the key has its limitations. The
incoming integration with Kubernetes Authentication through service
accounts is leading the usage of cURL to its end.
The logic is really complex and prone to errors. Re-implementing the
Go logic into Bash is not a viable option.

Also, using the lib in a binary allows us to keep a consistent behavior
throughout the life cycle of our code.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-19 17:29:25 +02:00
Blaine Gardner d9146f6f95 helm: create shared library chart
Create a Helm library chart to share content between the Operator Helm
chart (rook-ceph) and the CephCluster (etc.) Helm chart
(rook-ceph-cluster).

This will ensure that when resources related to the CephCluster change,
notably RBAC resources, they will change for both charts.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-18 12:33:47 -06:00
Blaine Gardner d2a03bb2b3 Merge pull request #8976 from BlaineEXE/try-fix-flaky-ci-osd-provision
test: get more partition info setting up ci disk
2021-10-18 12:33:24 -06:00
Blaine Gardner 65619c2677 test: get more partition info setting up ci disk
Add some commands to get more partition info when setting up the GH
action runner's disk for use in integration tests. This will both aid in
debugging and may "jog" the system such that it will no longer need to
reload the partition info when running the OSD prepare job.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-18 11:29:13 -06:00
Sébastien Han df2b6faa96 Merge pull request #8995 from leseb/no-error-merge
ceph: only merge stderr on error
2021-10-18 18:28:04 +02:00
Travis Nielsen 034018a8ae Merge pull request #8984 from travisn/support-arbiter-failover
ceph: Enable mon failover for the arbiter in stretch mode
2021-10-18 09:48:54 -06:00
Sébastien Han 0e41e36ade ceph: only merge stderr on error
Previously we were merging the stderr even if it was empty, leading to
unmarshall errors.
The error simulation was done here
https://play.golang.org/p/Sk2yw9GUWNu.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-18 17:18:57 +02:00
Travis Nielsen 12689bd119 ceph: enable mon failover for the arbiter in stretch mode
Prior to ceph v16.2.7 the failover of the arbiter mon was
not supported. Now the new tiebreaker mon can be set during
the failover event and provide more dynamic stability to
the mon quorum if another node is available in the arbiter
zone.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-10-18 08:58:25 -06:00
Sébastien Han ceae7604ac Merge pull request #8990 from Madhu-1/correct-cephfs
csi: fix comment for the provisioner and clusterID
2021-10-18 09:45:24 +02:00
Sébastien Han 748a98de89 Merge pull request #8988 from BlaineEXE/do-not-use-head
test: do not use head because of pipe errors
2021-10-18 09:30:58 +02:00
Madhu Rajanna 83cd8cb300 csi: fix comment for the provisioner and clusterID
fixed provisioner and clusterID comment to match
the correct namespace.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-10-18 10:39:13 +05:30
Blaine Gardner ba7745d4af test: do not use head because of pipe errors
The `head` command exits once it has output which can result in a
SIGPIPE error if the command piping its output to head hasn't yet
finished. Use `awk 'FNR <= 1'` instead, which waits on the input pipe to
close before it exits.

See here for more info:
https://unix.stackexchange.com/a/256047

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-15 10:14:41 -06:00
Sébastien Han b0c5f610c4 Merge pull request #8981 from leseb/fix-prepare-list
ci: fix prepare pod wrong selector
2021-10-15 17:58:19 +02:00
Sébastien Han efa60b44c8 ci: only validate osds for mirror test
The Filesystem is deployed later in the run so at this point we only
need to validate the OSDs are running. This was failing since we were
validating the cephfs pool which is not present yet.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-15 10:36:15 +02:00
Sébastien Han 73970a56c5 ci: fix prepare pod wrong selector
We missed the `app=` in the selector.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-15 10:36:15 +02:00
Sébastien Han ecd75a2a46 Merge pull request #8985 from aruniiird/fix-CephMgrIsAbsent-n-CephMgrIsMissingReplicas-queries
ceph: fixing the queries for alerts 'CephMgrIsAbsent' and 'CephMgrIsMissingReplicas'
2021-10-15 10:22:42 +02:00
Arun Kumar Mohan cfa2c2dd38 ceph: fixing the queries for alerts 'CephMgrIsAbsent' and 'CephMgrIsMissingReplicas'
CephMgrIsAbsent
----------------
This alert initially had the following query

absent(up{job="rook-ceph-mgr"})

which will fire when the 'up' query is not present, but had two flows
  a. it will not be fired if 'up' provides a result with ZERO value
  b. it will not give any fields in the metric, so 'namespace' was missing

when the above query was replaced with the following,

up{job="rook-ceph-mgr"} == 0

query had the following shortage
  a. whenever mgr pod is completely down (like 'replicas' set to ZERO
and 'mgr' is not coming up), 'up' query will not give any result.

Thus we had to combine both the queries to get results in both the scenarios.

CephMgrIsMissingReplicas
------------------------
This query previously was,

sum(up{job="rook-ceph-mgr"}) < 1

had the same structure as the above (Absent) query, but it's
intention was to check the no: of 'replicas' count for ceph mgr.
Now it is changed to a kube query which handles the replicas count.

Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
2021-10-15 13:16:45 +05:30
Blaine Gardner 9593bdd471 Merge pull request #8982 from BlaineEXE/fix-prepare-pod-log-collection
test: fix prepare pod log collection in CI
2021-10-14 11:20:28 -06:00
Blaine Gardner f287df70ee test: fix prepare pod log collection in CI
In the CI tests that use `validate_cluster.sh display_status` to gather
logs, the prepare pod log collection failed. Fix this.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-14 11:15:59 -06:00
Sébastien Han 07986bcf8c Merge pull request #8867 from leseb/kv-auto-detect-with-certs
ceph: fix kms auto-detection when full TLS
2021-10-14 11:30:13 +02:00
Blaine Gardner 6f39062284 Merge pull request #8962 from BlaineEXE/realm-make-idempotent
rgw: make CephObjectRealm controller idempotent
2021-10-13 12:57:33 -06:00
Blaine Gardner 34a8b42e97 test: try to un-flake multi-cluster-mirror test
Try to un-flake the multi-cluster-mirror test that keeps failing on this
PR.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-13 11:06:51 -06:00
Blaine Gardner 65c4972038 rgw: make CephObjectRealm controller idempotent
The CephObjectRealm controller would fail all subsequent reconciles if
the first reconcile created the Kubernetes Secret containing the access
keys for the realm but where the radosgw-admin command failed to create
the realm. This was the only idempotency issue found after reviewing the
CephObjectRealm controller.

Resolves #8954

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-13 11:06:51 -06:00
Blaine Gardner 79c191004e Merge pull request #8971 from leseb/more-output
ceph: print the c-v output when inventory command fails
2021-10-13 10:41:37 -06:00
Sébastien Han 5e3ecaadbb Merge pull request #8966 from leseb/rm-compression-mode
ceph: remove default value for pool compression
2021-10-13 18:13:24 +02:00
Sébastien Han 61afadd97e ceph: fix kms auto-detection when full TLS
When TLS is used and includes a caert, client key/cert, we need to copy
the content of the secret to a file in the operator's container
filesystem so that we can build the TLS config and thus the HTTP Client,
which reads those files.
Also, removing the files after each API call so they don't persist on
the filesystem forever.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-13 18:12:58 +02:00
Sébastien Han 1e9d24ac77 ceph: print the c-v output when inventory command fails
Hopefully, this will give us more hints when a failure occurs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-13 18:08:06 +02:00
Sébastien Han 28cc6f5514 ceph: remove default value for pool compression
Using a default value for CompressionMode to none effectively overrides
any values for Parameters. It is deprecated but still takes precedence.
Which means that in its previous form, Parameters was always ignored
since CompressionMode was always set to none when empty.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-13 17:06:22 +02:00
Travis Nielsen b4635f8b6b Merge pull request #8965 from Rakshith-R/add-affinity
ceph: add affinity to csi version check job
2021-10-13 07:42:41 -06:00
Sébastien Han 720a96e0cd Merge pull request #8166 from BlaineEXE/tidy-apply-multus
core: clean up ApplyMultus()
2021-10-13 11:58:34 +02:00
Rakshith R c1ef189b91 ceph: apply csi provisioner node-affinity to csi version check job
This commit makes sure csi provisioner node-affinity is applied
to the csi version check job as well, similar to the how
the existing csi provisioner toleration is applied to the job.

Fixes: #8323

Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-13 15:18:23 +05:30
Sébastien Han e33d905b59 Merge pull request #8501 from josephsawaya/ceph-nfs-remove-rados
ceph: remove RADOS options from CephNFS and use .nfs pool
2021-10-13 10:41:49 +02:00
Sébastien Han 7590bf0957 Merge pull request #8963 from travisn/daily-arm-tests
tests: Run the daily arm tests against default tag instead of local-build
2021-10-13 10:30:30 +02:00
Sébastien Han 860e818512 Merge pull request #8947 from GowthamShanmugam/1970354
Handle empty ceph_version in ceph_mon_metadata to avoid raising misleading alert
2021-10-13 10:25:18 +02:00
Sébastien Han 58f85e0722 Merge pull request #8959 from renan-campos/issue_8958
ceph: csi controller to use constants to fetch operator configmap
2021-10-13 09:29:34 +02:00