Remove the unspecific 'ceph' tag, and add a few other specific tags:
block (for block storage that is not pool-specific)
file (for file storage that is not mds-specific)
monitoring (for prometheus, etc.)
nfs (for CephNFS and nfs-ganesha)
object (for object storage that is not rgw-specific)
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When deploying a cluster-wde encrypted cluster we now use the rook
binary to execute some code to fetch the key encryption key.
Using cURL all the time to fetch the key has its limitations. The
incoming integration with Kubernetes Authentication through service
accounts is leading the usage of cURL to its end.
The logic is really complex and prone to errors. Re-implementing the
Go logic into Bash is not a viable option.
Also, using the lib in a binary allows us to keep a consistent behavior
throughout the life cycle of our code.
Signed-off-by: Sébastien Han <seb@redhat.com>
Create a Helm library chart to share content between the Operator Helm
chart (rook-ceph) and the CephCluster (etc.) Helm chart
(rook-ceph-cluster).
This will ensure that when resources related to the CephCluster change,
notably RBAC resources, they will change for both charts.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Add some commands to get more partition info when setting up the GH
action runner's disk for use in integration tests. This will both aid in
debugging and may "jog" the system such that it will no longer need to
reload the partition info when running the OSD prepare job.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Previously we were merging the stderr even if it was empty, leading to
unmarshall errors.
The error simulation was done here
https://play.golang.org/p/Sk2yw9GUWNu.
Signed-off-by: Sébastien Han <seb@redhat.com>
Prior to ceph v16.2.7 the failover of the arbiter mon was
not supported. Now the new tiebreaker mon can be set during
the failover event and provide more dynamic stability to
the mon quorum if another node is available in the arbiter
zone.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The `head` command exits once it has output which can result in a
SIGPIPE error if the command piping its output to head hasn't yet
finished. Use `awk 'FNR <= 1'` instead, which waits on the input pipe to
close before it exits.
See here for more info:
https://unix.stackexchange.com/a/256047
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The Filesystem is deployed later in the run so at this point we only
need to validate the OSDs are running. This was failing since we were
validating the cephfs pool which is not present yet.
Signed-off-by: Sébastien Han <seb@redhat.com>
CephMgrIsAbsent
----------------
This alert initially had the following query
absent(up{job="rook-ceph-mgr"})
which will fire when the 'up' query is not present, but had two flows
a. it will not be fired if 'up' provides a result with ZERO value
b. it will not give any fields in the metric, so 'namespace' was missing
when the above query was replaced with the following,
up{job="rook-ceph-mgr"} == 0
query had the following shortage
a. whenever mgr pod is completely down (like 'replicas' set to ZERO
and 'mgr' is not coming up), 'up' query will not give any result.
Thus we had to combine both the queries to get results in both the scenarios.
CephMgrIsMissingReplicas
------------------------
This query previously was,
sum(up{job="rook-ceph-mgr"}) < 1
had the same structure as the above (Absent) query, but it's
intention was to check the no: of 'replicas' count for ceph mgr.
Now it is changed to a kube query which handles the replicas count.
Signed-off-by: Arun Kumar Mohan <amohan@redhat.com>
In the CI tests that use `validate_cluster.sh display_status` to gather
logs, the prepare pod log collection failed. Fix this.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The CephObjectRealm controller would fail all subsequent reconciles if
the first reconcile created the Kubernetes Secret containing the access
keys for the realm but where the radosgw-admin command failed to create
the realm. This was the only idempotency issue found after reviewing the
CephObjectRealm controller.
Resolves#8954
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When TLS is used and includes a caert, client key/cert, we need to copy
the content of the secret to a file in the operator's container
filesystem so that we can build the TLS config and thus the HTTP Client,
which reads those files.
Also, removing the files after each API call so they don't persist on
the filesystem forever.
Signed-off-by: Sébastien Han <seb@redhat.com>
Using a default value for CompressionMode to none effectively overrides
any values for Parameters. It is deprecated but still takes precedence.
Which means that in its previous form, Parameters was always ignored
since CompressionMode was always set to none when empty.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit makes sure csi provisioner node-affinity is applied
to the csi version check job as well, similar to the how
the existing csi provisioner toleration is applied to the job.
Fixes: #8323
Signed-off-by: Rakshith R <rar@redhat.com>
The daily arm test suite is failing due to the new local-build
image tag. Instead of waiting for the arm build to complete,
we can just pick up the latest tag from the same branch
that was already pushed to dockerhub and no need to build
again.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The CSI controller loop watches ConfigMaps and CephClusters. When a
CephCluster triggers the reconcile loop, the Get call for the ConfigMap
will fail since the name in the request object is that of the
CephCluster. Setting the name and namespace to the constants for the
ConfigMap name and the operator namespace resolves this.
Closes: https://github.com/rook/rook/issues/8958
Signed-off-by: Renan Campos <rcampos@redhat.com>
earlier push build action was not triggered due
to github action limitation.
```
An action in a workflow run can’t trigger a new workflow run.
```
so now, we'll use user personal toke to create tag so that push
build action pick the user not github action who created the tag.
Closes: https://github.com/rook/rook/issues/8580
Signed-off-by: subhamkrai <srai@redhat.com>
When generating the offline image list, the threaded crossbuild can
try to generate the image list from muliple jobs at once, resulting in
undefined behavior (usually an empty file) for the output file. To fix
this, we can generate this file in the `build.common` step which is not
run in parallel.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Add to the RGW multisite integration test a verification that the RGW
period is committed on the first reconcile and not committed on the
second reconcile.
Do this in the multisite test so that we verify that this works for
both the primary and secondary multi-site cluster.
To add this test, the github-action-helper.sh script had to be modified
to
1. actually deploy the version of Rook under test
2. adjust how functions are called to not lose the `-e` in a subshell
3. fix wait_for_prepare_pod helper that had a failure in the middle
of its operation that didn't cause failures in the past
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Replace calls to 'radosgw-admin period update --commit' with an
idempotent function.
Resolves#8879
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
When the OSD is prepared, systemd-udev kicks in since the device has
been exclusively opened and thus reverts the permissions to root:disk.
We need a udev rule to force re-applying the correct ceph permission so
we can consume the disk.
Closes: https://github.com/rook/rook/issues/8942
Signed-off-by: Sébastien Han <seb@redhat.com>
Blaine Gardner
Travis Nielsen
Sébastien Han
Madhu Rajanna
Arun Kumar Mohan
Rakshith R
Blaine Gardner
Renan Campos
Gowtham Shanmugasundaram
subhamkrai