In order to generate common.yaml from Helm charts, we have to have a
different way of generating CSV than from meta-comments in common.yaml.
This implementation changes what appears in the CSV's RBAC somewhat, but
these changes could be considered bugs fixed by using the new Helm
generation method.
- a few PSP related resources are removed from CSV
- resources related to 'rook-ceph-purge-osd' Job are added to CSV
- ClusterRoleBinding 'rook-ceph-object-bucket' is added to CSV
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Prevent parallel build collisions in the 'ceph' image by building
prerequisites for the builds before running the build targets in
parallel. Build targets can collide and try to create the same target at
nearly the same time, causing failures.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Use yq instead of Python for parsing RBAC from the Helm chart. We need
to use yq v4.14.1 or higher to fix yq's handling of the yaml header
markers ('---'). Update the Makefile's yq version to v4, which also
requires updating the script to update the CRDs. This was quite easy.
It is very difficult, however, to change the version of yq used by the
CSV generating/parsing scripts, which already used their own yq
download. Continue using yq v3 for this.
In order to make sure the scripts are using the right version of yq, add
basic validation to them to verify they are running v3 or v4 as required
for their operation.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
On mac, the docker build was failing due to an unspecified
S5CMD_ARCH. Now we default to build amd64 always and override
it if the arch is arm64.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The toolbox can now interact with S3 gateways using the `s5cmd` tool.
The binary is only 12M so this does not add up too much to the operator
image size.
Closes: https://github.com/rook/rook/issues/4968
Signed-off-by: Sébastien Han <seb@redhat.com>
The rook operator as well as the toolbox pod run with the "rook" user
with UID 2016. The UID was chosen based on the year of the initial
commit in the rook/rook repository.
No more root user running.
Closes: https://github.com/rook/rook/issues/8734
Signed-off-by: Sébastien Han <seb@redhat.com>
When generating the offline image list, the threaded crossbuild can
try to generate the image list from muliple jobs at once, resulting in
undefined behavior (usually an empty file) for the output file. To fix
this, we can generate this file in the `build.common` step which is not
run in parallel.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The volume replication CRDs are an external component, not owned by Rook.
Therefore, they should be installed as any other independent component
in case the admin will install other consumers of the volumereplication CRDs
in the future in addition to Rook and the CSI driver.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Build rook in AWS/Graviton (ARM64) platform. Upon installation of
operator-sdk, make sure that the downloaded binary can actually be
executed on local machine by invoking 'operator-sdk version'.
issue: https://github.com/rook/rook/issues/8926
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
Reading from a file, processing the file in a pipe, and outputting the
piped content to the same file can have undefined results, often leading
to an empty file. Use an intermediate temp file for generating the
offline image list.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
While build.all we have race condition where images.txt
file was update with duplicate entries. So, removing `-a`
from 1st tee and also add `sort -h`and `uniq` command to
confirm no duplicate entry is in the file.
Signed-off-by: subhamkrai <srai@redhat.com>
We don't need to use tini.
We don't have anything in the rook operator that would
either create zombie processes (no threads) or use
exec (to fork). The Go binary has a really good
signal handling mechanism.
Closes: https://github.com/rook/rook/issues/8794
Signed-off-by: Sébastien Han <seb@redhat.com>
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
it has happened more than once that the manifests do not
reflect the images so now reading from `pkg/operator/ceph/csi/spec.go`
as this what code uses.
Closes: https://github.com/rook/rook/issues/8687
Signed-off-by: subhamkrai <srai@redhat.com>
Thanks to Golang 1.16, we can now embed files in the Go binary. This
means we don't need to add the CSI templates files to the container
image. They are added in the Go binary at build time.
The existing location of the template must be in the package calling it.
So they moved to pkg/operator/ceph/csi/template. All the files have been
symlinked back to cluster/examples/kubernetes/ceph/csi/template.
Closes: https://github.com/rook/rook/issues/7609
Signed-off-by: Sébastien Han <seb@redhat.com>
If a user were to `make build` Rook, and after the Ceph CSV was
generated and before make had finished, the Rook repo would be left with
changes caused by the build modifying files temporarily in-place.
All CSV template generation in the Ceph makefile is now self-contained
and can be configured to operate outside of the main repo so that files
are not modified in-place.
In-place modification of files was also incorrect. On Mac some files
with `-e` trailing were left, and in-place backups were also present in
the form `<filename>''`. Therefore, also fix the generation to fix
in-place modifications.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Recently, the builds of `ceph/ceph` image moved to quay.io, see
https://github.com/ceph/ceph-build/pull/1883 for more details.
Current images will remain but new builds will happen on quay.io only.
This means that tags such as `v14.2`, `v15.2`,`v16.2` will need to
switch to quay.io to get updates.
Signed-off-by: Sébastien Han <seb@redhat.com>
Ceph v16 will be the default in Rook v1.7. Update references in the
master branch from v15 (currently v15.2.11) to v16 (currently v16.2.4).
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Ceph v16.2.2 is released with dated tag v16.2.2-20210505. Update Ceph
images to match with or without date as appropriate.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
We now the generate the CRDs as well as the CSV on a single GOARCH only:
amd64. This helps us avoiding builds colliding with each others.
Closes: https://github.com/rook/rook/issues/7655
Signed-off-by: Sébastien Han <seb@redhat.com>
This is the first bugfix release in the Pacific stable series. It
addresses a security vulnerability in the Ceph authentication framework.
We recommend users to update to this release. For a detailed release
notes with links & changelog please refer to the official blog entry at
https://ceph.io/releases/v16-2-1-pacific-released.
Signed-off-by: Sébastien Han <seb@redhat.com>
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.
Signed-off-by: Sébastien Han <seb@redhat.com>
CSV generation now requires versions of yq and operator-sdk that are
downloaded by the makefiles. Previously Ceph image creation used
makefile versions but `make csv-ceph` used versions present on the
system.
`sed -i 's///g' $FILE` doesn't work on Unix/Mac.
`sed -i'' -e 's///g' $FILE` is portable.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
With the Ceph Pacific release coming this week we add support
in Rook for Pacific with the Rook v1.6 release coming soon.
The integration tests will now run across nautilus, octopus,
and pacific to cover all supported Ceph versions. The default
examples still specify Octopus until there is more bake time
for Pacific.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We don't want to include those CRDs anymore, as they get imported from
another operator.
This trims further down the size of our CSV generation.
Additionally, we always clean the csv directory before proceeding.
Signed-off-by: Sébastien Han <seb@redhat.com>
The CSV file size cannot be bigger than 1MB so let's remove all the
description fields make it lighter.
With that change the CRDs size went from 692K to 372K so the size was
reduced by almost 50%.
Signed-off-by: Sébastien Han <seb@redhat.com>
From inside the toolbox, the ceph debug level can now be changed for all
ceph logging subsystems, for instance, level 1 will be:
```
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- set-ceph-debug-level 1
```
To reset to default values just run:
```
kubectl -n rook-ceph exec deploy/rook-ceph-tools -- set-ceph-debug-level default
```
Signed-off-by: Sébastien Han <seb@redhat.com>
There are still references to v15.2.7 in integration tests to allow
testing OSDs on partitions.
The newest dated Ceph image is ceph/ceph:v15.2.9-20210224
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The latest octopus release is now out with v15.2.8. We update the
operator base image and the examples to run with this version.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With the release of the latest octopus v15.2.7 we update the base
of the operator image and set the examples to use the same release
to pick up the security and other bug fixes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
The operator should only connect to ceph with a single set of creds.
In a converged cluster this will be the admin creds and in an external
cluster it will be lower-privileged creds. Independent clusters were
implemented with a separate set of creds. To simplify the code these
are now merged to a single set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When hostname has hypen and/or numbers, the mon_endpoints is incorrectly
split and can point to invalid monitor names inside toolbox, Add fix to
split the hostname/ip that can include hypen, underscore or numbers.
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
Fixes
Signed-off-by: Vasu Kulkarni <vasu@redhat.com>
Update the base operator image and cluster examples to use the latest
octopus release v15.2.4. Also cleanup some old examples and unit tests
that were still based on mimic.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Blaine Gardner
Sébastien Han
Travis Nielsen
Shachar Sharon
subhamkrai
Michael Adam
Blaine Gardner
Arun Kumar Mohan
Mateusz Gozdek
Vasu Kulkarni