In order to generate common.yaml from Helm charts, we have to have a
different way of generating CSV than from meta-comments in common.yaml.
This implementation changes what appears in the CSV's RBAC somewhat, but
these changes could be considered bugs fixed by using the new Helm
generation method.
- a few PSP related resources are removed from CSV
- resources related to 'rook-ceph-purge-osd' Job are added to CSV
- ClusterRoleBinding 'rook-ceph-object-bucket' is added to CSV
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Prevent parallel build collisions in the 'ceph' image by building
prerequisites for the builds before running the build targets in
parallel. Build targets can collide and try to create the same target at
nearly the same time, causing failures.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Use yq instead of Python for parsing RBAC from the Helm chart. We need
to use yq v4.14.1 or higher to fix yq's handling of the yaml header
markers ('---'). Update the Makefile's yq version to v4, which also
requires updating the script to update the CRDs. This was quite easy.
It is very difficult, however, to change the version of yq used by the
CSV generating/parsing scripts, which already used their own yq
download. Continue using yq v3 for this.
In order to make sure the scripts are using the right version of yq, add
basic validation to them to verify they are running v3 or v4 as required
for their operation.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The current default target of Makefile is "_output/charts".
```
$ make -d
...
Successfully remade target file '/home/sat/go/src/github.com/rook/rook/_output/charts'.
```
It's because Makefile first includes "build/makelib/helm.mk", this file defines
some targets, and the first entry is regarded as the default target.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
When generating the offline image list, the threaded crossbuild can
try to generate the image list from muliple jobs at once, resulting in
undefined behavior (usually an empty file) for the output file. To fix
this, we can generate this file in the `build.common` step which is not
run in parallel.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This is a starting step to be able to generate common.yaml from Helm
charts. For right now, we merely want to be able to determine when the
rendered output of the Helm chart changes.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
If a user were to `make build` Rook, and after the Ceph CSV was
generated and before make had finished, the Rook repo would be left with
changes caused by the build modifying files temporarily in-place.
All CSV template generation in the Ceph makefile is now self-contained
and can be configured to operate outside of the main repo so that files
are not modified in-place.
In-place modification of files was also incorrect. On Mac some files
with `-e` trailing were left, and in-place backups were also present in
the form `<filename>''`. Therefore, also fix the generation to fix
in-place modifications.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Allow metadata on VolumeClaimTemplates in StorageClassDeviceSets to be
persisted. The latest changes to controller-gen allows us to embed
objectmeta for embedded types.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
No need to include darwin and windows in the list of platforms
since rook is only containerized and expected to run in a linux
environment.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.
Signed-off-by: Sébastien Han <seb@redhat.com>
CSV generation now requires versions of yq and operator-sdk that are
downloaded by the makefiles. Previously Ceph image creation used
makefile versions but `make csv-ceph` used versions present on the
system.
`sed -i 's///g' $FILE` doesn't work on Unix/Mac.
`sed -i'' -e 's///g' $FILE` is portable.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
We don't want to include those CRDs anymore, as they get imported from
another operator.
This trims further down the size of our CSV generation.
Additionally, we always clean the csv directory before proceeding.
Signed-off-by: Sébastien Han <seb@redhat.com>
The CSV file size cannot be bigger than 1MB so let's remove all the
description fields make it lighter.
With that change the CRDs size went from 692K to 372K so the size was
reduced by almost 50%.
Signed-off-by: Sébastien Han <seb@redhat.com>
The crds-gen make target was recently renamed to crds. This broke the
ceph-csv target since it still depends on the old name.
Signed-off-by: Boris Ranto <branto@redhat.com>
Let's not use any controllen-gen version we might find on the system,
always pull the version we need.
Same goes for yq.
Signed-off-by: Sébastien Han <seb@redhat.com>
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:
* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`
will be autogenerated for us.
We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.
It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:
```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```
The corresponding API type markers are:
```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```
For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html
Signed-off-by: Sébastien Han <seb@redhat.com>
Currently olm will be removed as part of clean command, but it cleans up
all the dependent modules as well
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
makefile items such as "check test", "mod.update", "mod.check" were
missing from help menu.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
This uses grep, sort and awk to auto generate the `make help` menu based
on comments after each target.
Removed `clean.all` and `prune.all` targets as they are not available
anymore.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Every developer should commit all the files that need to be changed
in a given set of changes. The go.sum file was not being generated
as expected since the update to go modules. Since the Jenkinsfile
was running make mod.check, the build was ending up with a modified
file, which caused a dirty tag on the build. Now it is expected
that each PR will merge with the corresponding go.sum changes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The longhaul tests have not been run for a very long time,
on the order of two years. The intention is to get them running
again, but until then it is better to purge the obsolete sources
than to attempt to keep them building. We'll need to rewrite them
when they are restored.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
remove all cosntraints that are not really needed from Gopkg.toml.
we should only be adding constraints that are direct dependencies
of the project and we require a specific version/revision of.
added a vendor.update makefile target that can be used to update
dependencies to the latest.
Signed-off-by: Bassam Tabbara <bassam@upbound.io>
this commit updates dep to version 0.5, and adds a check during
CI runs that enforces that the vendor dependencies have not changed.
There is a now a makefile target `make vendor.check` that performs
this check. Also removed the requirement that PR run make vendor as
this will be done by CI.
Signed-off-by: Bassam Tabbara <bassam@upbound.io>
Raise the scope of the `IMAGES` config variable in the
`build/release/Makefile` to the `build/makelib/common.mk` file, and use
the `IMAGES` variable in the `images/Makefile` to allow users to build
only select images.
Examples of new use:
make IMAGES='ceph ceph-toolbox' build
make -C build/release IMAGES='ceph ceph-toolbox' publish
The 'images/image.mk' clean.build target was changed to use
`CLEAN_IMAGES` instead of `IMAGES`, as it was conflicting with use of
the parameter in image builds.
Added IMAGES option to top-level make's help text. The text for this
option exceeded 80 lines in the `Makefile` itself, so the help text was
refactored to eliminate ` @echo '<text>'`. The `Makefile` text should
now match exactly what is output to the user including line length.
Added IMAGES option to `INSTALL.md` file's "Build" section and also
moved some text around and added a few extra points of clarification
there to hopefully improve readability, especially for first-time
readers.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
AArch64 platform doesn't support the `build.all` command due to the
`hypriot/qemu-register` issue, so we disable this kind of 'cross' build
on AArch64 platform.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Update all K8s dependencies to the release-1.8 branches.
Added build and make file support for generating code.
Fixed our use of client-go to match changes in new release.
- Operator is deployed under namespace rook-system
- Operator deploys rook-agent on all nodes as daemonset also on
rook-system
- Rook-agent installs flexvolume driver on hosts
- Rook-agent listens on unix socket for driver requests
- Rook-agent perform attach/detach on its node
- Rook-agent creates/delete CRD volumeattach objects
- Added fencing to support ROX and RWO
- Added unit and integration tests
- Updated examples and docks
fixes#432
- Add support for running individual test in a tests suite
- building integration and longhaul test binaries
- Addiing support for skipping CI or running just smoke test on a PR.
Blaine Gardner
Sébastien Han
Satoru Takeuchi
Travis Nielsen
Boris Ranto
Jiffin Tony Thottan
Alexander Trost
moricho
Mateusz Gozdek
Bassam Tabbara
Blaine Gardner
Dennis Chen
Bassam Tabbara
Steve Leon
Divesh Angula
Jared Watts
Bassam Tabbara