The upgrade of the csv is failing to the latest schema when
the crushDeviceClass is specified and when the volumeClaimTemplate
annotations do not preserve the unknown fields. To fix the upgrade,
we preserve the fields even though it should not be necessary
from the strict schema.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Allow metadata on VolumeClaimTemplates in StorageClassDeviceSets to be
persisted. The latest changes to controller-gen allows us to embed
objectmeta for embedded types.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.
Signed-off-by: Sébastien Han <seb@redhat.com>
The POSIX spec for sed requires the flag -i to have a string specified.
`sed -i'' -e '<expression>'` is portable
The POSIX spec for sed also requires the insert "i" option to have
newlines. The following is portable:
```
sed -i'' -e '<line-num>i\
text'
```
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
The schema generation was not generating the metadata properties
under the VolumeClaimTemplate type, resulting in the properties being
ignored by the api server. The preserve unknown fields tag was
not working recursively on the volumeClaimTemplates since the subtypes
were defined. Now we post-process the schema so we can preserve
the unknown fields for the volume claim templates metadata.
We now use 'yq' to hot-fix the CRDs, the only 'downside' is that 'yq'
will not preserve the existing formatting, this is a known problem:
https://github.com/mikefarah/yq/issues/465
Signed-off-by: Sébastien Han <seb@redhat.com>
This reverts commit a9fb9d20bb. During the
CSV generation we strip the description lines from the CRD spec, however
the 'sed' command relies on the 'descrition' section to operate. Next
commit will fix this.
Signed-off-by: Sébastien Han <seb@redhat.com>
Dependencies for `make go.test` have to run in order, so call them
explicitly from the target rather than allowing them to be done in
parallel if `make -j` flag is used.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
CI fails on builds warning with the warning below
[2021-04-15T14:15:28.461Z] go: inconsistent vendoring in /home/rook/go/src/github.com/rook/rook:
[2021-04-15T14:15:28.461Z] github.com/jstemmer/go-junit-report@v0.9.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
Fix this by running `go mod vendor` before each build.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
With the help of https://github.com/rook/rook/pull/7403/, rook will soon
integrate the volume replication controller as a side-car. Thus Rook
needs to inject the new CRDs.
Signed-off-by: Sébastien Han <seb@redhat.com>
The schema generation was not generating the metadata properties
under the VolumeClaimTemplate type, resulting in the properties being
ignored by the api server. The preserve unknown fields tag was
not working recursively on the volumeClaimTemplates since the subtypes
were defined. Now we post-process the schema so we can preserve
the unknown fields for the volume claim templates metadata.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With the release of go 1.16, we update the tools. The -i flag is
now deprecated, no longer necessary for building in a private directory.
The go modules are cleaned up at the same time.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
CSV generation now requires versions of yq and operator-sdk that are
downloaded by the makefiles. Previously Ceph image creation used
makefile versions but `make csv-ceph` used versions present on the
system.
`sed -i 's///g' $FILE` doesn't work on Unix/Mac.
`sed -i'' -e 's///g' $FILE` is portable.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
We don't want to include those CRDs anymore, as they get imported from
another operator.
This trims further down the size of our CSV generation.
Additionally, we always clean the csv directory before proceeding.
Signed-off-by: Sébastien Han <seb@redhat.com>
The CSV file size cannot be bigger than 1MB so let's remove all the
description fields make it lighter.
With that change the CRDs size went from 692K to 372K so the size was
reduced by almost 50%.
Signed-off-by: Sébastien Han <seb@redhat.com>
On mac the make crds command was failing on the sed commands.
Now we remove the dependency on sed and just append the files
to generate the helm resources
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Let's not use any controllen-gen version we might find on the system,
always pull the version we need.
Same goes for yq.
Signed-off-by: Sébastien Han <seb@redhat.com>
The helm chart was not installing the CRDs on pre-1.16 k8s due
to a regression in the conditionals for the chart that was introduced
with 7103.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:
* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`
will be autogenerated for us.
We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.
It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:
```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```
The corresponding API type markers are:
```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```
For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html
Signed-off-by: Sébastien Han <seb@redhat.com>
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
shasum is not always available and sha256sum seems to exist more often
since it's part of coreutils where sha256sum comes with perl.
Signed-off-by: Sébastien Han <seb@redhat.com>
Introducing the new flag to get the correct plurial version of CephNFS,
if not overriden the generator will be CephNFSs instead of CephNFSes.
We run deepcopy and client,lister,informer generations separetly so we can use the flag "--plural-exceptions"
which is only known by client,lister,informer binary and not the deepcopy binary.
Signed-off-by: Sébastien Han <seb@redhat.com>
Somehow, I had things mixed up in the previous patch. Now the code
generation ends up in the right directorty.
Signed-off-by: Sébastien Han <seb@redhat.com>
With the recent bump to Golang 1.15, the code generation was broken.
Changing the output-base to the main repo and not the vendor solves the
issue and feel more natural too. The command "go mod vendor" only
cherry-picks certain files for inclusion in the vendor folder. I
suspect the upgrade to Golang 1.15 introduced that behavior. Previously
our vendir dir was full and now it has nothing (from rook) that's why
the code generator was removing all our interfaces.
Signed-off-by: Sébastien Han <seb@redhat.com>
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This updates the chart to make use of helm3 which has been released
for some time, and also permits CRDs to be installed pror to other objects
allowing the chart to be deployed at the same time as CRs for rook objects.
Co-authored-by: Pete Birley <pete@port.direct>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Found by running the following command:
codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
To be able to switch to helm v3, helm itself needs to be downloaded from
get.helm.sh. The old location from googleapis.com will only contain
helm v2 binaries. See [1].
[1] https://helm.sh/blog/get-helm-sh/
Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
golangci-lint linter gosec showing more errors
than gosec gh. This commit resolve
new errors. And, removing
nosec comments from autogenerated files.
Signed-off-by: subhamkrai <srai@redhat.com>
this commit handles all the gosec g601
error code (i.e Implicit memory aliasing
of items from a range statement).
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
Currently, Helm v2.5.1 is used, but this version is released in 2017
and a little outdated.
To use the latest version of helm v2.x.x would be beneficial
as we can use more useful Helm commands like `helm template`.
This commit updates Helm version and stop using `helm lint`
with `--strict` option because Helm v2.16.7 does not allow
to use undefined values in values.yaml and
the Helm upstream is now stopping using the `--strict` option.
Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
The path was copied in the wrong location, ending up with a duplicated
directory in pkg/pkg/api. We need pkg/ only.
Signed-off-by: Sébastien Han <seb@redhat.com>
This modify codegen.sh to work correctly.
`make codegen` temporarily download code-generator into vendor directory and use it
to refer to the local version of rook.
Signed-off-by: moricho <ikeda.morito@gmail.com>
The manifest tool is needed for pushing images to dockerhub.
The tool has been failing the past two days due to running
an old version.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The types that are used in CRDs across the storage providers do not need
to be in the v1alpha2 package. These should be declared stable in the v1
package. The only types remaining in the v1alpha2 package are the flex
volume types, which will never graduate since their replacement is the
csi driver.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Travis Nielsen
Blaine Gardner
Satoru Takeuchi
Sébastien Han
Artur
Pete Birley
Mateusz Gozdek
Thomas Bechtold
subhamkrai
Alexander Trost
subhamkrai
binoue
moricho