Commit Graph
291 Commits
Author SHA1 Message Date
subhamkrai 1681cfe89d ceph: add -0 for helm k8s version check
to use prerelease of kubernetes it requires
to have `-0` added in kubernetes version check
ex:
```{{- if semverCompare ">=1.16.0-0" .Capabilities.KubeVersion.GitVersion }}```

Closes: https://github.com/rook/rook/issues/8548
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-18 19:15:48 +05:30
Blaine Gardner e51c078050 Merge pull request #8541 from BlaineEXE/print-go-version-on-build
build: print go version when building
2021-08-16 12:41:50 -06:00
Blaine Gardner a83745cbcd build: print go version when building
Print the Go version when running any Go build target. This can be
helpful in making sure that we are building with the Go version we want
and expect.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-16 12:16:07 -06:00
subhamkrai 7161afbb70 build: update go fmt version
till now we were using older go fmt version(1.11).
From now, go fmt version will be same as golang version

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-04 20:17:02 +05:30
Blaine Gardner 9138732c81 build: stage CSV files into tmp while building
If a user were to `make build` Rook, and after the Ceph CSV was
generated and before make had finished, the Rook repo would be left with
changes caused by the build modifying files temporarily in-place.

All CSV template generation in the Ceph makefile is now self-contained
and can be configured to operate outside of the main repo so that files
are not modified in-place.

In-place modification of files was also incorrect. On Mac some files
with `-e` trailing were left, and in-place backups were also present in
the form `<filename>''`. Therefore, also fix the generation to fix
in-place modifications.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-07-27 16:11:58 -06:00
Sébastien Han c8ee5674dd Merge pull request #8103 from leseb/rm-jenkins
ci: remove jenkins from master branch
2021-07-23 18:20:10 +02:00
Travis NielsenandNicolaj Græsholt f114db952b nfs: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for nfs for full fidelity of all settings.

Co-authored-by: Nicolaj Græsholt <figaw@hotmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
Travis Nielsen 093cf6dbbd cassandra: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for cassandra for full fidelity of all the
settings.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
subhamkrai bc1fabb6e4 ci: update helm version to 3.6.2
this commit update helm version to `3.6.2`

Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-07 20:09:26 +05:30
Sébastien Han 18b1477352 ci: remove jenkins from master branch
Thank you Jenkins for all those years, but it's to retire.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-10 15:34:15 +02:00
Travis Nielsen 1af84e34c0 Merge pull request #7778 from henryzhangsta/ceph-cluster-helm-chart
ceph: implement CephCluster CRD Helm Chart
2021-05-27 12:41:01 -06:00
Henry Zhang ecf6bdcfae ceph: add build and tests for rook-ceph-cluster Helm chart
Adds rook-ceph-cluster chart to chart build, add tests.
Pulled out some common functionality between the Helm and non-Helm installers

Signed-off-by: Henry Zhang <me@henry.dev>
2021-05-27 01:08:27 -07:00
Travis Nielsen e1b64e3624 Merge pull request #7945 from subhamkrai/bump-golang1.16
build: bump to golang version 1.16
2021-05-24 23:20:52 -06:00
subhamkrai bf7b4609b1 build: bump to golang version 1.16
this commit bump to golang version 1.16.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-19 13:35:20 +05:30
Travis Nielsen b0a63711f5 build: refactor to consolidate the rook.io/v1 package
The rook.io/v1 package was only an internal implementation detail and
does not have any CRDs that rely on it. The CRD deserialization should
handle the change in internal types without any issue. This separation
gives more flexibility for the storage providers to implement exactly
what is needed for their storage provider instead of forcing to use the
same types and risk affecting another storage provider.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-18 19:55:37 -06:00
Travis Nielsen 08d720f450 ceph: preserve volume claim template annotations during upgrade
The upgrade of the csv is failing to the latest schema when
the crushDeviceClass is specified and when the volumeClaimTemplate
annotations do not preserve the unknown fields. To fix the upgrade,
we preserve the fields even though it should not be necessary
from the strict schema.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-05 16:33:40 -06:00
Blaine Gardner d260b81e97 core: persist VolumeClaimTemplate metadtata
Allow metadata on VolumeClaimTemplates in StorageClassDeviceSets to be
persisted. The latest changes to controller-gen allows us to embed
objectmeta for embedded types.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-27 13:37:48 -06:00
Satoru Takeuchi 8a48dbb00d ceph: stop vendoring
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 02:01:56 +00:00
Sébastien Han 57131e1a90 ci: build and run various make commands on MacOS
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00
Blaine Gardner 02e2a3d2eb build: fix new sed line to work on mac
The POSIX spec for sed requires the flag -i to have a string specified.
`sed -i'' -e '<expression>'` is portable

The POSIX spec for sed also requires the insert "i" option to have
newlines. The following is portable:
```
sed -i'' -e '<line-num>i\
text'
```

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-20 17:20:05 +02:00
Sébastien Han 21d6001ed2 ceph: preserve volume claim template metadata in schema
The schema generation was not generating the metadata properties
under the VolumeClaimTemplate type, resulting in the properties being
ignored by the api server. The preserve unknown fields tag was
not working recursively on the volumeClaimTemplates since the subtypes
were defined. Now we post-process the schema so we can preserve
the unknown fields for the volume claim templates metadata.

We now use 'yq' to hot-fix the CRDs, the only 'downside' is that 'yq'
will not preserve the existing formatting, this is a known problem:

https://github.com/mikefarah/yq/issues/465

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 19:03:13 +02:00
Sébastien Han 341aefc95d ceph: revert "ceph: preserve volume claim template metadata in schema"
This reverts commit a9fb9d20bb. During the
CSV generation we strip the description lines from the CRD spec, however
the 'sed' command relies on the 'descrition' section to operate. Next
commit will fix this.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-19 19:01:13 +02:00
Blaine Gardner a818a01a13 test: do not allow go.test parallel dependencies
Dependencies for `make go.test` have to run in order, so call them
explicitly from the target rather than allowing them to be done in
parallel if `make -j` flag is used.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-16 12:24:39 -06:00
Blaine Gardner 37b31a7c5e build: run 'go mod vendor' on each build
CI fails on builds warning with the warning below

    [2021-04-15T14:15:28.461Z] go: inconsistent vendoring in /home/rook/go/src/github.com/rook/rook:
    [2021-04-15T14:15:28.461Z] 	github.com/jstemmer/go-junit-report@v0.9.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt

Fix this by running `go mod vendor` before each build.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-15 16:11:59 -06:00
Sébastien Han 9416e52250 Merge pull request #7619 from leseb/add-vol-rep-crds
ceph: add volume replication crds
2021-04-15 15:15:02 +02:00
Sébastien Han 83a44ae4df ceph: add volume replication crds
With the help of https://github.com/rook/rook/pull/7403/, rook will soon
integrate the volume replication controller as a side-car. Thus Rook
needs to inject the new CRDs.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-15 13:18:13 +02:00
Travis Nielsen a9fb9d20bb ceph: preserve volume claim template metadata in schema
The schema generation was not generating the metadata properties
under the VolumeClaimTemplate type, resulting in the properties being
ignored by the api server. The preserve unknown fields tag was
not working recursively on the volumeClaimTemplates since the subtypes
were defined. Now we post-process the schema so we can preserve
the unknown fields for the volume claim templates metadata.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-14 21:41:12 -06:00
Travis Nielsen 31c4ef622e build: update the build to use go 1.16
With the release of go 1.16, we update the tools. The -i flag is
now deprecated, no longer necessary for building in a private directory.
The go modules are cleaned up at the same time.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-04-12 15:01:14 -06:00
Blaine Gardner f2f622412c build: allow CSV gen on mac
CSV generation now requires versions of yq and operator-sdk that are
downloaded by the makefiles. Previously Ceph image creation used
makefile versions but `make csv-ceph` used versions present on the
system.

`sed -i 's///g' $FILE` doesn't work on Unix/Mac.
`sed -i'' -e 's///g' $FILE` is portable.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-09 13:22:15 -06:00
Sébastien Han 3d1d659e28 ci: do not include ob/obc/volumes crds in csv
We don't want to include those CRDs anymore, as they get imported from
another operator.
This trims further down the size of our CSV generation.

Additionally, we always clean the csv directory before proceeding.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-06 17:08:39 +02:00
Sébastien Han 5a194af3b1 ceph: strip crd description for csv generation
The CSV file size cannot be bigger than 1MB so let's remove all the
description fields make it lighter.

With that change the CRDs size went from 692K to 372K so the size was
reduced by almost 50%.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-06 14:13:07 +02:00
Travis Nielsen c73a2e943d ceph: compatibility for generating crds without sed
On mac the make crds command was failing on the sed commands.
Now we remove the dependency on sed and just append the files
to generate the helm resources

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-30 14:18:17 -06:00
Sébastien Han cf00a86fc2 ceph: force yq and controller-gen versions
Let's not use any controllen-gen version we might find on the system,
always pull the version we need.
Same goes for yq.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-26 09:21:33 +01:00
Travis Nielsen deb7c21f98 ceph: install pre-1.16 crds with helm chart
The helm chart was not installing the CRDs on pre-1.16 k8s due
to a regression in the conditionals for the chart that was introduced
with 7103.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-03-17 13:10:53 -06:00
Sébastien Han 31db03fece ceph: auto-gen crds
Finally! We can now stop editing manually our crds definition. Simply
run `make crds-gen`. These two files:

* `cluster/examples/kubernetes/ceph/crds.yaml`
* `cluster/charts/rook-ceph/templates/resources.yaml`

will be autogenerated for us.

We rely on the controller-gen tool, it reads our API definitions from
`pkg/apis/` and produces the CRD files accordingly.

It uses "markers" to add extra yaml fields to the CRD, for instance we
have a lot fields with:

```
nullable: true
x-kubernetes-preserve-unknown-fields: true
```

The corresponding API type markers are:

```
// +kubebuilder:pruning:PreserveUnknownFields
// +nullable
```

For more API convention see
https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#optional-vs-required
and for the markers see: https://book.kubebuilder.io/reference/markers/crd-validation.html

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-03-16 09:16:24 +01:00
Travis Nielsen 043f947321 build: remove yugabytedb operator
The YugabyteDB team has decided to move support of their operator
to a new repo at https://github.com/yugabyte/yugabyte-operator.
The operator in Rook is no longer needed. Further usage of
the YugabyteDB operator is recommended at the new location.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-02-06 00:03:41 -07:00
Sébastien Han 1bdee1e512 build: try sha256sum if shasum is not present
shasum is not always available and sha256sum seems to exist more often
since it's part of coreutils where sha256sum comes with perl.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-02 14:43:52 +01:00
Sébastien Han 8ea9b6b80f ceph: fix CephFNS plurial
Introducing the new flag to get the correct plurial version of CephNFS,
if not overriden the generator will be CephNFSs instead of CephNFSes.

We run deepcopy and client,lister,informer generations separetly so we can use the flag "--plural-exceptions"
which is only known by client,lister,informer binary and not the deepcopy binary.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-01 16:36:56 +01:00
Sébastien Han 3d556115ce ceph: fix code generation
Somehow, I had things mixed up in the previous patch. Now the code
generation ends up in the right directorty.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-02-01 15:13:37 +01:00
Sébastien Han 32f5bfd6bf ci: fix code generation
With the recent bump to Golang 1.15, the code generation was broken.
Changing the output-base to the main repo and not the vendor solves the
issue and feel more natural too. The command "go mod vendor" only
cherry-picks certain files for inclusion  in the vendor folder. I
suspect the upgrade to Golang 1.15 introduced that behavior. Previously
our vendir dir was full and now it has nothing (from rook) that's why
the code generator was removing all our interfaces.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-28 18:21:07 +01:00
Sébastien Han 6936706d81 ci: use code-gen 1.20
Let's just the same version as our go.mod suggests. This workaround
seems old and not necessary anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-01-28 17:06:09 +01:00
Travis Nielsen 53ed11f15b build: remove the edgefs operator from rook
The EdgeFS operator has been deprecated for some time in Rook.
If the replacement is added back to Rook it can be completed
according to the new guidelines in the documentation.
https://rook.io/docs/rook/master/storage-providers.html

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-25 17:51:24 -07:00
Travis Nielsen 2294851ca3 build: remove the cockroachdb operator from rook
The cockroachDB operator has not had community support in Rook.
Therefore, the time has come to deprecate and remove it.
If the sources are still needed, there is always git history.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-20 17:28:58 -07:00
Travis Nielsen 7f169a71a5 build: enforce golang 1.15 in the build
The build currently requires golang 1.15 and will fail if an
older version is used.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-19 11:20:18 -07:00
Travis Nielsen 9ba8c937dd build: print the golang version during the build
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-01-19 11:19:53 -07:00
Artur 0754695655 build: switch to new aws buckets
Signed-off-by: Artur <artur@upbound.io>
2021-01-15 23:58:23 -08:00
Pete Birley 152a05c85e ceph: update to helm 3 for the rook chart
This updates the chart to make use of helm3 which has been released
for some time, and also permits CRDs to be installed pror to other objects
allowing the chart to be deployed at the same time as CRs for rook objects.

Co-authored-by: Pete Birley <pete@port.direct>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-11-11 14:41:24 -07:00
Mateusz Gozdek 8ba3762fa4 docs: fix bunch of typos
Found by running the following command:

codespell -S .git,*.png,*.jpg -L \
aks,keyserver,atleast,dne,ser,ist,files\',ba,dum,iam,te -f -H

Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
2020-11-06 10:01:04 +01:00
Thomas Bechtold 7eae8086e4 ceph: use get.helm.sh instead of storage.googleapis.com
To be able to switch to helm v3, helm itself needs to be downloaded from
get.helm.sh. The old location from googleapis.com will only contain
helm v2 binaries. See [1].

[1] https://helm.sh/blog/get-helm-sh/

Signed-off-by: Thomas Bechtold <tbechtold@suse.com>
2020-10-20 14:47:09 +02:00
subhamkrai cb0ca66a6b ci: enable gosec linter in golangci-lint
golangci-lint linter gosec showing more errors
than gosec gh. This commit resolve
new errors. And, removing
nosec comments from autogenerated files.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-10-19 11:50:06 +05:30