Commit Graph
322 Commits
Author SHA1 Message Date
Blaine Gardner aba199fc4c docs: sort common.yaml example
In anticipation of generating common.yaml from the Helm charts, sort
common.yaml using the same script used to sort the output from Helm
charts.

This was done using the flow here:
```
cat build/rbac/common.yaml.header > new-common.yaml
cat deploy/examples/common.yaml | build/rbac/keep-rbac-yaml.sh >> new-common.yaml
mv new-common.yaml deploy/examples/common.yaml
```

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-12-02 14:24:14 -07:00
Blaine Gardner 2c4b66957c build: generate CSV from helm charts
In order to generate common.yaml from Helm charts, we have to have a
different way of generating CSV than from meta-comments in common.yaml.
This implementation changes what appears in the CSV's RBAC somewhat, but
these changes could be considered bugs fixed by using the new Helm
generation method.
- a few PSP related resources are removed from CSV
- resources related to 'rook-ceph-purge-osd' Job are added to CSV
- ClusterRoleBinding 'rook-ceph-object-bucket' is added to CSV

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-11-30 08:35:54 -07:00
Blaine Gardner 113e2f849d build: use yq for RBAC yaml parsing
Use yq instead of Python for parsing RBAC from the Helm chart. We need
to use yq v4.14.1 or higher to fix yq's handling of the yaml header
markers ('---'). Update the Makefile's yq version to v4, which also
requires updating the script to update the CRDs. This was quite easy.

It is very difficult, however, to change the version of yq used by the
CSV generating/parsing scripts, which already used their own yq
download. Continue using yq v3 for this.

In order to make sure the scripts are using the right version of yq, add
basic validation to them to verify they are running v3 or v4 as required
for their operation.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-11-30 08:08:08 -07:00
Sébastien Han c890710b63 core: change directory layout
As per discussion, proposing a new layout for the charts/yaml/olm files.

./deploy
├── charts
│   ├── rook-ceph
│   │   └── templates
│   └── rook-ceph-cluster
│       └── templates
├── examples
│   ├── csi
│   │   ├── cephfs
│   │   └── rbd
│   ├── flex
│   ├── monitoring
│   ├── pre-k8s-1.16
└── olm
    └── assemble

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-11-30 09:12:53 +01:00
Sébastien Han 6fb67a297d build: only produce binary for linux
We now build rook to build only for linux platforms and not darwin nor
windows anymore.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-11-04 15:13:40 +01:00
Sébastien Han e08755ef61 build: make CGO_ENABLED configurable
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-11-04 15:13:40 +01:00
Travis Nielsen 71683917fd Merge pull request #9051 from travisn/helm-app-version
helm: Add appVersion property to the charts
2021-10-28 11:01:27 -06:00
Travis Nielsen bffe99c39d helm: add appversion property to the charts
The appVersion should be set to the version of the application. Since the helm
charts are built by Rook in the same release version as Rook itself, the
version and appVersion values will be the same.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-10-28 09:26:18 -06:00
Travis Nielsen 8715964fd5 docs: update release process for github actions
Jenkins has been decommissioned so the release instructions are
updated to work more clearly with the new processes around
the github actions.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-10-20 12:10:21 -06:00
PixelJonas 8733bf6272 ceph: use normal quotation marks for comment
replaces all occurences of lduo/rduo quotation marks to make
sure that using the snippets in a k8s manifest will work

This fixes an issue with ArgoCD not being able
to apply `common.yaml` because of an encoding issue

Signed-off-by: PixelJonas <jonas@janz.digital>
2021-10-20 10:09:22 +02:00
Blaine Gardner d9146f6f95 helm: create shared library chart
Create a Helm library chart to share content between the Operator Helm
chart (rook-ceph) and the CephCluster (etc.) Helm chart
(rook-ceph-cluster).

This will ensure that when resources related to the CephCluster change,
notably RBAC resources, they will change for both charts.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-18 12:33:47 -06:00
Travis Nielsen 084dc41045 Merge pull request #8884 from BlaineEXE/remove-unused-helm-content
helm: remove chart content not in common.yaml
2021-10-07 13:43:17 -06:00
Travis Nielsen c420f2309c csi: no longer install the volumereplication crds from rook
The volume replication CRDs are an external component, not owned by Rook.
Therefore, they should be installed as any other independent component
in case the admin will install other consumers of the volumereplication CRDs
in the future in addition to Rook and the CSI driver.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-10-06 16:27:44 +00:00
Blaine Gardner 507dfe11f6 helm: remove chart content not in common.yaml
Assume common.yaml is the right source of truth, and remove major
content from helm charts that does not exist in common.yaml.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-10-06 09:18:02 +00:00
Blaine Gardner e2ba16f710 build: start tracking rbac generated from helm chart
This is a starting step to be able to generate common.yaml from Helm
charts. For right now, we merely want to be able to determine when the
rendered output of the Helm chart changes.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-09-30 12:20:24 -06:00
Sébastien Han 12506c7918 Merge pull request #8779 from travisn/obsolete-test-scripts
test: Remove obsolete test scripts for minishift, kubeadm, and update test readme
2021-09-24 09:38:11 +02:00
Travis Nielsen f5c1543ddf build: update min k8s version to 1.16
In Rook v1.8 the min version of K8s supported is updated to 1.16.
Users running on older versions of K8s are recommended to update
to 1.16 or newer before updating to Rook v1.8.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:56 -06:00
Travis Nielsen 0a0b9c98bd build: remove obsolete flex driver
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:20 -06:00
Travis Nielsen ce52a07477 test: simplify instructions for minikube
Simplify the test documentation on running the integration tests
to clarify that any K8s cluster can be used, and only support
minikube in the instructions.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 22:16:53 +00:00
Travis Nielsen 76748f0419 test: remove obsolete minishift and kubeadm scripts
The test scripts to start minishift and kubeadm are no
longer in use by Jenkins or anyone else.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 22:16:53 +00:00
Travis Nielsen fee7e82416 docs: publish master docs to latest path
Rename the master docs to latest. This will be easier
to understand for users who are new to the project.
The url will also contain latest, which will be more
clear to reference instead of the master path. This
also follows the pattern of the ceph docs site.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-10 17:55:14 -06:00
Blaine Gardner 7febd4e303 ci: add debugging to release build script
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-09-01 13:58:21 -06:00
Travis Nielsen 66c7c961cc Merge pull request #8617 from subhamkrai/push-build-failed
ci: build failed due unbound var
2021-09-01 10:24:56 -06:00
subhamkrai 3b99d24340 ci: build failed due unbound var
removing unused var which was making push build
action to fail due set -u which was added recently.

Closes: https://github.com/rook/rook/issues/8616
Signed-off-by: subhamkrai <srai@redhat.com>
2021-09-01 15:15:45 +05:30
Blaine Gardner 7a14eb58a0 ceph: remove NFS and Cassandra from codegen
Remove NFS and Cassandra from codegen scripts, BUT leave generated code
to be removed later.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:35 -06:00
Blaine Gardner 1b1736a3c5 ceph: remove NFS and Cassandra example manifests
Additionally remove NFS and Cassandra from CRD generator script.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:34 -06:00
Blaine Gardner 9531c252a5 ceph: remove NFS and Cassandra make targets
Remove all make-related references to NFS and Cassandra. This is limited
purely to make targets. Leave CRD and codegen script references to be
removed in a different commit.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-31 14:05:33 -06:00
subhamkrai ebe21c8f68 ci: add action for shellcheck linter
we are adding new linter for shellcheck.
As we are writing more shell scripts this
will help maintain quality.

Also, doing all the changes required in
bash files to pass this shellcheck.

Closes: https://github.com/rook/rook/issues/8431
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-26 10:03:32 +05:30
Sébastien Han 06a5571149 core: parse go files only subdirs
Instead of using the root let's only lookup our packages. Otherwise we
will format files from ./.work.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 18:52:39 +02:00
Sébastien Han cd9754e2f1 core: do not format deepcopy-gen files
deepcopy-gen appends a header starting with `// +build` which makes
gofmt to complain in Go 1.17 so until this is fixed in
https://github.com/kubernetes/gengo/pull/210 let's skip those files for
formatting.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 12:03:17 +02:00
Sébastien Han 7c14b32435 ci: build on golang 1.16 and 1.17
The CI now runs the build test against Golang 1.16 and 1.17.
So 1.17 is been added as a supported version.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-08-24 11:18:51 +02:00
subhamkrai 1681cfe89d ceph: add -0 for helm k8s version check
to use prerelease of kubernetes it requires
to have `-0` added in kubernetes version check
ex:
```{{- if semverCompare ">=1.16.0-0" .Capabilities.KubeVersion.GitVersion }}```

Closes: https://github.com/rook/rook/issues/8548
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-18 19:15:48 +05:30
Blaine Gardner e51c078050 Merge pull request #8541 from BlaineEXE/print-go-version-on-build
build: print go version when building
2021-08-16 12:41:50 -06:00
Blaine Gardner a83745cbcd build: print go version when building
Print the Go version when running any Go build target. This can be
helpful in making sure that we are building with the Go version we want
and expect.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-08-16 12:16:07 -06:00
subhamkrai 7161afbb70 build: update go fmt version
till now we were using older go fmt version(1.11).
From now, go fmt version will be same as golang version

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-04 20:17:02 +05:30
Blaine Gardner 9138732c81 build: stage CSV files into tmp while building
If a user were to `make build` Rook, and after the Ceph CSV was
generated and before make had finished, the Rook repo would be left with
changes caused by the build modifying files temporarily in-place.

All CSV template generation in the Ceph makefile is now self-contained
and can be configured to operate outside of the main repo so that files
are not modified in-place.

In-place modification of files was also incorrect. On Mac some files
with `-e` trailing were left, and in-place backups were also present in
the form `<filename>''`. Therefore, also fix the generation to fix
in-place modifications.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-07-27 16:11:58 -06:00
Sébastien Han c8ee5674dd Merge pull request #8103 from leseb/rm-jenkins
ci: remove jenkins from master branch
2021-07-23 18:20:10 +02:00
Travis NielsenandNicolaj Græsholt f114db952b nfs: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for nfs for full fidelity of all settings.

Co-authored-by: Nicolaj Græsholt <figaw@hotmail.com>
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
Travis Nielsen 093cf6dbbd cassandra: generate the crds from types for v1
The CRDs v1 requires the full schema for all settings, so we now
generate the CRDs for cassandra for full fidelity of all the
settings.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-07-20 08:18:05 -06:00
subhamkrai bc1fabb6e4 ci: update helm version to 3.6.2
this commit update helm version to `3.6.2`

Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-07 20:09:26 +05:30
Sébastien Han 18b1477352 ci: remove jenkins from master branch
Thank you Jenkins for all those years, but it's to retire.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-10 15:34:15 +02:00
Travis Nielsen 1af84e34c0 Merge pull request #7778 from henryzhangsta/ceph-cluster-helm-chart
ceph: implement CephCluster CRD Helm Chart
2021-05-27 12:41:01 -06:00
Henry Zhang ecf6bdcfae ceph: add build and tests for rook-ceph-cluster Helm chart
Adds rook-ceph-cluster chart to chart build, add tests.
Pulled out some common functionality between the Helm and non-Helm installers

Signed-off-by: Henry Zhang <me@henry.dev>
2021-05-27 01:08:27 -07:00
Travis Nielsen e1b64e3624 Merge pull request #7945 from subhamkrai/bump-golang1.16
build: bump to golang version 1.16
2021-05-24 23:20:52 -06:00
subhamkrai bf7b4609b1 build: bump to golang version 1.16
this commit bump to golang version 1.16.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-05-19 13:35:20 +05:30
Travis Nielsen b0a63711f5 build: refactor to consolidate the rook.io/v1 package
The rook.io/v1 package was only an internal implementation detail and
does not have any CRDs that rely on it. The CRD deserialization should
handle the change in internal types without any issue. This separation
gives more flexibility for the storage providers to implement exactly
what is needed for their storage provider instead of forcing to use the
same types and risk affecting another storage provider.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-18 19:55:37 -06:00
Travis Nielsen 08d720f450 ceph: preserve volume claim template annotations during upgrade
The upgrade of the csv is failing to the latest schema when
the crushDeviceClass is specified and when the volumeClaimTemplate
annotations do not preserve the unknown fields. To fix the upgrade,
we preserve the fields even though it should not be necessary
from the strict schema.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-05-05 16:33:40 -06:00
Blaine Gardner d260b81e97 core: persist VolumeClaimTemplate metadtata
Allow metadata on VolumeClaimTemplates in StorageClassDeviceSets to be
persisted. The latest changes to controller-gen allows us to embed
objectmeta for embedded types.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-04-27 13:37:48 -06:00
Satoru Takeuchi 8a48dbb00d ceph: stop vendoring
We generate vendor directory in building process. It doesn't reduce
build time. In addition, we have encountered some vendoring-related
problems. So let's stop vendoring.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-04-21 02:01:56 +00:00
Sébastien Han 57131e1a90 ci: build and run various make commands on MacOS
Introducing a small action on MacOS to help us catch errors with various
'make' command on OSX, most notably 'sed' versus 'gnu-sed'.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-04-20 17:21:08 +02:00