Commit Graph
3676 Commits
Author SHA1 Message Date
Blaine Gardner 3584e8ccce ceph: osd: fix wrong log file path on host
Log file path on host for OSDs was dataDirHostPath/log/<namespace>
instead of dataDirHostPath/<namespace>/log as with other daemons.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-10-15 10:10:06 -06:00
Travis Nielsen fdd92ec883 Merge pull request #4100 from morimoto-cybozu/clarify-errors-in-clusterd
Return explicit errors
2019-10-15 08:57:06 -06:00
Sébastien Han ff4b31ff0a Merge pull request #4038 from travisn/sanitize-master-tag
Sanitize the master tag
2019-10-15 11:38:22 +02:00
Sébastien Han 12d69e6d0c Merge pull request #4096 from travisn/run-min-test-matrix
Run the distributed test matrix in PRs
2019-10-15 10:34:23 +02:00
morimoto-cybozu 815927d943 clusterd: return explicit errors
PopulateDeviceInfo() in pkg/clusterd/disk.go returns nil as *sys.LocalDisk
if an error has occurred.  This causes a nil pointer exception at
getAvailableDevices() in pkg/daemon/ceph/osd/daemon.go.  At least the returned
value should be checked at the caller.
I added an explicit error to the return value of PopulateDeviceInfo().  This
naturally revokes an error check at the caller.
I modified PopulateDeviceUdevInfo() in the same file too.

Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
2019-10-15 08:30:45 +00:00
Travis Nielsen 8c6c211ab0 Merge pull request #4090 from travisn/owner-ref-cleanup
OwnerReference and test pod cleanup
2019-10-14 13:47:53 -06:00
travisn 1d97a6089d build: sanitize the master tag
The master branch uses the first tag from the last release
to create the version tag. Since the 1.1 release this tag
can include the alpha, beta, or rc tags, which only make
sense for the release tags, but not the master tag. This
change removes the suffix from the master tag.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 10:58:51 -06:00
travisn ac8f0dff09 tests: by default run the minimum test matrix in PRs
The CI runs the integration tests on several versions of K8s. By default
the integration tests have all been running on all versions of K8s. With the
recent option to run a minimum matrix for ceph tests it has shown to be reliable
for PRs and is much faster to have the test distribution.

Master and release builds will still run the full set of test suites on all
versions of K8s.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 10:23:04 -06:00
travisn bc68440006 ceph: more robust removal of cluster finalizer
The cluster finalizer in some scenarios was not being removed
during cluster removal. Specifically, if the cluster CR had been
modified, the operator would always fail to remove the finalizer.
This could occur when the CR status is updated around the same time
that the cluster is deleted. Therefore, we need to remove the finalizer
from a freshly retrieved instance of the cluster.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:46 -06:00
travisn d1680b2cca tests: ensure cluster finalizer removed during cleanup
The tests must ensure that the cluster is cleaned up so that
other test suites will not be affected by cleanup of a previous
test suite. Removing the cluster finalizer is critical to this
cleanup.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:46 -06:00
travisn 2b3d1f45ef tests: consolidate the filesystem tests
There are two separate integration tests for file. To simplify
and streamline and potentially eliminate some random failures,
these tests now consume the same file system instead of creating
a separate file system.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:45 -06:00
travisn a92850b307 tests: simplify filesystem tests and client pod deletion
The filesystem test deletes the client test pod by deleting the entire yaml
and sometimes hangs the integration tests. This is the most common intermittent
error in the CI. Now the deletion will happen more directly with a pod delete
command. If it does still fail deletion at least the deletion should timeout
sooner.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:45 -06:00
travisn b0a3bea421 ownerrefs: set owners based on name instead of namespace
The owner references should be set based on the name, otherwise the
garbage collector will delete the resources at some point.
These references to yugabyte, minio, and cockroach were
missed in the original fix for these references.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:45 -06:00
travisn 5db209fc5b ceph: allow resources to be deleted via owner references
The resources for rgw, mds, and nfs were being cleaned up explicitly when
the CR was deleted. This is legacy from before the owner references were
set on the CRs. Now the owner references are set to the appropriate CRs so
the resources will be cleaned up upon deletion of the specific CR
rather than the cluster CR.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:45 -06:00
travisn 22391377a8 ceph: remove full path from vgchange call
The full path to vgchange should not be necessary. For some
downstream releases the path may even be in a different path.
Therefore we require that vgchange be in the path.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:45 -06:00
travisn f4d8ad74af version: print the golang version with the rook version
The version of golang used to build rook is helpful
troubleshooting information. The rook version command will
now print the version of golang.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-14 09:29:44 -06:00
Sébastien Han 43b5a9692e Merge pull request #4070 from Madhu-1/fix-cephfs-reg
csi: update cephfs registration directory name
2019-10-11 12:14:15 +02:00
Sébastien Han ee5c02c7c3 Merge pull request #4081 from binoue/fix-links
doc: update outdated links
2019-10-11 11:20:07 +02:00
binoue 170f45cd42 doc: update wrong/outdated links
this fix not working links

Signed-off-by: binoue <banji-inoue@cybozu.co.jp>
2019-10-11 18:11:49 +09:00
Sébastien Han d7956facdb Merge pull request #4075 from rohantmp/mdsFix
[Ceph] Fix: PDB for MDS not created when activeStandby:true and activeCount: 1
2019-10-11 11:02:12 +02:00
Travis Nielsen ec75db503f Merge pull request #4024 from thotz/brownfield
Fixing issues for object bucket claims for brownfield cases (delete policy)
2019-10-10 15:48:35 -06:00
Rohan CJ 0d9442982b ceph: Fix PDB for MDS is not created if minAvailable <= 1.
It should be minAvailable < 1.

Signed-off-by: Rohan CJ <rohantmp@gmail.com>
2019-10-10 21:34:20 +05:30
Rohan CJ 3c5856fc2a Ceph: Take activeStandby into account for CephFileSystem disruption budget.
When activeStandby is set on the CephFilesystem, the effective
activeCount will be incremented.

Signed-off-by: Rohan CJ <rohantmp@gmail.com>
2019-10-10 21:31:58 +05:30
Sébastien Han 776f3951dc Merge pull request #4072 from leseb/mergify-rebase
mergify: change config to allow auto-merging
2019-10-10 17:36:19 +02:00
Sébastien Han e17d22a363 mergify: change config to allow auto-merging
Let's not do a rebase, but only merge. Rook is configured to refuse the
rebase and mergify fails with the following Github error:

GitHub error message: Rebase merges are not allowed on this repository.

This commit fixes the error.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-10 17:29:43 +02:00
Jiffin Tony Thottan 3f3af19a13 objectstore: remove user unlink in Revoke()
The access of user will be revoked by Droppolicy(). And user unlink
need to perform only if it is the owner. For Delete() it makes sense
to call user unlink, but it is not required for Revoke()

Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2019-10-10 19:21:45 +05:30
Jiffin Tony Thottan 69e4144b0f objectstore: modify existing bucket policy for brownfield cases
Following corrections are needed for bucket policy to work properly

* Change Principal:{"AWS":["<username>"]} to Principal:{"AWS":["arn:aws:iam:::user/<username>"]}
* Add missing "PutObject" policy in Action[]
* Add <bucket/*> to Resource[] for accessing contents inside the bucket

Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
2019-10-10 19:21:17 +05:30
Sébastien Han 28a898d9f3 Merge pull request #4046 from eknudtson/osd-memory-target-fix
remove decimal point for osdMemoryTargetValue
2019-10-10 09:27:50 +02:00
Madhu Rajanna 0056e58ba6 csi: update cephfs registration directory name
when multiple CSI drivers are running in a cluster,
each should use unique socket for registration.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-10 10:52:27 +05:30
Travis Nielsen 3c26fb1096 Merge pull request #4025 from leseb/external-more-flexible
ceph: relax pre-requisite for external cluster
2019-10-09 13:36:19 -06:00
Sébastien Han 148f8da1fb ceph: relax pre-requisite for external cluster
We now differentiate the cases where:

* we only consume the external cluster
* we consume the external cluster as well as creating stateless
resources in Kubernetes (bootstrap mds,rgw, nfs)

This is mostly controlled via the image property spec. If not defined,
not extra CRs won't be able to be created.

Now the external cluster feature supports Ceph cluster as of Luminous 12.2.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-08 10:52:01 +02:00
Erik KnudtsonandSébastien Han ec4d0c8925 ceph: remove decimal places for osdMemoryTargetValue
Update pkg/operator/ceph/cluster/osd/spec.go

Rook v1.1.2 uses a different command to start an OSD, and seems to
have problems when --osd-memory-target contains a number with a
decimal point.

Co-Authored-By: Sébastien Han <seb@redhat.com>
Signed-off-by: Erik Knudtson <eknudtson@discogsinc.com>
2019-10-07 09:32:49 -07:00
Sébastien Han 233df963e7 ceph: fix various spelling issues
Correct spelling.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-03 11:42:40 +02:00
Sébastien Han 9b0c73ca6b Merge pull request #3817 from mantissahz/ceph_osd_nodes_timeout
Nodes always in OrchestrationStatusStarting
2019-10-03 11:07:19 +02:00
James Lu b8d7382b7a Ceph: nodes always in OrchestrationStatusStarting
When osd nodes do not finish the prepare jobs, the user remove the
nodes manully and it causes nodes always is
'OrchestrationStatusStarting' status.
Try to remove them after completeProvisionTimeout.

Signed-off-by: James Lu <jamesluhz@gmail.com>
2019-10-03 08:08:46 +08:00
Travis Nielsen b40321fa1f Merge pull request #4032 from travisn/no-hyperkube-helm
Remove obsolete hyperkube settings from the helm chart
2019-10-02 09:16:37 -06:00
travisn 66e7562045 ceph: no hyperkube settings in the helm chart
In the original help chart hyperkube was used in the prototype.
The usage of hyperkube has long since been removed from the
chart. This is cleanup to remove the reference to hyperkube
in the chart settings.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-01 17:45:57 -06:00
Travis Nielsen 102923cd18 Merge pull request #4030 from rohan47/ceph-helm
Updated rook-ceph-osd clusterrole for helm chart
2019-10-01 16:35:20 -06:00
rohan47 47dc6c0347 Updated rook-ceph-osd clusterrole for helm chart
Signed-off-by: rohan47 <rohgupta@redhat.com>
2019-10-02 01:46:39 +05:30
Sébastien Han b400a09317 Merge pull request #4026 from varshar16/wip-turn-on-ssl
ceph: set ssl to true in cluster-test.yaml
2019-10-01 17:52:02 +02:00
Varsha Rao a83939319d ceph: Set ssl to true in cluster-test.yaml and cluster-on-pvc.yaml
Dashboard requires ssl to be turned on for running backend requests in rook.

Signed-off-by: Varsha Rao <varao@redhat.com>
2019-10-01 21:11:49 +05:30
Sébastien Han 418cbb9464 Merge pull request #4018 from SUSE/var-csi-images
ceph: set default Ceph CSI images as var not const
2019-10-01 09:39:06 +02:00
Sébastien Han 6f4a7cf6c8 Merge pull request #3974 from jmolmo/issue_2206
New setting to avoid delete pools on <filesystem>/<object store> deletion
2019-10-01 09:19:56 +02:00
Travis Nielsen 610de42cbb Merge pull request #4016 from jtlayton/wip-42089
ceph: set ssl to true in cluster-minimal.yaml
2019-09-30 14:56:32 -06:00
Blaine Gardner 24d2187f1a ceph: set default Ceph CSI images as var not const
Set the default Ceph CSI images as vars in the code instead of consts.
This allows these values to be overridden at build time with Go linker
-X flags. This allows users to build Rook in such a way that it will
automatically update the CSI images to a custom opinionated default
without having to manually manage the environment variable-based
overrides at upgrade time.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-09-30 12:15:32 -06:00
Jeff Layton b5a5a65666 ceph: set ssl to true in cluster-minimal.yaml
The scripts for running backend dashboard requests in rook expect this
to be an SSL-enabled port. Ensure that it is.

https://tracker.ceph.com/issues/42089
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2019-09-30 10:58:38 -04:00
Juan Miguel Olmo Martínez 2de0787fc8 New setting to avoid delete pools on <fs>/<os> resources deletion
A new CRD property `PreservePoolsOnDelete` has been added to Filesystem(fs) and
Object Store(os) resources in order to increase protection against data loss.
If it is set to `true`, associated pools won't be deleted when the main
resource(fs/os) is deleted. Creating again the deleted fs/os with the same name
 will reuse the preserved pools.

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@redhat.com>
2019-09-30 08:13:39 +02:00
Travis Nielsen 91b8a96fb6 Merge pull request #3994 from leseb/rm-osd-config
Ceph: fix rook-config-overrides for OSDs
2019-09-27 10:07:41 -06:00
Sébastien Han f1e57a5d3f Merge pull request #4005 from Madhu-1/fix-3987
CSI: Make metrics and liveness port configurable
2019-09-27 16:41:48 +02:00
Sébastien Han c8d8d30103 Merge pull request #3943 from galexrt/updated_disaster
docs: updated Ceph mon disaster recovery guide
2019-09-27 15:40:20 +02:00