Commit Graph
7961 Commits
Author SHA1 Message Date
kesavan 453200045e ceph: adding deviceclasses info to cephcluster cr
The admin needs to start the toolbox and run a command such as ceph osd tree to see the device classes for each OSD. This PR simplifies it by storing the deviceclass info in the CephCluster CR status

closes: https://github.com/rook/rook/issues/5754
Signed-off-by: kesavan <kvellalo@redhat.com>
2020-07-22 17:46:51 +05:30
Alexander Trost 29e51cb1e9 cassandra: use latest 3.11.6 version in examples
This also updates the cassandra version in the tests.
This is to provide safe and secure defaults to users running Cassandra.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-07-22 13:18:43 +02:00
Sébastien Han 907c8bf85f Merge pull request #5719 from alimaredia/wip-object-store-multisite-enhancements
ceph: check for adding endpoints into zone group
2020-07-22 11:19:15 +02:00
Madhu Rajanna 73e1ee41c3 ceph: remove unwanted metadatastore and mountcache flag
metadatastore and mountcache are required for cephcsi v1.0.0

as rook doesnot used ceph-csi v1.+ version
we can remove this flag. Removing these flags doesnot
affect the functionality of anything.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-22 14:36:11 +05:30
Sébastien Han 68bcc18a1c Merge pull request #5853 from cybozu-go/fix-parameter-fields-of-crds
ceph: fix the parameters fields of cephfilesystem crd and cephblockpool crd
2020-07-22 09:11:47 +02:00
Travis Nielsen e4026560c5 Merge pull request #5860 from travisn/edgefs-security-review
Error checking for EdgeFS related to the security review
2020-07-21 20:54:57 -06:00
Ali Maredia e6ed4ff8ea ceph: minor fixes + add realm/zg/zone to object context
- Add realm/zone group/zone to Object Context,
so that any call to runAdminCommand has the
same realm, zone group, and zone as the object
store that the call is going to.

- Also modify the delete code for the object-store
using multisite to remove the endpoints of an
object store from a zone instead of the normal
clean-up

- Added more debug logging all around the object-store
code related to multisite

- files generated by rerun of `make codegen`

- change back the edit on the Copyright in object.go

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-21 16:47:59 -04:00
Ali Maredia 9dd0ea894a ceph: documentation for pulling a realm for multisite
This commit adds design revisions, examples, and
documentation for object multisite.

Changes to the CephObjectRealm to pull a realm
from another cluster, and the zone for creating
pools are amongst the larger changes

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-21 16:46:18 -04:00
Travis Nielsen eaca35e7af Merge pull request #5855 from caffeinism/patch-2
ceph: change command to add 3300 port for monmap
2020-07-21 13:12:56 -06:00
Ali Maredia 0f7e55780c ceph: add pool creation for ceph-object-zone
When zones are created the ceph RGWs inside the
those zones should be using pools with the zones
name not the object-store's name

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-21 13:30:41 -04:00
Ali Maredia 5d9c2f09a9 ceph: enable the realm pull on ceph-object-realm
This commit:
- adds the pull section on the CephObjectRealm spec
to enables realms to be pulled instead of created.

- creates the system user and generates the access
key and secret key for the user and zones in a realm.

- adds "omitempty" to fields in multisite CRs that
are not required

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-21 13:30:19 -04:00
Sébastien Han 632818bab9 Merge pull request #5861 from SUSE/docs-fix-requiresafereplicasize-placement
docs: fix placement of param in Ceph Pool CRD
2020-07-21 17:43:20 +02:00
Blaine Gardner 0a08480c3a docs: fix placement of param in ceph pool crd
`requireSafeReplicaSize` is now a sub-param of `replicated` and not of
Ceph's `parameters`.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2020-07-21 09:37:59 -06:00
Sébastien Han 0a6cce43fb ceph: add clusterResourceDeleted flag
During the deletion sequence of the CR, we now set a success flag at the
end, right before removing the finalizer.
This avoids the case where we fail to remove the finalizer.

2020-07-21 13:29:46.334816 E | ceph-object-controller: failed to reconcile failed to remove finalizer: failed to remove finalizer "cephobjectstore.ceph.rook.io" on "teststore": Operation cannot be fulfilled on cephobjectstores.ceph.rook.io "teststore": the object has been modified; please apply your changes to the latest version and try again

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-21 17:11:29 +02:00
Sébastien Han 698f46e978 ci: purge after flexsuite
If we don't the mgr tests cannot verify the osd creation and will fail
with:

```
--- FAIL: TestCephMgrSuite (169.32s)
    --- FAIL: TestCephMgrSuite/TestCreateOSD (1.65s)
        ceph_mgr_test.go:217:
            	Error Trace:	ceph_mgr_test.go:217
            	Error:      	Should not be: ""
            	Test:       	TestCephMgrSuite/TestCreateOSD
            	Messages:   	No devices available to create test OSD
        ceph_mgr_test.go:218:
            	Error Trace:	ceph_mgr_test.go:218
            	Error:      	Should not be: ""
            	Test:       	TestCephMgrSuite/TestCreateOSD
            	Messages:   	No nodes available to create test OSD
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-21 17:10:28 +02:00
Sébastien Han bc4875a5eb ci: only apply cleanup policy on converged cluster
There is no point of asking for a cleanup when the cluster is external,
also the cleanup pod will never exist and the CI will fail waiting for
it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-21 17:10:28 +02:00
Sébastien Han 6444675d4d ci: debug cleanup job logs
Add logs of the cleanup jobs on success.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-21 17:10:26 +02:00
Sébastien Han 10ed662bdb ceph: enhancement disk sanitize options
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisk which holds more details:

* method: indicates if the entire disk should be sanitized or simply ceph's metadata.
Possible choices are 'complete' or 'quick' (default)
* dataSource: indicate where to get random bytes from to write on the disk.
Possible choices are 'zero' (default) or 'random'
Using random sources will consume entropy from the system and will take much more time then the zero source
* iteration: overwrite N times instead of the default (1). Takes an integer value

See the documentation for more details.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-21 14:07:11 +02:00
Hiroshi Muraoka 478ee124ee ceph: run make codegen
Signed-off-by: Hiroshi Muraoka <h.muraoka714@gmail.com>
2020-07-21 10:34:54 +00:00
Hiroshi Muraoka 5f4e9cf38b ceph: fix the parameters fields of crds
CephBlockPool: remove the duplicated field
CephFilesystem: change the nesting level

Related to #5779

Signed-off-by: Hiroshi Muraoka <h.muraoka714@gmail.com>
2020-07-21 10:34:54 +00:00
Sébastien Han a343c597f6 Merge pull request #5857 from travisn/more-clusterinfo
ceph: Additional clusterInfo cleanup for user creds
2020-07-21 12:16:44 +02:00
Sébastien Han eda37f0f25 Merge pull request #5859 from sfynx/master
ceph: Default .Values.agent.mountSecurityMode to empty string.
2020-07-21 12:14:43 +02:00
Sébastien Han 6e99c0933f Merge pull request #5796 from rajatsingh25aug/finalizer-deletion
Ceph: Delete Filesystem only when it exists
2020-07-21 12:13:41 +02:00
Sébastien Han c9657e69e7 Merge pull request #5856 from leseb/report-error
ceph: report error on timeout
2020-07-21 09:14:45 +02:00
Henno Schooljan c47180e37c ceph: default .Values.agent.mountSecurityMode to empty string
Signed-off-by: Henno Schooljan <github@sfynx.nl>
2020-07-21 01:13:36 +02:00
Travis Nielsen d06b18067b edgefs: simplify string formatting for labels
The label generated to apply to a node are simplified with the
backtick string delimeter to avoid the escaping of quotes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-20 17:11:38 -06:00
Travis Nielsen 710b3e8b19 ceph: remove obsolete log method
The ClusterInfo does not use the Log method to write the fields
anymore. The formatting was not necessary since the struct can
be logged if needed with the standard %v golang formatting.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-20 17:11:37 -06:00
Travis Nielsen 7bef54c089 edgefs: error checking for ineffectual assignments
All assignments made to errors or other variables should be
used instead of ignored. This checks for errors in a couple
places that were previously ignored, and also initializes
a variable such that it won't be always overwritten by the
various if else statements.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-20 17:11:37 -06:00
Travis Nielsen e7998b3c00 ceph: additional clusterInfo cleanup for user creds
The user creds should not be passed to the ceph helper methods
since the clusterInfo already has the needed context. Removing
a final reference that was missed in the refactor.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-20 16:26:20 -06:00
Travis Nielsen 14d4ed447e Merge pull request #5845 from leseb/external-clusterinfo
ceph: re-hydrate clusterInfo for external mode
2020-07-20 16:20:50 -06:00
Travis Nielsen 7c65df506f Merge pull request #5852 from yanniszark/feature-cassandra-update-jolokia-lib
cassandra: Update Jolokia lib in Cassandra sidecar image
2020-07-20 15:17:56 -06:00
RAJAT SINGH e3c0d2530d ceph: delete filesystem only when it exists
ceph-file-controller gets stuck in loops, this happens when it tires to delete the filesystem which doesn't exist.Filesystem should only be deleted when it exist.

Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
2020-07-21 01:58:11 +05:30
Sébastien Han d98e49f5b3 ceph: re-hydrate clusterInfo for external mode
Whether we run an external cluster or not we need to load the
appropriate clusterInfo.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-20 19:24:54 +02:00
subhamkrai 23f0ac7f9c ceph: ability to configure livenessprobe for cephobjectstore crd
this commit allows us to disable livenessprobe for
cephobjectstore crd.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-07-20 22:53:32 +05:30
Sébastien Han 97a029e691 Merge pull request #5844 from vbnrh/ci-issue-admission
ceph:  disables admission controller tests for k8 v1.14 and older.
2020-07-20 17:48:54 +02:00
Sébastien Han 2c48357f8b ceph: fix error message on empty buffer
Do not print the empty buffer to get a clean error message.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-20 17:41:49 +02:00
Sébastien Han 954074b707 ceph: report error on timeout
If the healthcheck goroutine cannot check for the ceph status/times out
(if the mons are not in quorum) this should still be reflected in the
CRD status field.

Example:

```
status:
  ceph:
    details:
      error:
        message: 'failed to get status. . timed out: exit status 1'
        severity: Urgent
    health: HEALTH_ERR
    lastChanged: "2020-07-20T14:11:05Z"
    lastChecked: "2020-07-20T14:11:05Z"
    previousHealth: HEALTH_WARN
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-20 16:15:08 +02:00
Vineet Badrinath f0bb6b11bd ceph: disables admission controller tests for k8 v1.14 and older
Adds checks to see whether the k8s version is greater than v1.15 in integration tests.
when true, it will run the scripts for the admission controllers.

Closes: https://github.com/rook/rook/issues/5841
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
2020-07-20 19:36:17 +05:30
Minjong Kim 7ac0ce6873 ceph: change command to add 3300 port for monmap
In the latest version of rook-ceph, disaster recovery does not work properly with the following command. Therefore, the command was updated to the latest command.

Closes: https://github.com/rook/rook/issues/5662
Closes: https://github.com/rook/rook/issues/5649
Signed-off-by: caffeinism <make.dirty.code@gmail.com>
2020-07-20 11:55:10 +00:00
Sébastien Han 46e64a1d16 Merge pull request #5633 from leseb/fail-if-no-lvm
ceph: fail prepare pod if lvm2 pkg is missing
2020-07-20 10:29:13 +02:00
Sébastien Han f3c4975e35 ceph: fail prepare pod if lvm2 pkg is missing
A long-standing issue that was only solved via a documentation note. Now,
during the prepare pod instantiation, we check for the presence of the
lvm binary on the host. Success will indicate that we can bootstrap
OSDs where failure will refuse to prepare the OSD.

Closes: https://github.com/rook/rook/issues/5627
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-07-20 10:05:05 +02:00
Ahmad Nurus S 22574ac965 docs: update nfs openshift scc documentation
Signed-off-by: Ahmad Nurus S <prksu.sh@gmail.com>
2020-07-20 10:58:34 +07:00
Yannis Zarkadas bb24d8efa0 cassandra: update jolokia lib in operator sidecar image
The Cassandra sidecar uses the Jolokia javaagent to expose a REST API of
Cassandra's administrative interface, which is normally only exposed via
JMX, a Java-only RPC protocol. Update the Jolokia javaagent to a newer
version, containing new features and security bug fixes.

Signed-off-by: Yannis Zarkadas <yanniszark@arrikto.com>
2020-07-20 01:05:57 +03:00
Blaine Gardner 1d2d748ff3 Merge pull request #5825 from SUSE/fix-5056
ceph: support device names with special chars
2020-07-17 16:38:48 -06:00
Travis Nielsen a1880e6589 Merge pull request #5849 from travisn/target-size-string
ceph: Pool parameters must be strings
2020-07-17 12:24:21 -06:00
Ali Maredia 48ef107271 ceph: fix names of variables for requeue times
Name for requeue time variables were the same for
the object-zone and the object-zonegroup.

This commit makes them more descriptive and
accurate

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-17 11:32:31 -06:00
Ali Maredia f29c650ce1 ceph: allow creation for non-master zonegroups/zones
The masterArg flag cannot be "" so a different
command without masterArg must be run for zone
and zone group creation

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-17 11:32:31 -06:00
Ali Maredia 49e6796b90 ceph: allow multiple object-stores in a zone
Make it so that zones & zone groups can be
modified with more than one endpoint so that
more than one object store can be in a zone

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-17 11:32:31 -06:00
Ali Maredia 1efb3d4bd1 ceph: check for adding endpoints into zone group
Endpoints can only be added to a zone group if the
zone the endpoint is part of is master.

Signed-off-by: Ali Maredia <amaredia@redhat.com>
2020-07-17 11:32:28 -06:00
Travis Nielsen 5be965c1c3 ceph: target_size_ratio must be a string instead of float
The pool parameters must be strings. In the pool examples now we add
quotes around the sample target_size_ratio to avoid deserialization
errors.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-07-17 11:16:00 -06:00