The admin needs to start the toolbox and run a command such as ceph osd tree to see the device classes for each OSD. This PR simplifies it by storing the deviceclass info in the CephCluster CR status
closes: https://github.com/rook/rook/issues/5754
Signed-off-by: kesavan <kvellalo@redhat.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisk which holds more details:
* method: indicates if the entire disk should be sanitized or simply ceph's metadata.
Possible choices are 'complete' or 'quick' (default)
* dataSource: indicate where to get random bytes from to write on the disk.
Possible choices are 'zero' (default) or 'random'
Using random sources will consume entropy from the system and will take much more time then the zero source
* iteration: overwrite N times instead of the default (1). Takes an integer value
See the documentation for more details.
Signed-off-by: Sébastien Han <seb@redhat.com>
The ClusterInfo does not use the Log method to write the fields
anymore. The formatting was not necessary since the struct can
be logged if needed with the standard %v golang formatting.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The user creds should not be passed to the ceph helper methods
since the clusterInfo already has the needed context. Removing
a final reference that was missed in the refactor.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph-file-controller gets stuck in loops, this happens when it tires to delete the filesystem which doesn't exist.Filesystem should only be deleted when it exist.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
If the healthcheck goroutine cannot check for the ceph status/times out
(if the mons are not in quorum) this should still be reflected in the
CRD status field.
Example:
```
status:
ceph:
details:
error:
message: 'failed to get status. . timed out: exit status 1'
severity: Urgent
health: HEALTH_ERR
lastChanged: "2020-07-20T14:11:05Z"
lastChecked: "2020-07-20T14:11:05Z"
previousHealth: HEALTH_WARN
```
Signed-off-by: Sébastien Han <seb@redhat.com>
A long-standing issue that was only solved via a documentation note. Now,
during the prepare pod instantiation, we check for the presence of the
lvm binary on the host. Success will indicate that we can bootstrap
OSDs where failure will refuse to prepare the OSD.
Closes: https://github.com/rook/rook/issues/5627
Signed-off-by: Sébastien Han <seb@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph config and client packages are conceptually the same.
To avoid circular dependencies in some cases, we simplify by combining
the packages into the client package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator should only connect to ceph with a single set of creds.
In a converged cluster this will be the admin creds and in an external
cluster it will be lower-privileged creds. Independent clusters were
implemented with a separate set of creds. To simplify the code these
are now merged to a single set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add the ability to provision Ceph OSDs with Drive Groups.
This adds Drive Groups to the CephCluster CRD, and it sets code
in place for propagating this config to the OSD provisioning pod.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
moves the admission-controller server to use controller-runtime library for better support, maintainibility of code.
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
adds deploy.sh script to deploy validatingwebhookconfiguration and create secrets.
adds new command ceph admission-controller to start webhook servers.
adds validation for various rook custom resources
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
Fix the following bugs.
- CephVersion is not back to Octopus
- `cephadm' here is called with three arguments,
"info --query=property <devname>". We should use 'args[2]' instead.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
We can now explicitly set any property on a given pool by using the new
Property field in the CephBlockPool Spec.
Also, this fixes the case where both `CephBlockPool` and `CephCluster`
are created at the same time. When Rook creates the pool, the cluster is
still being bootstrapped and the global option
`osd_pool_default_pg_autoscale_mode` has not bee set yet. So the pool
gets created but its `pg_autoscale_mode` property is set to `warn`
instead of `on`.
Closes: https://github.com/rook/rook/issues/5608V
Signed-off-by: Sébastien Han <seb@redhat.com>
Fix the info log statement to log the discovered devices as their string
representation instead of pointers.
This changes the info log to only be the device names and the debug log
to be the whole disk struct as well.
Resolves#5625
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
When running Octopus, we still need to turn on the balancer module. The
distinction is tricky, the module is "enabled" but it has to be "turned
on" and configured.
This was only working when the `pg_autoscaler` module was explicitly
listed in the CephCluster spec mgr section module section but not
automatically configured when running Octopus (the `pg_autoscaler`
module is already enabled).
Signed-off-by: Sébastien Han <seb@redhat.com>
If the cluster has 3 OSDs, during the upgrade when the OSD restarts it
will temporarily disappear from the "ceph versions" list so we need a
more robust way to list the number of OSDs and always returns the total
number of OSDs.
Closes: https://github.com/rook/rook/issues/5521
Signed-off-by: Sébastien Han <seb@redhat.com>
Even if we cannot determine wheter all the OSDs are running on the host,
we still need to perform the ok-to-stop check. It is far more common to
have more than 3 OSDs anyways.
Signed-off-by: Sébastien Han <seb@redhat.com>
Updated initializeBlockPVC method to add
crushDeviceClass along with
immediateExecuteArgs instead of
metadataArg. Also added unit tests for this method.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisks which holds more details:
* confirmation: the confirmation message to sanitize disks,
use "yes-really-sanitize-disks" to confirm
This will **only** wipe the metadata, so it's a fast cleanup allowing
you to re-install later but won't remove all the data from the drive.
Signed-off-by: Sébastien Han <seb@redhat.com>
The Ceph version information is not needed to be passed
to the osd health checker since mimic support was dropped.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The activation of the lvs for OSDs had an invalid
argument to vgchange. The invalid argument is now reverted
to enable the OSDs in this scenario again.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
For multipath devices, ceph-volume inventory is failing
to detect that the device is available, complaining about
insufficient space. Instead of relying on c-v inventory
to determine if the device is available, just call
c-v raw list to determine if there is already an osd
configured on the pv. Otherwise, just attempt to configure
the OSD.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph mds deactivate is not valid command for Nautilus, changing the mechnaism of upgrading so that it can be done in the way Nautilus does it.
Closes: https://github.com/rook/rook/issues/5278
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
If we fail to release an LVM device, we should not fail the deployment.
We just print the error and let the backend handle the rest. Some
CSI backend will force detach the device anyway.
This prevents the deployment to get stuck in terminating.
Signed-off-by: Sébastien Han <seb@redhat.com>
During the sgdisk command test, there was no output assignment,
causing uuid not found for device /dev/testa. Follow the log.
2020-04-27 18:24:21.921337 I | rook-discover: RUN Command lsblk
[--all --noheadings --list --output KNAME]
2020-04-27 18:24:21.921491 I | rook-discover: RUN Command lsblk
[/dev/testa --bytes --nodeps --pairs --paths --output SIZE,ROTA,
RO,TYPE,PKNAME,NAME]
2020-04-27 18:24:21.921549 I | rook-discover: RUN Command sgdisk
[--print /dev/testa]
2020-04-27 18:24:21.921576 W | inventory: skipping device "testa"
because 'lsblk' failed. uuid not found for device /dev/testa. output=
2020-04-27 18:24:21.921592 I | rook-discover: localdevices: []
2020-04-27 18:24:21.921598 I | rook-discover: available devices: []
--- FAIL: TestProbeDevices (0.00s)
discover_test.go:95:
Error Trace: discover_test.go:95
Error: Not equal:
expected: 1
actual : 0
Test: TestProbeDevices
panic: runtime error: index out of range [0] with length 0 [recovered]
panic: runtime error: index out of range [0] with length 0
FAIL github.com/rook/rook/pkg/daemon/discover 0.050s
Signed-off-by: Haichao Li <haichao.li@arm.com>
This is the final conversion to controller-runtime conversion. This time the
CephCluster CRD has been converted to use the controller-runtime
library.
The controller incorporates all the previous watchers too, so the Node
and hot-plug configmap are been watched too.
Only the operator setting configmap is not being watcher since it's not
related to the CephCluster CRD.
Not only the patch converts to controller-runtime but also tries to
re-organize the tree of the repo to actually make the code more
readable and have better functions/methods/tests separations.
Closes: https://github.com/rook/rook/issues/4939
Signed-off-by: Sébastien Han <seb@redhat.com>
When the metadataDevice is specified, it was a regression that it was
being interpreted also as a data device. ceph-volume was failing
when the metadataDevice was added as both a data and metadata device.
So we ensure it is not added to the list of data devices.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Currently, we need to configure the Ceph external Admin keyi
in the Rook deployment to be able to connect to an external Ceph cluster.
If we wanted to run in a multi-tenant fashion
were several K8s/Rook clusters wanted to connect to the same external ceph cluster,
each K8s deployment would have the access to the External Ceph Admin key
and could potentially access or delete the Data
from pools that belong to other k8s/Rook Clusters.
Now the admin key is optional but the helper script create-external-cluster-resources.sh
will help create the necessary keys/users to connect to that cluster.
Closes: https://github.com/rook/rook/issues/4917 and https://github.com/rook/rook/pull/5227
Signed-off-by: Sébastien Han <seb@redhat.com>
Add size parameter to pool create command instead of executing pool set command only for size
This makes pool min_size reflect replica size.
Co-authored-by: donggyu_park <donggyu_park@tmax.co.kr>
Signed-off-by: taeuk_kim <taeuk_kim@tmax.co.kr>
unhandled type assertions can lead to runtime panic. using
the boolean 'ok' flag to handle failure to assert type.
Issue: #4566
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
subhamkrai
Travis Nielsen
Sébastien Han
kesavan
RAJAT SINGH
Blaine Gardner
Vineet Badrinath
Satoru Takeuchi
Alexander Trost
Madhu Rajanna
Haichao Li
taeuk_kimanddonggyu_park
Umanga Chapagain
ShyamsundarR