When c-v runs after the end of the prepare sequence, it will look for
lvm and raw-based OSDs. The raw mode detects LVM OSDs so we must trim
the list of OSD, only to report the LVM ones if their ID exists.
Closes: https://github.com/rook/rook/issues/7359
Signed-off-by: Sébastien Han <seb@redhat.com>
this commit adds new crd to admission webhooks to check
if one of the port or securePort are set to non-zero
value in objectstore crd.
If both are set to 0 in crd we'll see error like
```
Error from server (invalid create: either of port or securePort
fields should be not be zero): error when creating "object-test.yaml":
admission webhook
"cephobjectstore-wh-rook-ceph-admission-controller-rook-ceph.rook.io"
denied the request: invalid create: either of port or securePort fields
should be not be zero.
```
Signed-off-by: subhamkrai <srai@redhat.com>
For simple OSD scenario that don't require the following:
* an encrypted osd
* an osd with a backing device
* multiple osd on the same block device
The OSD will be prepared using the ceph-volume raw mode which does
not use LVM on non-pvc case.
Closes: https://github.com/rook/rook/issues/4768
Signed-off-by: Sébastien Han <seb@redhat.com>
block if not defined was used with osdInfo.Device so not empty anymore
in the loop, which means it will have the same value for all OSDs in the
loop.
This is only called when running on PVC, where a single device is
present so the loop has a single iteration, so not a big issue.
Signed-off-by: Sébastien Han <seb@redhat.com>
The conditions on the cephcluster CR were being set to false
when not in progress, which is misleading to the purpose of
conditions. The conditions will now always represent current
state of the cluster. Transient conditions that show progress of
a reconcile will be removed from the status.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The upcoming Rook release 1.6 will support Ceph Pacific which introduces
stable support for multiple Ceph Filesystems in the same cluster. So we
now allow the creation of multiple Filesystem if the release is Pacific.
Also, we remove the Operator config flag which was not necessary.
Here I have 2 filesystems:
```
[root@rook-ceph-tools-6b4889fdfd-vncck /]# ceph fs ls
name: myfs, metadata pool: myfs-metadata, data pools: [myfs-data0 ]
name: myfs2, metadata pool: myfs2-metadata, data pools: [myfs2-data0 ]
```
```
cluster:
id: cc747950-589a-4862-bad2-ff7de1217b1a
health: HEALTH_WARN
mons a,b,c are low on available space
6 pool(s) have no replicas configured
services:
mon: 3 daemons, quorum a,b,c (age 2d)
mgr: a(active, since 2d)
mds: myfs:1 myfs2:1 {myfs2:0=myfs2-b=up:active,myfs:0=myfs-b=up:active} 2 up:standby-replay
osd: 3 osds: 3 up (since 2m), 3 in (since 6d)
data:
pools: 6 pools, 717 pgs
objects: 44 objects, 4.2 KiB
usage: 46 MiB used, 90 GiB / 90 GiB avail
pgs: 717 active+clean
io:
client: 1.8 KiB/s rd, 4 op/s rd, 0 op/s wr
progress:
Global Recovery Event (45s)
[===========================.]
```
Signed-off-by: Sébastien Han <seb@redhat.com>
If there is a warning coming from the Vault server we must catch it and
exit immediately to reflect it if no key was retrieved.
Signed-off-by: Sébastien Han <seb@redhat.com>
The livenessprobe for swift/health always check with RGW internal port if host network disabled,
but won't work if only secure port is configured
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
The vault env variable VAULT_BACKEND is read by the secret library so we
must not change it. This is reverting
ee6a03dfb8 with a few extra fixes for the
rgw code.
Signed-off-by: Sébastien Han <seb@redhat.com>
Do not reconcile ServiceMonitor/PrometheusRule on label updates.
This allows users to add custom labels which can be used by
custom Prometheus instances for monitoring.
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
Remove features and design that supports adding OSDs to Ceph clusters
via `spec:driveGroups`. Update the Ceph upgrade doc that informs users
who currently use Drive Groups (we believe there are none of these
users) how to migrate to using the `spec:storage` config.
Resolves https://github.com/rook/rook/issues/7275
Revert "ceph: fix drive group deployment failure"
This reverts commit 76f1d9944e.
Revert "ceph: osd: add drive groups spec to cluster CR"
This reverts commit 7117fc12b7.
Revert "design: ceph orchestrator module add/remove OSDs"
This reverts commit 178187d035.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Change the TestOSDsOnPVC unit test to use t.Log()/t.Logf() to avoid
having a special infof() function that was unnecessary since our unit
tests run with `go test -v` where the `-v` flag interleaves t.Log()
output.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
After adopting `ceph-volume`, secret key for
user osd.x will be generated by `ceph-volume` and
thus method `generateKeyring` is not necessary
anymore.
Executing ceph command takes
extra half of second to complete, executing it
in serial is rather expensive, remove it for better
performance.
Signed-off-by: shenjiatong <yshxxsjt715@gmail.com>
This was a race condition where the test would sometimes finish before a
deferred watcher.Stop() command was called and sometimes would not.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
There are still references to v15.2.7 in integration tests to allow
testing OSDs on partitions.
The newest dated Ceph image is ceph/ceph:v15.2.9-20210224
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
cURL's ca-cert and Golang TLS config for ca-certs have a different
meaning, it seems that the Golang implementation will always lookup the
local certs first and then proceeds with the specificed cert, if the
system's certs works it will never go to the specificied certificate.
cURL however, does the opposite by strictly checking the specified cert.
If the certificate is signed we don't need to force a specific
certificate, thus by using capath instead we let cURL determine what's
best to validate against.
Signed-off-by: Sébastien Han <seb@redhat.com>
The --silent option will remove the progress bar of the curl but also
silence any errors of stderr, so adding --show-error will display the
error even if --silent is used.
Signed-off-by: Sébastien Han <seb@redhat.com>
removed the code for force delete pods when stuck in terminating state
for mons and osds as there is common code to delete pods for stuck components
Signed-off-by: rohan47 <rohgupta@redhat.com>
When deploying the ceph-fs-mirror daemon we don't need to name the
deployment like "rook-ceph-fs-mirror-a", we can simply go with
"rook-ceph-fs-mirror". Today, in Pacific, the daemon only supports a
single replica. In the future, the Ceph Quincy release should support
mulitiple concurrent daemons, at this point Rook will introduce a
"count" field in the CephFilesystemMirror CRD specification. Internally,
this field will map with the "replica" value of the Deployment.
Signed-off-by: Sébastien Han <seb@redhat.com>
The pool quotas were added in #7264, however the codegen CI
action did not catch that an update was needed until it
was backported.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd prepare job detects the failure domain for the OSD by querying
the node topology labels. This topology is then assigned to the OSD
daemon in the CRUSH map. Previously, the affinity was required to be
set in the cluster CR, but it was very difficult to get right. Now the
operator will enforce the correct topology label on the OSD daemon
nodeAffinity by setting the label of the lowest topology in the hierarcy.
For example, if there are region, zone, and rack labels, the rack label
would be used to set the node affinity for the OSD daemon. If an
OSD prepare job is run in rack1, the corresponding OSD daemon will
have node affinity to rack1 to ensure the same topology. Previously,
the OSD could have ended up in rack2 unless the storageClassDeviceSet
placement was very carefully crafted.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
this commit update `osdProps.placement` instead
of `osdProps.getPreparePlacement()` in the osd.go
this was the copy paste issues coming from other pr.
Signed-off-by: subhamkrai <srai@redhat.com>
If the cluster was expanded we would iterate over the entire list of
OSDs again, create a new key and override in Vault... The Secret store
is not impacted since we have a handler for "already exist".
Signed-off-by: Sébastien Han <seb@redhat.com>
Sébastien Han
subhamkrai
Travis Nielsen
Jiffin Tony Thottan
Umanga Chapagain
Blaine Gardner
Blaine Gardner
shenjiatong
rohan47
Satoru Takeuchi
Frank Ritchie
crombus