Due to #6492, preservePoolsOnDelete is not useful at all for CephFS;
after the filesystem is deleted, the leftover pools cannot be
reassocaited with a newly created filesystem without wiping all
metadata. The only way we can actually preserve data is keeping around
the entire filesystem.
This commit implements a `preserveFilesystemOnDelete` option which work
similar to the existing pool preservation option but instead keeps the
whole CephFS while taking it down and removing all MDSes.
This commit also changes all documentation to refer to this new option
with the intent of essentially deprecating `preservePoolsOnDelete`. IMO,
keeping around `preservePoolsOnDelete` is actively harmful because it
lulls users into thinking their data will be safe but, in reality,
recovering from this situation is highly complex and has large potential
for data loss.
Signed-off-by: Lalit Maganti <lalitm@google.com>
When the Ceph cluster runs on PVC and the OSDs are encrypted we can
store LUKS's Key Encryption Key inside a Key Management System. Today,
Rook only supports HashiCorp Vault: https://www.vaultproject.io/
The CephCluster has now a new "security" field which will plug onto the
KMS. Here is an example:
security:
kms:
tokenSecretName: <name of the secret containing a Vault token, used
to authenticate>
connectionDetails: < a map of strings containing connection
information>
Refer to the ceph-cluster-crd documentation to lear more.
Closes: https://github.com/rook/rook/issues/6105
Signed-off-by: Sébastien Han <seb@redhat.com>
To clarify the spec of ApplyToObjectMeta This also fixes the misleading
comments on the method.
Signed-off-by: Yusei Nishiyama <yusei.nishiyama@soundcloud.com>
adding fields `TotalBytes`, `UsedBytes`, `AvailableBytes`
`LastUpdated` to the ceph cluster CR's status section to
report the storage details of ceph cluster.
`TotalBytes`: Total storage capacity in bytes
`UsedBytes`: Total storage used in bytes
`AvailableBytes`: Total storage used in bytes(Total - Used)
`LastUpdated`: Last update time
for example(I have added 10GB disk):
`kubectl get cephclusters.ceph.rook.io -o yaml`
```
status:
ceph:
capacity:
bytesAvailable: 9657188352
bytesTotal: 10733223936
bytesUsed: 1076035584
lastUpdated: "2020-10-23T12:57:48Z"
```
Signed-off-by: subhamkrai <srai@redhat.com>
golangci-lint linter gosec showing more errors
than gosec gh. This commit resolve
new errors. And, removing
nosec comments from autogenerated files.
Signed-off-by: subhamkrai <srai@redhat.com>
The pool spec has now a new property called "replicasPerFailureDomain"
which essentially represents the number of replicas to store in each
failure domain.
Assuming the failure domain is a datacenter (if the cluster is
stretched) then you will have 2 replicas per datacenter where each
replica ends up on a different host. This gives you a total of 4
replicas and for this, the "size" must be set to 4.
Closes: https://github.com/rook/rook/issues/5591
Signed-off-by: Sébastien Han <seb@redhat.com>
The ceph mons should never be started with an even number in quorum.
The desired state should always be an odd number of mons to ensure
a healthy majority quorum. The operator now rejects a request for
an even number of mons.
If there are not a sufficient number of nodes for mons to be on a
unique host, the operator also returns an early error instead of
getting stuck waiting for a pending mon. This is not foolproof since
there might be fewer available nodes for the mons, but at least it
is a quick check for the common case.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
on Azure, all SSDs are reporting as a rotational disk.
This commit will add `tuneFastDeviceClass` flag which will
tune for an SSD even if the cloud provider is reporting
rotational. We have a flag `tuneDeviceClass`
which works vice-versa.
Signed-off-by: subhamkrai <srai@redhat.com>
this commit will enable one more linter ineffassign
in golangci-lint.
This linter throws an error when variable is assigned and never used.
`golangci-lint run --disable-all -E ineffassign` is used detects ineffassign
errors only.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
Rook is now capable of configuring mirroring between sites. The
implementation works at different levels:
* CephBlockPool: which introduces a new `mirroring` configuration as well
as `statusCheck`. When turned on, Rook will enable mirroring on the
pool. It will also create a bootstrap peer token and store it in a
Kubernetes Secret. The name of that Secret can be found in the Status
field of the CephBlockPool CRD. This token can be fetched and used by
other clusters to configure the site as a peer. Mirroring can be
configured either at the pool or the image level.
* CephRBDMirror: which introduces a new `peers` configuration allowing
Rook to connect to peers by passing a Secret name. The administrator will
create a Kubernetes Secret with 2 keys: 'token' for the bootstrap peer
token and 'pool' for the name of pool. Once detected the rbd-mirror
controller will go ahead and import the peer configuration.
Pool mirroring status example:
```
status:
info:
rbdMirrorBootstrapPeerSecretName: pool-peer-token-test
mirroringInfo:
lastChanged: "2020-09-17T14:47:27Z"
lastChecked: "2020-09-17T14:48:27Z"
summary:
summary:
mode: image
peers:
- client_name: client.rbd-mirror-peer
direction: rx-tx
mirror_uuid: ""
site_name: rhcs
uuid: c50522a4-28a4-4bd3-ba68-e11780308882
site_name: 91eae0dd-06b1-4d2c-91f3-1311c9df382b-rook-ceph
mirroringStatus:
lastChecked: "2020-09-17T14:48:27Z"
summary:
summary:
daemon_health: OK
health: OK
image_health: OK
states:
replaying: 1
```
Signed-off-by: Sébastien Han <seb@redhat.com>
This adds an uninstall mode to the ceph cluster. If
allowUninstallWithVolumes in the CleanupPolicySpec is set to true then
the cluster deletion does not wait for the PVs to be deleted.
Co-authored-by: Nitin Goyal <nigoyal@redhat.com>
Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
This allows users to specify additional labels to be added to the components
created by the Rook Ceph operator.
Additionally the rbd mirrors are now getting their annotations and
labels set.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
The operator will now allow the resource limits to be applied below the
recommended minimums. In small clusters, even the recommended minimums
may not be necessary. A warning is still printed to the operator log,
but we allow the configuration to continue.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The osd prepare jobs and osd daemons may have different placement
requirements. In particular, the topology spread constraints
or pod anti-affinity should only work on either the prepare job or
osd daemon, but does not work effectively on both types at the same
time. Therefore, we split out the osd prepare placement to be separate
from the osd daemon placement. If the osd prepare placement is not set
the operator will fall back to the osd daemon placement for
backward compatibility.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The helper to convert annotations to a string map is not needed in the
production code, so it is moved to the unit tests.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Applying annotations was crashing when there were no annotations in the
original map. Now the map is initialized when no annotations are set
in the original map.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We can now encrypted OSD device that were provisioned via a storage
class using the PV interface.
The encryption works at the storageClassDeviceSets level, which means we
can have encrypted and non-encrypted sets.
Using the new key `encrypted` we can turn it on.
Signed-off-by: Sébastien Han <seb@redhat.com>
this commit handles all the gosec g601
error code (i.e Implicit memory aliasing
of items from a range statement).
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
Instead of setting the Phase of the CR once the reconcile is done, let's
actually set in from the healthcheck so it is more accurate.
Closes: https://github.com/rook/rook/issues/5249
Signed-off-by: Sébastien Han <seb@redhat.com>
The admin needs to start the toolbox and run a command such as ceph osd tree to see the device classes for each OSD. This PR simplifies it by storing the deviceclass info in the CephCluster CR status
closes: https://github.com/rook/rook/issues/5754
Signed-off-by: kesavan <kvellalo@redhat.com>
- Add realm/zone group/zone to Object Context,
so that any call to runAdminCommand has the
same realm, zone group, and zone as the object
store that the call is going to.
- Also modify the delete code for the object-store
using multisite to remove the endpoints of an
object store from a zone instead of the normal
clean-up
- Added more debug logging all around the object-store
code related to multisite
- files generated by rerun of `make codegen`
- change back the edit on the Copyright in object.go
Signed-off-by: Ali Maredia <amaredia@redhat.com>
When zones are created the ceph RGWs inside the
those zones should be using pools with the zones
name not the object-store's name
Signed-off-by: Ali Maredia <amaredia@redhat.com>
This commit:
- adds the pull section on the CephObjectRealm spec
to enables realms to be pulled instead of created.
- creates the system user and generates the access
key and secret key for the user and zones in a realm.
- adds "omitempty" to fields in multisite CRs that
are not required
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisk which holds more details:
* method: indicates if the entire disk should be sanitized or simply ceph's metadata.
Possible choices are 'complete' or 'quick' (default)
* dataSource: indicate where to get random bytes from to write on the disk.
Possible choices are 'zero' (default) or 'random'
Using random sources will consume entropy from the system and will take much more time then the zero source
* iteration: overwrite N times instead of the default (1). Takes an integer value
See the documentation for more details.
Signed-off-by: Sébastien Han <seb@redhat.com>
Travis Nielsen
Lalit Maganti
Sébastien Han
Yusei Nishiyama
subhamkrai
subhamkrai
Santosh Pillai
Raghavendra TalurandNitin Goyal
Alexander Trost
Umanga Chapagain
kesavan
Ali Maredia
Hiroshi Muraoka