The PG count on metadata pools should default to rgw_rados_pool_pg_num_min
instead of the more general default pg count. This means rgw pools
will default to 8 PGs instead of 32 PGs, which means a lot more pools
can be created before hitting the default PG limit.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit adds support for LVs to the device availability check
in the OSD prepare pod.
The availability of an LV is checked by "ceph-volume lvm list".
If it returns non-empty result, the LV is in use and not available.
Closes: https://github.com/rook/rook/issues/5075
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
This commit fixes the argument for "ceph-volume inventory".
When a device "/dev/mapper/foo" is being checked for its availability,
the argument should not be "/dev/foo" nor "/dev/dm-1".
Signed-off-by: morimoto-cybozu <kenji_morimoto@cybozu.co.jp>
The pools had some legacy structs that translated between the
ceph.v1 types used by the CRDs and the internal implementation
of the pools. This simplifies the pool implementation by removing
the intermediate model and leaving us only with the ceph v1
pool types and no unnecessary translation.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rook should always set the pgnum of the new block pool
to the default value("0"). Current implementation accidentally
works fine because newPool.Number is always 0 here.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The helpers for executing a process have long required an actionName
param which is not being used. Now we remove the old param
while also cleaning up various other usages of the exec
package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph commands are now only written to the log in debug mode.
For commands that change the system state we now ensure that
a useful log entry is written. If all the details of the ceph
commands are needed, debug logging should still be enabled.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rook no longer relies on its own process management, now we can rely
completely on Kubernetes to manage the pod lifecycle. The code
to check for running processes and replacement them hasn't been
used for a long while.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The OSD prepare pod must always update status on the status configmap
in order to alert the operator that the OSD configuration on the node
is completed. This fixes an issue where the status was not being reported
if no devices were configured.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The communication between the operator and OSD prepare job
is done through updates to the configmap. If an update fails
to the configmap, we are anyway going to need to check the logs
for failure. So let's just log the error and make a best effort
to continue the orchestration. If the failure was temporary,
the next update to the status would succeed and likely continue
working as expected.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Now, the CephBlockPool CRD is managed with the controller-runtime.
So the watcher is outside of the main controller reconciliation loop of
CephCluster which brings numerous benefit such as:
* having its own reconciliation loop
* won't block anything from the main CephCluster controller loop
* fast than waiting for CephCluster loop to completion
Partially close: https://github.com/rook/rook/issues/1981
Signed-off-by: Sébastien Han <seb@redhat.com>
The rook types used across the storage providers moved from the v1alpha2
package to the v1 package. This commit points the packages at their new
location. Implementation is expected to remain unchanged.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
As of Octopus, Ceph will prevent you from creating a pool with a
replica size of 1. Allowing such pool could lead to data loss, so enable
the new option: requireSafeReplicaSize: false if you are **ABSOLUTELY**
certain that is what you want.
Closes: https://github.com/rook/rook/issues/4889
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is to handle all those unhandled errors which raises the gosec warning.
Fixed G104: Unhandled Errors are handled now
Signed-off-by: Nizamudeen <nia@redhat.com>
The Ceph daemons no longer use the ceph.conf for settings generated
by the operator. The operator only generates the ceph.conf in order
to connect to the mons and configure the cluster. Now we reduce
to the minimal required settings.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When the OSD on PVC is backed by a metadata block PVC, the Ceph CRUSH
device class should be set to something else rather than the rotational
property of the drive.
Closes: https://github.com/rook/rook/issues/4881
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit is also part of gosec error handling which handles the following issue:
Fixed G304: Potential File inclusion by cleaning up the file path
Signed-off-by: Nizamudeen <nia@redhat.com>
When running old client version, we must force the compatibility
features so that the orchestration does not fail in a loop.
Closes: https://github.com/rook/rook/issues/4842
Signed-off-by: Sébastien Han <seb@redhat.com>
We now support the addition of the PVC that acts as a metadata device
for a given OSD.
For this, you need to create a new `volumeClaimTemplates`, its name must
be "metadata" otherwise, Rook won't pick it up.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block and block.db directly inside the container instead of
running ceph-volume activate. This is much cleaner.
Closes: https://github.com/rook/rook/issues/3852
Signed-off-by: Sébastien Han <seb@redhat.com>
This will likely happen when re-running an orchestration and
bootstrapping a new cluster with unclean drives.
The prepare pod won't prepare nor list existing disks, thus the code
will later fail on reading disks properties.
Closes: https://github.com/rook/rook/issues/4810
Signed-off-by: Sébastien Han <seb@redhat.com>
If we detect a partition we should not use it if the ceph version is not
at least 14.2.8 since it has the necessary changes to support partitions
with ceph-volume.
Signed-off-by: Sébastien Han <seb@redhat.com>
14.2.7 will only be a CVE fix release so 14.2.8 is the release that will
contain the fixes for ceph-volume partition.
Signed-off-by: Sébastien Han <seb@redhat.com>
Multiple things:
1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.
2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.
3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.
Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
We now use a defined directory for each prepare job so that logs won't
be entangled with each other.
Also, if an error occurs, we print the log file too so we don't have to
ssh onto a host to find the c-v debug log.
Closes: https://github.com/rook/rook/issues/3888
Signed-off-by: Sébastien Han <seb@redhat.com>
We now support partitions via 2 ways:
* if `useAllDevice: true`: partitions will be taken into account and
presented as OSD candidate
* if specified in the cluster CR: it'll picked up as well
Signed-off-by: Sébastien Han <seb@redhat.com>
The operator kit had more utility originally when the operator
was creating and managing the TPRs and CRDs directly. Since
the CRDs are now created from a manifest and no longer by the
operators, the utility of operator kit is limited to the
controller watcher. Since we are moving to the controller runtime
we simplify the code to make the transition smoother. Now
there is only a simple WatchCR method that will need to be
replaced as we maek that transition.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Enabling pg_autoscalar module fails most probably becasue `ceph mgr module enable <>` is happening very quickly after ceph mgr starts.
This PR adds a retry (5 times) with a sleep interval of 5 seconds (for each interval) to handle failure while enabling mgr modules.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
When using the "errors" package, using `%+v` (extended format),
each Frame of the error's StackTrace will be printed in detail.
Let's only print `%v` to print the error.
If the error has a Cause it will be printed recursively.
Basically `%+v` has been replaced with `%v` for all `error` type
interfaces, whether the logger is Info, Warning or Error.
Signed-off-by: Sébastien Han <seb@redhat.com>
Travis Nielsen
morimoto-cybozu
Satoru Takeuchi
Sébastien Han
Kristoffer Grönlund
Nizamudeen
crombus
Dustin Guerrero
Santosh Pillai