currently device class and device type
labels were clubbed with eachother
create a seperate label, as these two labels
can be seperate
Signed-off-by: parth-gr <partharora1010@gmail.com>
If osd store is updated in the ceph cluster, then
delete OSDs one by one, cleanup disks and provision a new OSD on
the same disk
Signed-off-by: sp98 <sapillai@redhat.com>
Rook uses global or node-local device class configuration if device-level
configurations doesn't exist. However, after introducing device-class
level resource configuration, rook set the default value of device class.
So global or node-level device class configuration has never been used
after that.
Closes: https://github.com/rook/rook/issues/11871
Closes: https://github.com/rook/rook/issues/11826
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Even if we can use raw mode, do NOT use raw mode on disks. Ceph bluestore disks can
sometimes appear as though they have "phantom" Atari (AHDI) partitions created on them
when they don't in reality. This is due to a series of bugs in the Linux kernel when it
is built with Atari support enabled. This behavior does not appear for raw mode OSDs on
partitions, and we need the raw mode to create partition-based OSDs. We cannot merely
skip creating OSDs on "phantom" partitions due to a bug in `ceph-volume raw inventory`
which reports only the phantom partitions (and malformed OSD info) when they exist and
ignores the original (correct) OSDs created on the raw disk.
Resolves https://github.com/rook/rook/issues/7940
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Ceph support the option '--osd-crush-initial-weight' upon OSD start,
which sets an explicit weight (in TiB units) to specific OSD. Allow
passing this option all the way from the user (similar to
'DeviceClass'), for the special case where end users wants it cluster
to have non-even balance over specific OSDs (e.g., one of the OSDs is
placed over a partition alongside OS-partition).
ROOK issue: https://github.com/rook/rook/issues/7448
Signed-off-by: Shachar Sharon <ssharon@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph config and client packages are conceptually the same.
To avoid circular dependencies in some cases, we simplify by combining
the packages into the client package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Rook no longer relies on its own process management, now we can rely
completely on Kubernetes to manage the pod lifecycle. The code
to check for running processes and replacement them hasn't been
used for a long while.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
We now support the addition of the PVC that acts as a metadata device
for a given OSD.
For this, you need to create a new `volumeClaimTemplates`, its name must
be "metadata" otherwise, Rook won't pick it up.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block and block.db directly inside the container instead of
running ceph-volume activate. This is much cleaner.
Closes: https://github.com/rook/rook/issues/3852
Signed-off-by: Sébastien Han <seb@redhat.com>
Multiple things:
1. We removed all the function/methods/tests that were used to
create and manage rook legacy OSDS as well as bringing support to
Bluestore OSD only.
It also fixes various go-lint issues in the respectives files.
2. use c-v inventory to detect available devices:
Now we rely on the 'ceph-volume inventory' command to tell us if a
device is available or not.
3. implement raw mode for osd on pvc
When an OSD will be bootstrap on a PVC, the new c-v raw mode will be
used. It consists of putting block, db and wal under the same device.
Here LVM is out of the picture and the raw device is used as is. The
implementation is backward compatible so existing OSD on PVC will LVM
will continue to operate.
Closes: https://github.com/rook/rook/issues/4363
Signed-off-by: Sébastien Han <seb@redhat.com>
When using the "errors" package, using `%+v` (extended format),
each Frame of the error's StackTrace will be printed in detail.
Let's only print `%v` to print the error.
If the error has a Cause it will be printed recursively.
Basically `%+v` has been replaced with `%v` for all `error` type
interfaces, whether the logger is Info, Warning or Error.
Signed-off-by: Sébastien Han <seb@redhat.com>
`ceph-volume` cannot handle a device correctly if the device has an alias
under `/dev/mapper/`.
This commit fixes it by checking whether the device has that kind of alias
and using the alias as an argument for `ceph-volume`.
Signed-off-by: Kazuhito MATSUDA <kazuto.jinnai@gmail.com>
The topology of the cluster should be based on the node labels rather
than a setting in the cluster CR. This allows a much richer and more
dynamic topology to be configured. The location will now be ignored
if specified in the cluster CR.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Previously, "run dir" was used to place a number of file and config on
dataDirHostPath. Now, since a lot of configs and options have moved
either on the OSD's startup CLI line or remove with bluestore we don't
need to change the default.
Also, it was confusing for user and difficult to find the socket.
The is is not breaking any config in dataDirHostPath since all the
elements (keyrings) are hardcoded in the config file.
Closes: https://github.com/rook/rook/issues/3966
Signed-off-by: Sébastien Han <seb@redhat.com>
Add deviceClass as a per OSD config option under
storage/nodes/devices/config for setting a custom crush device class
per OSD.
Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
Adding support for the db-devices flag for ceph versions 14.2.1 and
newer. This flag will allow for explictly setting the db device for
bluestore configurations to the device specified as metadataDevice.
Fixing the databaseSizeMB config parameter to ensure it is used to set
the size of the db-device partition when specified at either the global
or individual device level. (#3652)
Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.
Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
this adds a structure to hold various settings related to executing ceph
cli commands, and introduces a common interface for configuring and
running such commands.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
We must expose the ceph version to the osd pod so that we can generate
the proper ceph configuration file. If we don't do this all the version
check will assume they run the opposite of the desired version and the
formatting of the `ceph.conf` will be wrong.
Signed-off-by: Sébastien Han <seb@redhat.com>
All devices detected by the discovery pod were being passed to the OSD provisioning pod
thus not always honoring the desired device list that should be provisioned.
Now the provisioning pod will be given the desired state from the crd,
then apply that state depending on the actual devices detected.
Also added a helper to ensure OSDsPerDevice is always valid.
Signed-off-by: travisn <tnielsen@redhat.com>
Create a cephconfig module in Ceph's daemon pkg source, and refactor the
config and keyring generation that exists in the mon package into the
new cephconfig package. The config/keyring generation code is used by
most all daemons and not just mon, so a new package is a more
appropriate place for this.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Cleaning out the below entities.
- JournalPartitionType is defined but will never be used given rook will always
place journal file in the filestore's block partition.
- GetMetadataPartitionType and getDataPartitionDetails functions are legacy
code and no longer used by any of the rook components.
Resolves: #1775
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
run "rook discover" on storage nodes and discover devices on each node. The discovered disks are saved in a per node configmap, local-device-nodename.
Device information consits of name and persistent names, uuid, partition, filesystem, rotational, readonly, size, etc.
Signed-off-by: Huamin Chen <hchen@redhat.com>
If watching the osd orchestration status map fails, ensure the loop is broken out of so a new watcher can be created.
Log the osd device/dir mapping structures in a more friendly way
Signed-off-by: Jared Watts <jbw976@gmail.com>
parth-gr
sp98
Satoru Takeuchi
Blaine Gardner
Shachar Sharon
subhamkrai
Travis Nielsen
Madhu Rajanna
Sébastien Han
Kazuhito MATSUDA
Michael Vollman
rohan47andAshish Ranjan
Noah Watkins
Blaine Gardner
Huamin Chen
Ganesh Maharaj Mahalingam
Jared Watts
Travis Nielsen