Some parameter like bluestore_min_alloc_size, bluestore_min_alloc_size_hdd need to be set during the bootstrap of the
ceph cluster in order to have bluestore created accordingly.
To do so rook users rely on the rook-config-override configmap.
Sadly the rook-ceph-osd-prepare job does not take in account this configmap to generate its ceph config.
Changing the behaviour to merge the config from rook-config-override configmap to the default ceph config
Signed-off-by: n.fraison <n.fraison@criteo.com>
If an OSD is down and needs to be removed from the cluster,
a job can be run that will remove the OSD deployment
and purge the OSD from ceph. If the OSD is still up,
the purge will be rejected.
To purge multiple OSDs at the same time, the OSD IDs can be
specified as a comma-separated list.
Signed-off-by: Servesha Dudhgaonkar <sdudhgao@redhat.com>
Trying to execute a binary from the host within the container can result
in mismatched libraries. The host version is compiled with paths of
libraries on the host where the libs inside the container can differ.
Because the path is known and we only need to make sure whether the bin
exists or not then simply doing a lookup when we fallback on /rootfs is
sufficient.
Closes: https://github.com/rook/rook/issues/6078
Signed-off-by: Sébastien Han <seb@redhat.com>
On rancher OS, the lvm binary is in a different location and cannot be
accessed via the root namespace. So we now also check directly from
/rootfs. When using nsenter, itbreaks out of container's mount
namespace and executes an lvm command.
Closes: https://github.com/rook/rook/issues/6078
Signed-off-by: Sébastien Han <seb@redhat.com>
callCephVolume() was using our exec interface with CombinedOutput which
means return both stdout and stderr. This is fine when preparing OSDs
since we read back the stdout and also want any errors if any (stderr).
However when listing OSD we only want the output since ceph-volume tends
to mix stdout and stderr in the output and does not always silence the
warning types/harmless errors. If the listing has a problem we report it
by printing the ceph-volume log file.
Signed-off-by: Sébastien Han <seb@redhat.com>
Older versions of lvm do not implement "--help" but "help" only where
new versions support both so let's use "help" so the check works for all
the versions.
Closes: https://github.com/rook/rook/issues/6057
Signed-off-by: Sébastien Han <seb@redhat.com>
Now the metadata pvc will also be encrypted if the storageClassDeviceSet
has the "encrypted" flag turned on.
Signed-off-by: Sébastien Han <seb@redhat.com>
Based on the feature of metadata PVC for OSD: #4818, add the wal device.
For this, you need to create a new volumeClaimTemplates, its name must be "wal", if "wal" is
set, "metadata" device will refers specially for block.db device, and "wal" refers for block.wal
device in BlueStore.
A template will look like this:
```
volumeClaimTemplates:
- metadata:
name: data
spec:
resources:
requests:
storage: 10Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, gp2)
storageClassName: gp2
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: metadata
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
storageClassName: io1
volumeMode: Block
accessModes:
- ReadWriteOnce
- metadata:
name: wal
spec:
resources:
requests:
storage: 6Gi
# IMPORTANT: Change the storage class depending on your environment (e.g. local-storage, io1)
storageClassName: io1
volumeMode: Block
accessModes:
- ReadWriteOnce
```
We now map block, block.db and block.wal directly inside the container instead of
running ceph-volume activate.
Signed-off-by: Chen, Tingjie <tingjie.chen@intel.com>
We can now encrypted OSD device that were provisioned via a storage
class using the PV interface.
The encryption works at the storageClassDeviceSets level, which means we
can have encrypted and non-encrypted sets.
Using the new key `encrypted` we can turn it on.
Signed-off-by: Sébastien Han <seb@redhat.com>
using defer for closing file which are open
for writing is not safe. so closing file again
following below steps:
1. open files
2. defer file.close()
3. write
4.file.close()
these will make sure files are closed.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
File permissions need to be reduced to the minimum to avoid config
files from being accessed. While a connection to the pod would be
required to access the files, min privileges are set just as
another layer of security. This fixes gosec error G306.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
This commit fixes where there are areas in which files and directories are
written and created with statically defined permissions. Many of these
permissions are rather open, potentially allowing other system tenants to
view and interact with their contents, which may be sensitive.
Fixed G303: Poor file permissions used for directory by using the most narrow permission
Resoves: https://github.com/rook/rook/issues/4580
Signed-off-by: Nizamudeen <nia@redhat.com>
this commit suppress the gosec errors for
g204: Audit use of command execution.
g304: File path provided as taint input.
g101: Look for hard coded credentials.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
To ensure a file handle is closed, we defer the close command
so it is guaranteed to run when the method returns. The closing
of the file handle is not going to fail in our usage since we aren't
using the SetDeadline on the files that would cancel a request
and return an error.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The admin needs to start the toolbox and run a command such as ceph osd tree to see the device classes for each OSD. This PR simplifies it by storing the deviceclass info in the CephCluster CR status
closes: https://github.com/rook/rook/issues/5754
Signed-off-by: kesavan <kvellalo@redhat.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisk which holds more details:
* method: indicates if the entire disk should be sanitized or simply ceph's metadata.
Possible choices are 'complete' or 'quick' (default)
* dataSource: indicate where to get random bytes from to write on the disk.
Possible choices are 'zero' (default) or 'random'
Using random sources will consume entropy from the system and will take much more time then the zero source
* iteration: overwrite N times instead of the default (1). Takes an integer value
See the documentation for more details.
Signed-off-by: Sébastien Han <seb@redhat.com>
The ClusterInfo does not use the Log method to write the fields
anymore. The formatting was not necessary since the struct can
be logged if needed with the standard %v golang formatting.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The user creds should not be passed to the ceph helper methods
since the clusterInfo already has the needed context. Removing
a final reference that was missed in the refactor.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph-file-controller gets stuck in loops, this happens when it tires to delete the filesystem which doesn't exist.Filesystem should only be deleted when it exist.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
If the healthcheck goroutine cannot check for the ceph status/times out
(if the mons are not in quorum) this should still be reflected in the
CRD status field.
Example:
```
status:
ceph:
details:
error:
message: 'failed to get status. . timed out: exit status 1'
severity: Urgent
health: HEALTH_ERR
lastChanged: "2020-07-20T14:11:05Z"
lastChecked: "2020-07-20T14:11:05Z"
previousHealth: HEALTH_WARN
```
Signed-off-by: Sébastien Han <seb@redhat.com>
A long-standing issue that was only solved via a documentation note. Now,
during the prepare pod instantiation, we check for the presence of the
lvm binary on the host. Success will indicate that we can bootstrap
OSDs where failure will refuse to prepare the OSD.
Closes: https://github.com/rook/rook/issues/5627
Signed-off-by: Sébastien Han <seb@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph config and client packages are conceptually the same.
To avoid circular dependencies in some cases, we simplify by combining
the packages into the client package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator should only connect to ceph with a single set of creds.
In a converged cluster this will be the admin creds and in an external
cluster it will be lower-privileged creds. Independent clusters were
implemented with a separate set of creds. To simplify the code these
are now merged to a single set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add the ability to provision Ceph OSDs with Drive Groups.
This adds Drive Groups to the CephCluster CRD, and it sets code
in place for propagating this config to the OSD provisioning pod.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
moves the admission-controller server to use controller-runtime library for better support, maintainibility of code.
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
adds deploy.sh script to deploy validatingwebhookconfiguration and create secrets.
adds new command ceph admission-controller to start webhook servers.
adds validation for various rook custom resources
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
Fix the following bugs.
- CephVersion is not back to Octopus
- `cephadm' here is called with three arguments,
"info --query=property <devname>". We should use 'args[2]' instead.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
subhamkrai
n.fraison
Servesha Dudhgaonkar
Sébastien Han
Travis Nielsen
Chen, Tingjie
Satoru Takeuchi
Nizamudeen
kesavan
RAJAT SINGH
Blaine Gardner
Vineet Badrinath