ceph: Use the new NetworkSpec api
ceph: Change all tests to use the NetworkSpec
ceph: Remove TypeMeta field from NetworkSpec
ceph: Change IsHost network func signature
Signed-off-by: giovanism <giovanism@outlook.co.id>
Fix variable scoping issue that is ignoring osdsPerDevice config
setting. Ensure that in all scenarios both the global and per OSD
osdsPerDevice setting are honored. When both are set the more specific
setting will be used. (#3599)
Signed-off-by: Michael Vollman <michael.b.vollman@gmail.com>
Previously, the value was set to commonArgs, resulting in dead code,
since common arg isn't used after the append.
Signed-off-by: Odin Ugedal <odin@ugedal.com>
This commit fixes resource limits and placements not getting applied to osd pods provisioned using storageClassDeviceSets
Signed-off-by: Ashish Ranjan <aranjan@redhat.com>
- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.
Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Replace usage of the custom osd copybins command with the new top-level
copy-binaries command introduced with cmd-reporter.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
At the beginning of every container startup we now change the ownership
of every known ceph directories to ceph:ceph.
This will allow daemons to start properly as well as allowing daemon to
writes their logs on files.
This is needed because ceph daemon now run as 'ceph' instead of 'root'
Signed-off-by: Sébastien Han <seb@redhat.com>
We do not determine the daemon type and name by string parsing anymore
but passing that info directly in the update call, it's more reliable.
Also removing a bunch of unused functions.
Signed-off-by: Sébastien Han <seb@redhat.com>
Do not consider clusterInfo Ceph version since it's the one picked by
the spec image.
The CephVersion provided here needs to be the current running version
of Ceph for when it is later passed to OkToStop and checked with
IsAtLeast. If you are upgrading from a pre 14.2.1 version to a
version 14.2.1 or newer, IsAtLeast will incorrectly return true and
then fail trying to run mon ok-to-stop on a mon pod running a pre 14.2.1 version.
Signed-off-by: Sébastien Han <seb@redhat.com>
These were used on a previous patch as a debug, let's remove them. They
are not intended to be displayed.
Signed-off-by: Sébastien Han <seb@redhat.com>
When a cluster is updated with a different image version, this triggers
a serialized restart of all the pods. Prior to this commit, no safety
check were performed and rook was hoping for the best outcome.
Now before doing restarting a daemon we check it can be restarted. Once
it's restarted we also check we can pursue with the rest of the
platform. For instance, with monitors we check that they are in quorum,
for OSD we check that PGs are clean and for MDS we make sure they are
all active.
Fixes: https://github.com/rook/rook/issues/2889
Signed-off-by: Sébastien Han <seb@redhat.com>
Empty deployment list made the operator to crash as the condition check
was checking if the value at the 0th location is nill wich fails in case
of empty list. Updated it to check length of list not equals 0.
Signed-off-by: rohan47 <rohgupta@redhat.com>
Somehow during startup the arg got misinterpreted by ceph-osd and was
failing with:
2019-06-19 15:23:09.098967 I | exec: Running command: ceph-osd --foreground --id 0 --osd-uuid feceb7d7-3148-4a0d-af7a-753ba2f21a92 --conf /var/lib/rook/osd0/rook-ceph.config --cluster ceph --default-log-to-file false --ms-learn-addr-from-peer false
2019-06-19 15:23:09.135921 I | unrecognized arg false
failed to start osd. Failed to complete '': exit status 1.
Now we concatenate the whole flag to avoid that.
Signed-off-by: Sébastien Han <seb@redhat.com>
The check for whether to set the OwnerReferences is no longer
necessary. When passing the correct value to the apiVersion,
the OwnerReferences can be successfully set for OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
- When an osd is marked out, and it is safe to be destroyed then delete the osds deployment.
- Removed osdGracePeriod
- Updated unit tests to test osd marked out action
Signed-off-by: rohan47 <rohgupta@redhat.com>
When OSDs are running on Nautilus we always disable old osd features and
aplpy the onces for Nautilus as described in the upgrade doc.
During an upgrade or the next time an orchestration will be called the
command will be applied. The command is idempotent so we can run it each
time.
This can be backported for 1.0.3
Closes: https://github.com/rook/rook/issues/2960
Signed-off-by: Sébastien Han <seb@redhat.com>
for the osd recovery case (DOWN->UP) log at a matching log level as the
message indicating the OSD went down.
fixes: #2904
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
The list of nodes to be queried for configuring OSDs
should include all of the K8s nodes before the placement
or other filters are applied. The operator was sometimes
missing configuring OSDs on nodes if the discover pod
hadn't finished running on a node. Now the osd prepare
job will be triggered on a node even if the discover pod
hasn't yet run there.
Signed-off-by: travisn <tnielsen@redhat.com>
The ceph controller will delete and recreate the detec-version job, when
an already existing job is found. This helps avoid cluster creation from
getting hung up on waiting for an already existing/stale job to finish.
Fixes#3081.
Signed-off-by: Kaushal M <kshlmster@gmail.com>
This reverts commit b7a564e929, reversing
changes made to 6996571b5c.
Reverting: Linking ceph noout to a node taint.
The taint is applied for reasons other than maintenance as well.
This also needs a way to respect user-set noout.
Need to revisit this with a better design.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
Add a 'ceph-version' label to application controllers in the same manner
as the prior 'rook-version' label to help with upgrades.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
As per https://github.com/ceph/ceph/pull/26856 Ceph tunes the value of
osd-memory-target when running the OSD process so there is no need to
enforce it.
We also set a safety factor to that value which has been calculated by
the Ceph eng/perf team. Without this factor, the OSD will get OOM killed
since it is consuming memory for other purposes too.
Signed-off-by: Sébastien Han <seb@redhat.com>
Allow the user to set annotations in the CRDs of the following operators which will then be set on
the resulting Pods (Deployments, StatefulSets and so on).
Operators:
* Cassandra
* Ceph
* CockroachDB
* EdgeFS
* Minio
* NFS
The Annotations related structures have been added to the
rook.io/v1alpha2 pkg.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
When Ceph OSD spins up using ceph-volume, `ceph-volume lvm activate
--no-systemd --bluestore...` runs `lvs` which can cause significant
delays when /run/udev is not mounted from the host.
Before:
[root@rook-ceph-osd-0-858f49b5-md4t9 /]# time lvs
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
WARNING: Device /dev/sda not initialized in udev database even after waiting 10000000 microseconds.
...
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
osd-data-729c1fb6-440f-434c-be49-f199fa59f1c0 ceph-3bf7823b-6db3-42b6-bf8f-c36301e85238 -wi-ao---- <10.00g
real 1m30.545s
After:
[root@rook-ceph-osd-0-d8c4698b8-5vqnk /]# time lvs
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
osd-data-6c7249cc-cbd1-4f3b-9027-03725886bb30 ceph-9808319f-8567-4930-803a-41f70e537c1f -wi-ao---- <10.00g
real 0m0.043s
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
We must expose the ceph version to the osd pod so that we can generate
the proper ceph configuration file. If we don't do this all the version
check will assume they run the opposite of the desired version and the
formatting of the `ceph.conf` will be wrong.
Signed-off-by: Sébastien Han <seb@redhat.com>
We now run all the daemon with a new option from Nautilus 14.2.1 which
allows us to tell to a daemon to not log on file. However, we can decide
to activate logging by editing the configuration flag 'log_to_file' via
the centralized config store like this for a particular daemon:
ceph config set mon.a log_to_file true
This is useful when a daemon keeps crashing and we want to collect log
files on the system.
Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
It's quite convinient to expose /var/log/ceph so that we can decide to
activate logs locally on the machine and see what's going on.
This is only a placeholder when a daemon is stuck crashlooping and we
want to allow administrator to gather log files.
We still do not log on file but this can be activated via a config
option passed to the centralized config option store.
For some daemons, which typically do not store any data (rgw, rbd-mirror
and mds) we had to propagate dataDirHostPath from the cluster spec to
each creation call so that the bindmount can happen.
Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
When instanciating an osd, we now pass clusterInfo which basically
represents a collection of information about a particular Ceph cluster. Rook uses information
// about the cluster to configure daemons to connect to the desired cluster.
From clusterInfo, we use the CephVersion which helps us putting
conditions for our code execution.
Signed-off-by: Sébastien Han <seb@redhat.com>
Add a `rook-version` label to all controller resources used by Ceph:
deployments, daemonsets, and jobs. Labels are added to controller
resources only and not to the pod templates within because changing pod
templates causes updates due to the label change. The controller
resources themselves are not updated with the label addition and
therefore don't cause an unnecessary upgrade.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
All usages of k8s go client are now also using the versioned `AppsV1() `
call for the client.
Updated MySQL and Wordpress, and Kube Registy examples to use apps/v1
Deployments.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>