Commit Graph
36 Commits
Author SHA1 Message Date
Travis Nielsen bcb99d86ce crds: pick up the rook types in the v1 package
The rook types used across the storage providers moved from the v1alpha2
package to the v1 package. This commit points the packages at their new
location. Implementation is expected to remain unchanged.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-02-26 11:18:17 -07:00
Nizamudeen 53883f68cf ceph: Handling Unhandled errors
This commit is to handle all those unhandled errors which raises the gosec warning.

Fixed G104: Unhandled Errors are handled now

Signed-off-by: Nizamudeen <nia@redhat.com>
2020-02-21 22:48:24 +05:30
Sébastien Han 45118185cf ceph: remove mimic support, default to nautilus
As of 1.3, Rook will only support Ceph Nautilus 14.2.5.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-02-13 09:17:34 +01:00
Sébastien Han 92c1696bfc ceph: cleanup/trim isUpgrade variable a bit
We now always perform ok-to-stop change when the deployment is going to
be updated. This could be for various reasons like change in the spec or
a new Ceph image.
In any case, if Ceph resources are going to be restarted this must be
done carefully.

Also trimmed the isUpgrade variable, the upcoming commit will remove the
need of that variable.

Keeping isUpgrade from the controller code since we need it to detect
whether an orchestration upgrade should be triggered or not (based on
the ceph cluster status).

Also, we keep it on the child controllers so we don't go into an
orchestration phase if the cluster image hasn't changed. The child does
not need to be updated/go into its orchestration phase because the
cluster CR changed.

Closes: https://github.com/rook/rook/issues/4379
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-01-24 17:02:37 +01:00
Sébastien Han 929853b652 ceph: remove ROOK_VERSION from the mgr pod
Because of this, the mgr would be restarted if the operator image
change. This must not happen.
Added unit tests for cephMgrOrchestratorModuleEnvs() as well.

Closes: https://github.com/rook/rook/issues/4361
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-11-26 15:40:36 +01:00
d-luu 9c755fac66 ceph: added priority classes to components
Adds priority class support to Ceph components
  to influence scheduler's pod preemption

Signed-off-by: d-luu <david@davidluu.info>
2019-11-15 16:28:41 -06:00
Sébastien HanandRohan CJ 90cd8224f4 ceph: add ceph-crash controller
A new controller to bootstrap ceph-crash pod on Ceph nodes running Ceph
pods only.

This implementation is unfortunately a mix of 'hacks' to make the daemon
working correclty. The ceph-crash script faced different issues:

* it's not a daemon
* it does not have any key in cephx
* it runs ceph commands under the hood which need an admin key and a
ceph.conf

On upstream Ceph, ceph-crash needs to grow better, once that happens we
will improve our implementation.

This also enhances osd provisionConfig struct with DataPathMap

Pod volumes and volume mount needs DataPathMap to perform the right
actions on log and crash dir. Exposing DataPathMap makes that possible.

Obviously, this is exposing ceph crash reports on the host as
well as pushing them into the mgr.
Basically, if a daemon fails, it'll put its core dump into
/var/lib/ceph/crash, bindmounting this dir on the host, ensures that the
crashes don't get lost when the pod dies.

Example:

```
[leseb@tarox~/go/src/github.com/rook/rook][rgw-liveprobe !] kubectl -n rook-ceph exec -ti rook-ceph-crashcollector-minikube-574858b99c-zvg4z bash
bash: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_COLLATE: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_TIME: cannot change locale (en_US.UTF-8): No such file or directory
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]# ceph crash ls

[root@rook-ceph-osd-2-7644f99695-cljzh /]# pidof ceph-osd
13258 5415 5397
[root@rook-ceph-osd-2-7644f99695-cljzh /]# kill -SIGABRT 13258
[root@rook-ceph-osd-2-7644f99695-cljzh /]# ls /var/lib/ceph/crash/
2019-11-12_12:56:51.404109Z_39f060e1-776d-4605-8f28-85c97e53de96     posted

... wait maximum 10 min (ceph-crash scraps every 10 minutes)
... the container will exit

[root@rook-ceph-osd-2-7644f99695-cljzh /]# exit
[leseb@tarox~/go/src/github.com/rook/rook][rgw-liveprobe !]
[leseb@tarox~/go/src/github.com/rook/rook][rgw-liveprobe !] kubectl -n rook-ceph exec -ti rook-ceph-crashcollector-minikube-574858b99c-zvg4z bash
bash: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_COLLATE: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.UTF-8): No such file or directory
bash: warning: setlocale: LC_TIME: cannot change locale (en_US.UTF-8): No such file or directory
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]#
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]#
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]# ceph crash ls
2019-11-12_12:56:51.404109Z_39f060e1-776d-4605-8f28-85c97e53de96 osd.1
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]# ls /var/lib/ceph/crash/
posted
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]# ls /var/lib/ceph/crash/posted/
2019-11-12_12:56:51.404109Z_39f060e1-776d-4605-8f28-85c97e53de96
[root@rook-ceph-crashcollector-minikube-574858b99c-zvg4z /]# tail /var/lib/ceph/crash/posted/2019-11-12_12\:56\:51.404109Z_39f060e1-776d-4605-8f28-85c97e53de96/log
   1/ 5 mgr
   1/ 5 mgrc
   1/ 5 dpdk
   1/ 5 eventtrace
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/lib/ceph/crash/2019-11-12_12:56:51.404109Z_39f060e1-776d-4605-8f28-85c97e53de96/log
--- end dump of recent events ---
```

Signed-off-by: Sébastien Han <seb@redhat.com>
Co-authored-by: Rohan CJ <rohantmp@gmail.com>
2019-11-12 16:06:07 +01:00
travisn 9d94245918 ceph: configure mgr modules asynchronously
The mgr modules take some time to run and have also been known
to hang. While the hang issues should have been resolved separately,
an improvement is still to allow the configuration to happen
in parallel for unrelated modules. All of the module configuration
will be completed before continuing with osd configuration
in order to keep the logging simpler.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-10-23 11:38:00 -06:00
Sébastien Han f6a5caa164 ceph: mgr do not override annotation
The current implementation was overriding any previous annotations set on
the object meta.
Moving the logic to its own method as well as adding unit tests.

Closes: https://github.com/rook/rook/issues/4106
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-10-16 11:28:11 +02:00
Blaine Gardner db7a6da09e ceph: do ceph:ceph chown in init containers
Following the Ceph OSD rework, my test environment regularly failed to
start OSDs, as they would start before the postStart lifecyle hook which
was chown'ing the log and data directories could finish. Pods failed to
start for an arbitrarily long time before starting successfully. This
was a manifestation of a race condition identified in Ceph monitors in
https://github.com/rook/rook/pull/3594#discussion_r312279176

Create a Ceph operator-level method to create an init container that
will chown the necessary directories for all Ceph daemons (any daemon
which runs as the ceph:ceph user-and-group). Also add this container to
expectations for Ceph pod templates in unit tests.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-10-15 12:12:46 -06:00
Sébastien Han 087d71106b ceph: add a new CR property to force upgrades
On rare occasion, the ceph checks might not be 100% correct and the user
might decide to force an upgrade anyway.
Correctly, if we are in an upgrade, we perform `ok-to-stop` check for
each daemon, if one of them fails, we retry and eventually fail.
They are corner-cases where users would like to force the upgrade anyway.

Enhance the new CR property: "skipUpgradeChecks: true".

Closes: https://github.com/rook/rook/issues/3872
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-09-19 10:55:56 +02:00
travisnandSébastien Han c30cc697c7 ceph: ability to enable mgr module via CRD
We can now enable via the cluster CR any manager module with a
setting under the new mgr element

mgr:
  modules:
  - name: pg_autoscaler
    enabled: true

Co-authored-by: Sébastien Han <seb@redhat.com>
Co-authored-by: travisn <tnielsen@redhat.com>
Signed-off-by: travisn <tnielsen@redhat.com>
2019-09-06 11:32:51 -06:00
Sébastien Han bed14cfd71 ceph: improve upgrade on image change
From now on, Rook will only perform check before upgrades when there is
an actual upgrade. So if the Ceph image changed and a new version is
desired Rook will go through all the daemons and update them one by one
and perform checks in between.

Closes: https://github.com/rook/rook/issues/3583
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-08-29 10:18:19 +02:00
giovanism cc62702d39 ceph: Use rook's NetworkSpec with type embedding
ceph: Use the new NetworkSpec api

ceph: Change all tests to use the NetworkSpec

ceph: Remove TypeMeta field from NetworkSpec

ceph: Change IsHost network func signature

Signed-off-by: giovanism <giovanism@outlook.co.id>
2019-08-26 16:17:03 +07:00
travisn 9a3dc52a11 ceph: minimum version allowed is mimic 13.2.4
With the 1.1 release we should not be creating bluestore OSDs
with rook's legacy partitioning scheme. All new bluestore OSDs
should be created with ceph-volume. Therefore, the miminum
version allowed is a version of mimic supporting ceph-volume.

This commit also adds consistent version checking between
the Add and Update methods for the operator handling the
CR events.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-08-13 07:22:59 -06:00
Dinar Valeev 38c4e5ed7b Mgr: fix prometheus annotations
This fixes https://github.com/rook/rook/issues/3204

Move prometheus Annoation logic before Deployment is initialized,
otherwise, podSpec is not uptodate

Signed-off-by: Dinar Valeev <k0da@opensuse.org>
2019-07-16 17:01:27 +02:00
Umanga Chapagain 732f3030d0 integrate prometheus based monitoring with rook-ceph
> add new field to cluster spec
> deploy servicemonitor
> deploy prometheusRule

Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
2019-07-08 11:40:17 +05:30
Noah Watkins fc7baef6aa ceph: add tests for dashboard http bind fix
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-04-30 10:57:37 -07:00
Alexander Trost 0c6eb7aa3d crds: Set annotations pods, deployments and so on
Allow the user to set annotations in the CRDs of the following operators which will then be set on
the resulting Pods (Deployments, StatefulSets and so on).
Operators:
* Cassandra
* Ceph
* CockroachDB
* EdgeFS
* Minio
* NFS

The Annotations related structures have been added to the
rook.io/v1alpha2 pkg.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2019-04-16 20:47:05 +02:00
Sébastien Han d4c48bb293 ceph: expose /var/log/ceph to the host
It's quite convinient to expose /var/log/ceph so that we can decide to
activate logs locally on the machine and see what's going on.
This is only a placeholder when a daemon is stuck crashlooping and we
want to allow administrator to gather log files.
We still do not log on file but this can be activated via a config
option passed to the centralized config option store.

For some daemons, which typically do not store any data (rgw, rbd-mirror
and mds) we had to propagate dataDirHostPath from the cluster spec to
each creation call so that the bindmount can happen.

Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-04-15 10:47:09 +02:00
rohan47 0ecce1015c ceph: update env vars for ceph-mgr containers
pass env vars with rook-ceph-system namespace and API version to
ceph-mgr containers to remove hardcoded values from rook orchestrator in
ceph-mgr and updated the unit tests for the same

Signed-off-by: rohan47 <rohgupta@redhat.com>
2019-04-04 02:11:44 +05:30
Blaine Gardner 7f26f4aa13 ceph spec tests: allow adding required env vars
Allow adding additional required environment variables on a situational
basis to Ceph's spec unit tests.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
(cherry picked from commit 94bf2a6e14b88a8e965b4ad5ce1289b606e5c32d)
2019-04-04 02:11:44 +05:30
Sébastien Han 871054e887 ceph: implement pod restriction on memory
If someone sets limits to pod, we want to ensure the possible
experience, so we want to make sure that people do not configure
inapropriate values for certain daemons.
We decide to fail if the memory.limit is too low.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-03-15 17:35:44 +01:00
Sébastien Han fa61c0441b ceph: pod: expose resource limits and requests
We now expose resource as environment variable in the pod.
They will be used by Ceph to report as a daemon metadata.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-03-15 11:45:29 +01:00
Blaine Gardner c12aa6c7c5 ceph rbd-mirror: configure entirely in operator
In the form of mon, mgr, mds, and rgw, convert the rbd-mirror to be
configured completely from the operator. This is a straightforward
conversion with one functional addition: the rbd-mirror daemon stores
*no* data and has no default data dir, so the concept of a `Dataless`
daemon is here introduced.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-03-13 09:54:09 -06:00
Blaine Gardner 4eb4fb7aa6 ceph mds: configure completely from operator
Configure the Ceph mds daemon completely from the operator a la the
recent changes to the Ceph mon and mgr operators.

Create the mds deployments first and then
create the keyring secrets for them with their owner reference as the
corresponding deployment. This will mean that the secrets do not need to
be micromanaged. When the deployment is deleted, the secret is also
deleted. This has not been necessary for the mons or the manager since
the mons share a keyring with a lifespan of the cluster, as does the
mgr, which currently has single-mgr support only.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-02-28 12:31:43 -07:00
Blaine Gardner 03ea5f0a7c ceph mgr: configure completely from operator
Configure the Ceph mgr daemon completely from the operator a la the
recent changes to the Ceph mon operator.

The pod spec for the mgr changed quite a bit, and instead of updating
the unit tests of questionable use, some additional unit test tools
applicable to any Ceph daemon have been added and used with the mgr. The
mgrs unit tests should now be more useful and get in the way of devs
less, and they can be used by other daemons later.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2019-02-11 14:35:01 -07:00
travisn 261aae3f46 mgr: set the server_addr for the dashboard and prometheus modules
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-20 12:08:37 -07:00
travisn 62f7e5d6ea ceph: update docs, code, and tests to use v1 crd types
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-05 14:33:11 -07:00
travisn 16c2a56591 add ceph mgr roles to the tests and helm chart
Signed-off-by: travisn <tnielsen@redhat.com>
2018-11-30 13:39:58 -07:00
travisn 6cfd2ee11a port 7000 for the luminous dashboard and 8443 for mimic and above
Signed-off-by: travisn <tnielsen@redhat.com>
2018-11-01 11:49:37 -06:00
travisn 94a7a0d4bc run daemons with the ceph image instead of rook
Signed-off-by: travisn <tnielsen@redhat.com>
2018-11-01 11:49:35 -06:00
Jeff Layton 6bab32b6f9 Ensure that $ROOK_CLUSTER_NAME is set for mgr daemon
We need this environment variable in the main container, not the init
container, as it's used by the rook ceph-mgr module.

Also fix up the env var counts in the testcase.

Fixes: ca71b7a311 (Ceph mgr: Set up config in init container)
Signed-off-by: Jeff Layton <jlayton@redhat.com>
2018-10-11 12:46:36 -04:00
Blaine Gardner ce803fbf9d Ceph mds/file: Set up config in init container
Progress toward issue #2003.
Includes design from design doc PR #1578

Use init containers to create configuration for Ceph mgrs. There is only
one init container in this design. The init container calls the Rook
binary to create Ceph config files which are then shared with the mds
daemon main container.

Once this init is run, the main mds daemon is run. Leaving room to use
the Ceph-versioned image in the future, call `ceph-mds --foreground ...`
to run the Ceph mds.

The refactor to using an init container also necessitated refactoring
the mdses replicaset implementation to a deployment-per-pod
implementation due to a chicken-egg problem. With a single container (in
the before times) the Rook binary was able to call the ceph-mds daemon
with an id generated from the pod name. Since the pod name is not known
before runtime, and the id is one of the few params that must be
specified to Ceph daemons on run, it is necessary to know the id
beforehand; thus the move to a deployment architecture following the
likes of the mon and mgr daemons.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2018-10-05 19:01:12 -06:00
Noah Watkins e015a21cbd ceph: expose node,pod,ns names to ceph daemons
Ceph expects to be able to reason about physical hardware / nodes.
Before this patch the NODE_NAME that was available referred to the pod
name. This patch exposes the underlying node name, pod, and pod
namespace for the ceph daemons to consume.

fixes: #2078

Signed-off-by: Noah Watkins <nwatkins@redhat.com>
2018-09-24 13:26:24 -07:00
Blaine Gardner ca71b7a311 Ceph mgr: Set up config in init container
Progress toward issue #2003.
Includes design from design doc PR #1578

Use init containers to create configuration for Ceph mgrs. There is only
1 init container in this design:
 1. Using the Rook image, call the Rook binary to create Ceph config
files shared with the mgr daemgr container.

Once this init is run, the main mgr daemgr is run. Leaving room to use
the Ceph-versioned image in the future, call `ceph-mgr --foreground ...`
to run the Ceph mgr.

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2018-09-21 13:11:08 -06:00