We need this environment variable in the main container, not the init
container, as it's used by the rook ceph-mgr module.
Also fix up the env var counts in the testcase.
Fixes: ca71b7a311 (Ceph mgr: Set up config in init container)
Signed-off-by: Jeff Layton <jlayton@redhat.com>
Progress toward issue #2003.
Includes design from design doc PR #1578
Use init containers to create configuration for Ceph mgrs. There is only
one init container in this design. The init container calls the Rook
binary to create Ceph config files which are then shared with the mds
daemon main container.
Once this init is run, the main mds daemon is run. Leaving room to use
the Ceph-versioned image in the future, call `ceph-mds --foreground ...`
to run the Ceph mds.
The refactor to using an init container also necessitated refactoring
the mdses replicaset implementation to a deployment-per-pod
implementation due to a chicken-egg problem. With a single container (in
the before times) the Rook binary was able to call the ceph-mds daemon
with an id generated from the pod name. Since the pod name is not known
before runtime, and the id is one of the few params that must be
specified to Ceph daemons on run, it is necessary to know the id
beforehand; thus the move to a deployment architecture following the
likes of the mon and mgr daemons.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Ceph expects to be able to reason about physical hardware / nodes.
Before this patch the NODE_NAME that was available referred to the pod
name. This patch exposes the underlying node name, pod, and pod
namespace for the ceph daemons to consume.
fixes: #2078
Signed-off-by: Noah Watkins <nwatkins@redhat.com>
Progress toward issue #2003.
Includes design from design doc PR #1578
Use init containers to create configuration for Ceph mgrs. There is only
1 init container in this design:
1. Using the Rook image, call the Rook binary to create Ceph config
files shared with the mgr daemgr container.
Once this init is run, the main mgr daemgr is run. Leaving room to use
the Ceph-versioned image in the future, call `ceph-mgr --foreground ...`
to run the Ceph mgr.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Add a 'test' package to the Ceph operator and define a test to verify
that containers produced by Rook-Ceph match what is expected. Because
this is for 'containers' and not strictly for 'ceph containers', this
could be moved a level up to the operator test package; however, if
other backends wish to use the container tests, they will likely need to
make modifications, and there is concern that this might make the Ceph
tests brittle.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Add a package called 'spec' to the Ceph operator to store resource
definition specs that are shared by all Ceph daemons.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Add functions for helping test Kubernetes volumes and volume mounts.
- Create functions for testing the existence of volumes/mounts by
name in a list of vols/mounts without needing to know the index of
the vol/mount in the list.
- Create functions for printing vols/mounts in a human-readable format
so that tests may output more useful errors.
- Create a test definition for ensuring that all the volumes a pod
provides and all the mounts from each pod container match. For each
volume, there must be at least one container which mounts the volume,
and for each mount, there must be a source volume.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Progress toward issue #2003.
Includes design from design doc PR #1578
Use init containers to create configuration for Ceph mons. There are 3
init containers in this design:
1. Using the Rook image, call the Rook binary to create Ceph config
files ultimately stored in a k8s EmptyDir which is shared between all
containers in a mon pod.
2. Leaving room to use the Ceph-versioned image in the future, call
`monmaptool` to generate the monmap, stored in a shared config dir.
3. Leaving room to use the Ceph-versioned image in the future, call
`ceph-mon --mkfs ...` to set up the mon filesystem on the host's disk.
Once this init is run, the main mon daemon is run. Leaving room to use
the Ceph-versioned image in the future, call `ceph-mon --foreground ...`
to run the Ceph mon.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
To help with code readability, include daemons from `pkg/daemon/ceph`
as `<dmn>daemon "github.com/rook/rook/pkg/daemon/ceph/<dmn>`. This will
make it easier to understand code lines without needing to scroll up to
check the includes, as there are mon/mds/mgr/osd/rgw packages for
daemons as well as operator and cmd.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Create a cephconfig module in Ceph's daemon pkg source, and refactor the
config and keyring generation that exists in the mon package into the
new cephconfig package. The config/keyring generation code is used by
most all daemons and not just mon, so a new package is a more
appropriate place for this.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Rook no longer supports Kubernetes < 1.7
Remove a workaround for k8s < 1.6.4 where `runAsNonRoot` could not be
explicitly set to false.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
We don't need to handle other osd(s) related tasks if there's
no valid nodes available in the cluster to run osd(s), log a
warning message to help user identify the cause of the issue.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Fix some basic spellcheck errors. Also remove trailing spaces and make
sure files have a newline (my editor does automatically).
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
This fixes issues with a new OSD Pod in the same Deployment coming up and
directly entering CrashLoopBackOff because the old OSD Pod hasn't been fully
terminated yet (released it's locks).
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Previously, if this failed mid-way it would then throw
an exception at the first operation when retrying (because it's
trying to reweight an OSD that might not be in the map any more)
Signed-off-by: John Spray <john.spray@redhat.com>
travisn
Jeff Layton
Blaine Gardner
Noah Watkins
Huamin Chen
Sage Weil
Alexander Trost
Dennis Chen
John Spray
Jared Watts