We must expose the ceph version to the osd pod so that we can generate
the proper ceph configuration file. If we don't do this all the version
check will assume they run the opposite of the desired version and the
formatting of the `ceph.conf` will be wrong.
Signed-off-by: Sébastien Han <seb@redhat.com>
Let's return the expected pattern so that can be consumed by
ExtractCephVersion later. We don't modify the built-in String() so we
can more easily remove that once we switch the osd to be configured from
the operator.
Signed-off-by: Sébastien Han <seb@redhat.com>
We now run all the daemon with a new option from Nautilus 14.2.1 which
allows us to tell to a daemon to not log on file. However, we can decide
to activate logging by editing the configuration flag 'log_to_file' via
the centralized config store like this for a particular daemon:
ceph config set mon.a log_to_file true
This is useful when a daemon keeps crashing and we want to collect log
files on the system.
Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
It's quite convinient to expose /var/log/ceph so that we can decide to
activate logs locally on the machine and see what's going on.
This is only a placeholder when a daemon is stuck crashlooping and we
want to allow administrator to gather log files.
We still do not log on file but this can be activated via a config
option passed to the centralized config option store.
For some daemons, which typically do not store any data (rgw, rbd-mirror
and mds) we had to propagate dataDirHostPath from the cluster spec to
each creation call so that the bindmount can happen.
Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
When instanciating an osd, we now pass clusterInfo which basically
represents a collection of information about a particular Ceph cluster. Rook uses information
// about the cluster to configure daemons to connect to the desired cluster.
From clusterInfo, we use the CephVersion which helps us putting
conditions for our code execution.
Signed-off-by: Sébastien Han <seb@redhat.com>
Add a `rook-version` label to all controller resources used by Ceph:
deployments, daemonsets, and jobs. Labels are added to controller
resources only and not to the pod templates within because changing pod
templates causes updates due to the label change. The controller
resources themselves are not updated with the label addition and
therefore don't cause an unnecessary upgrade.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
This fixes issues for users trying to delete volumes and failing because
they only have `clusterNames` set on them in Rook earlier v0.8.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
A ceph filesystem was brought down by marking the filesystem
as down and individually failing the MDSes. From nautilus onwards,
this could be more simply and efficiently done by using
`fs fail` command. See,
https://github.com/ceph/ceph/commit/4c49f165ec1Resolves: #2841
Signed-off-by: Ramana Raja <rraja@redhat.com>
All usages of k8s go client are now also using the versioned `AppsV1() `
call for the client.
Updated MySQL and Wordpress, and Kube Registy examples to use apps/v1
Deployments.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Make the Ceph operator more cautious about when it decides to remove
nodes from the Rook-Ceph cluster which are acting as osd hosts.
When `useAllNodes` is set to `true` we assume that the user wants to
have the most hands-off experience. Node removals are allowed when a
node is delted from Kubernetes and when a node has its taints/affinities
modified by the user (but not by automatic k8s modification as much as
possible).
When `useAllnodes` is set to `false` the only time a node is removed is
if it is removed from the Ceph cluster definition.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
The CephCluster resource will now expose the health of the ceph cluster
so admins can query the status with k8s api or kubectl instead of
needing to run the rook toolbox. The operator will query the
ceph status periodically and update the status in the custom resource.
Signed-off-by: travisn <tnielsen@redhat.com>
Node validity is not always as simple as a true/false. Refactor each
validity test into its own function for operators that wish to check
validity more granularly. Ceph will use these changes.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
The Ceph upgrade test is the only one which uses the
`WaitForDeploymentImage` method, and it has to be configured to wait
longer after upgrade at this point. Since the wait time is still
hard-coded, this method is moved to the operator's test dir to make
it clear that the method is suitable only for tests currently.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
When deleting a CephObjectStoreUser the deleteUser function
attempts to delete the stored secret using the wrong name. This
patch updates deleteUser to use the same fmt.Sprintf call used in
the createUser function.
Updated CI to delete the correct CRD and check the secret is deleted.
Signed-off-by: Keith Schincke <keith.schincke@gmail.com>
To simplify the deployment of Rook, both the operator and cluster crd are now created in the
same namespace in the examples. By default the operator will only manage a cluster crd
in the same namespace instead of watching for clusters in all namespaces. To manage a cluster
in another namespace, the roles in latter half of operator-common.yaml need to point to the new
namespace and the operator env var ROOK_CURRENT_NAMESPACE_ONLY should be set to false.
Signed-off-by: travisn <tnielsen@redhat.com>
Upgrading a Ceph objectstore was failing the upgrade test because the
existing method for handling the case where update errors occurred
because labels and label selectors have changed was not robust enough.
Make this more robust by attempting delete-and-create if a normal update
does not work on an rgw deployment.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Update integration test suite fails on an error with getting the keyring
for the mgr. The mgr's cephx capabilities have changed and need to be
updated before get, create, or get-or-create can be called. Since other
daemons' caps may change in the future, make an attempt to update the
user's caps on any key generation. This makes the key generation more
declarative from within the operator code for individual Ceph daemons.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
The Ceph mon flag `--public-bind-addr` does not keep the same port as
the previous daemon run when the port is left off the flag's value. A
port is undesired since that could interfere with Nautilus' msgr2
protocol, so instead configure the mon's services to forward the default
bind port on pods (6789) to the mon endpoint's port. The endpoint port
will be 6789 for new mons, but it may be 6790 for legacy clusters.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
pass env vars with rook-ceph-system namespace and API version to
ceph-mgr containers to remove hardcoded values from rook orchestrator in
ceph-mgr and updated the unit tests for the same
Signed-off-by: rohan47 <rohgupta@redhat.com>
Allow adding additional required environment variables on a situational
basis to Ceph's spec unit tests.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
(cherry picked from commit 94bf2a6e14b88a8e965b4ad5ce1289b606e5c32d)
The desired number of mons could change depending on the number of nodes in a cluster.
For example, three mons would be the min number of mons in a production cluster with at
least three nodes. If there are five or more nodes, the number of mons could increase
to five in order to increase the failure tolerance to two nodes.
This is accomplished by a new setting in the cluster CRD preferredCount. If the number
of hosts exceeds preferredCount, more mons are added to quorum. If the number
of hosts drops below the preferred count, the operator would reduce the quorum size to
the smaller desired count.
Signed-off-by: travisn <tnielsen@redhat.com>
The basic idea here is to replace the release name in the ClusterInfo
object with the fine grained version information queried at runtime.
This patch also removes the Name field from the cluster spec, which was
also runtime determined and was redundant with ClusterInfo relase name.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Add a new package for parsing and interacting with ceph version
information. Previously ceph versions were limited to the release name,
and this patch supports ceph versions parts X.Y.Z for more fine grained
behavior.
The main structure introduced here in CephVersion which contains the
version parts. This will be used in the next patch to replace the Name
string component of the cluster CRD spec which was populated at runtime.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Sébastien Han
Alexander Trost
Travis Nielsen
Blaine Gardner
Ramana Raja
Stefan Haas
John Mulligan
Madhu Rajanna
Dmitry Yusupov
Keith Schincke
rohan47
Anton Skriptsov
Noah Watkins