In order to cleanup all cluster resources in case of cluster deletion
all resources have an ownerreference to the cluster crd instance.
Because the name of the reference was incorrectly set to the name of the
namespace the cluster resources can unintentionally be deleted by the
kubernetes garbage collector after a kubernetes cluster anomaly.
Signed-off-by: René Kalff <r.kalff@intermax.nl>
The status messages on the CRs had much duplicated code
for logging an error if the status update failed.
Now any failures around the status update are logged
consistently within the updateClusterStatus method
so there is no need to return an error and duplicate
the error handling everywhere.
Signed-off-by: travisn <tnielsen@redhat.com>
Currently the ClusterController logs out a message like:
```
Cluster %s is not ready. Skipping orchestration.rook-ceph
```
without the "placeholders" fullfilled. This is because the code uses the
logger.Info() instead of logger.Infof() function.
This change fixes that to use Infof and therefore corrects the log
message.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
This commit makes OnAdd() and OnUpdate() consistent in behavior. They
will do their best detecting if we can upgrade, continue an upgrade or
refuse to do one. It also refuses to downgrade to any version.
It also load ClusterInfo earlier in the controller so that we can
compare the spec version versus the cluster running one.
Signed-off-by: Sébastien Han <seb@redhat.com>
If we successfully changed all the Ceph version of all the daemon we
consider the upgrade complete and successfully, thus we display a
message.
Signed-off-by: Sébastien Han <seb@redhat.com>
In the 0.9 release rook converted the v1beta1 CRD resources
to v1 resources. This conversion code is no longer necessary
as we will be using v1 resources going forward.
The code paths will not be triggered anymore, thus
removing the dead code.
Signed-off-by: travisn <tnielsen@redhat.com>
Use cmd-reporter to detect Ceph version.
The job to detect Ceph version needs to run in a security account context
capable of getting/creating ConfigMaps, so a new security account, role,
and role binding are created for the cmd-runner jobs. In the future, the
Ceph mgr may create jobs in this service account for running
`ceph-volume` on nodes, and this also keeps Rook operators controlling
multiple Ceph clusters independent.
Fixes#2686 - ceph version job not working correctly with graylog
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
If we are running a Ceph cluster that has an unsupported version and if
we have set allowUnsupported: false, a newly bootstrap operator will
fail to start. To fix this, the user will update the CR and set
allowUnsupported: true, however at this point clusterInfo hasn't been
initialized so the operator will panic because we were referencing a nil
pointer.
To fix this, we re-populate clusterInfo if necessary and things go
smoothly.
Closes: https://github.com/rook/rook/issues/3137
Signed-off-by: Sébastien Han <seb@redhat.com>
OwnerReferences are now created with the fully qualified apiVersion
such that the references will work properly on OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
When the operator first starts, the only operation needed
is to watch for new cephcluster crds to be created and
start the discovery to find available devices. The flexvolume
agent, the csi driver, and the volume provisioning can all be delayed
starting until the first cluster is created.
Signed-off-by: travisn <tnielsen@redhat.com>
Sometime the kube engine needs a bit of time to return the logs of a
given job and fails to read the stream.
Retrying up to detect the Ceph version seems reasonnable to
overcome this issue.
Fixes: https://github.com/rook/rook/issues/3227
Signed-off-by: Sébastien Han <seb@redhat.com>
This removes the unnecessary check for taints on nodes in the
`GetNodeSchedulable()` function. This fixes that even though the user
has specified `tolerations` for, e.g., `NoSchedule` taints, that would
be ignored and the node(s) directly be "marked" as unschedulable.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
this skips orchestration when the device configmap receives an update
but the cluster hasn't been fully configured, which may leave some state
undefined and cause nil pointer dereferences accessing the cluster info
structure.
fixes: #3001
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
this is an attempt to reduce the number of false positive config map
updates used to trigger orchestration for hotplugging.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
This reverts commit b7a564e929, reversing
changes made to 6996571b5c.
Reverting: Linking ceph noout to a node taint.
The taint is applied for reasons other than maintenance as well.
This also needs a way to respect user-set noout.
Need to revisit this with a better design.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
this adds a new operator environment variable that allows automatic
orchestration triggered by device discovery to be disabled.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
The CephCluster CR contains settings that are needed by other
CRs to configure the Ceph daemons. When the CephCluster CR
is updated, the updates will now be passed on to each of the
CR controllers to ensure the daemons are updated properly
without requiring an operator restart.
When calling the controllers from another controller,
we ensure that only a single goroutine is handling
CRs at any given time to prevent contention across
multiple CRs of the same type.
Signed-off-by: travisn <tnielsen@redhat.com>
the k8s Node add/update events can fire before a cluster is fully
configured, which may leave some state undefined. skip orchestration
when the cluster is in this state.
fixes: #3001
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
It's quite convinient to expose /var/log/ceph so that we can decide to
activate logs locally on the machine and see what's going on.
This is only a placeholder when a daemon is stuck crashlooping and we
want to allow administrator to gather log files.
We still do not log on file but this can be activated via a config
option passed to the centralized config option store.
For some daemons, which typically do not store any data (rgw, rbd-mirror
and mds) we had to propagate dataDirHostPath from the cluster spec to
each creation call so that the bindmount can happen.
Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
The CephCluster resource will now expose the health of the ceph cluster
so admins can query the status with k8s api or kubectl instead of
needing to run the rook toolbox. The operator will query the
ceph status periodically and update the status in the custom resource.
Signed-off-by: travisn <tnielsen@redhat.com>
The basic idea here is to replace the release name in the ClusterInfo
object with the fine grained version information queried at runtime.
This patch also removes the Name field from the cluster spec, which was
also runtime determined and was redundant with ClusterInfo relase name.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Configure the Ceph rgw daemon completely from the operator a la the
recent changes to the Ceph mon, mgr, and mds operators.
Create the rgw deployment or daemonset first, and then create the
keyring secret for the object store with its owner reference as the
corresponding deployment or daemonset. When the replication controller
is deleted, the secret is also deleted.
The RGW's mime.types file is now stored in a configmap with a different
file created for each object store. This is primarily just a means to
get the mime.types file into the rgw pod, but the added benefit is that
the administrator can modify the configmap, which could reduce
susceptibility to file type execution vulnerabilities (worst case).
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Configure the Ceph mds daemon completely from the operator a la the
recent changes to the Ceph mon and mgr operators.
Create the mds deployments first and then
create the keyring secrets for them with their owner reference as the
corresponding deployment. This will mean that the secrets do not need to
be micromanaged. When the deployment is deleted, the secret is also
deleted. This has not been necessary for the mons or the manager since
the mons share a keyring with a lifespan of the cluster, as does the
mgr, which currently has single-mgr support only.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
All devices detected by the discovery pod were being passed to the OSD provisioning pod
thus not always honoring the desired device list that should be provisioned.
Now the provisioning pod will be given the desired state from the crd,
then apply that state depending on the actual devices detected.
Also added a helper to ensure OSDsPerDevice is always valid.
Signed-off-by: travisn <tnielsen@redhat.com>
Noah Watkins
Travis Nielsen
René Kalff
ethan
Alexander Trost
Sébastien Han
Blaine Gardner
Rohan CJ
Stefan Haas
Ben Zieglmeier