If we are running a Ceph cluster that has an unsupported version and if
we have set allowUnsupported: false, a newly bootstrap operator will
fail to start. To fix this, the user will update the CR and set
allowUnsupported: true, however at this point clusterInfo hasn't been
initialized so the operator will panic because we were referencing a nil
pointer.
To fix this, we re-populate clusterInfo if necessary and things go
smoothly.
Closes: https://github.com/rook/rook/issues/3137
Signed-off-by: Sébastien Han <seb@redhat.com>
OwnerReferences are now created with the fully qualified apiVersion
such that the references will work properly on OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
When the operator first starts, the only operation needed
is to watch for new cephcluster crds to be created and
start the discovery to find available devices. The flexvolume
agent, the csi driver, and the volume provisioning can all be delayed
starting until the first cluster is created.
Signed-off-by: travisn <tnielsen@redhat.com>
Sometime the kube engine needs a bit of time to return the logs of a
given job and fails to read the stream.
Retrying up to detect the Ceph version seems reasonnable to
overcome this issue.
Fixes: https://github.com/rook/rook/issues/3227
Signed-off-by: Sébastien Han <seb@redhat.com>
This removes the unnecessary check for taints on nodes in the
`GetNodeSchedulable()` function. This fixes that even though the user
has specified `tolerations` for, e.g., `NoSchedule` taints, that would
be ignored and the node(s) directly be "marked" as unschedulable.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
this skips orchestration when the device configmap receives an update
but the cluster hasn't been fully configured, which may leave some state
undefined and cause nil pointer dereferences accessing the cluster info
structure.
fixes: #3001
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
this is an attempt to reduce the number of false positive config map
updates used to trigger orchestration for hotplugging.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
This reverts commit b7a564e929, reversing
changes made to 6996571b5c.
Reverting: Linking ceph noout to a node taint.
The taint is applied for reasons other than maintenance as well.
This also needs a way to respect user-set noout.
Need to revisit this with a better design.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
this adds a new operator environment variable that allows automatic
orchestration triggered by device discovery to be disabled.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
The CephCluster CR contains settings that are needed by other
CRs to configure the Ceph daemons. When the CephCluster CR
is updated, the updates will now be passed on to each of the
CR controllers to ensure the daemons are updated properly
without requiring an operator restart.
When calling the controllers from another controller,
we ensure that only a single goroutine is handling
CRs at any given time to prevent contention across
multiple CRs of the same type.
Signed-off-by: travisn <tnielsen@redhat.com>
the k8s Node add/update events can fire before a cluster is fully
configured, which may leave some state undefined. skip orchestration
when the cluster is in this state.
fixes: #3001
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
It's quite convinient to expose /var/log/ceph so that we can decide to
activate logs locally on the machine and see what's going on.
This is only a placeholder when a daemon is stuck crashlooping and we
want to allow administrator to gather log files.
We still do not log on file but this can be activated via a config
option passed to the centralized config option store.
For some daemons, which typically do not store any data (rgw, rbd-mirror
and mds) we had to propagate dataDirHostPath from the cluster spec to
each creation call so that the bindmount can happen.
Fixes: https://github.com/rook/rook/issues/2881
Signed-off-by: Sébastien Han <seb@redhat.com>
The CephCluster resource will now expose the health of the ceph cluster
so admins can query the status with k8s api or kubectl instead of
needing to run the rook toolbox. The operator will query the
ceph status periodically and update the status in the custom resource.
Signed-off-by: travisn <tnielsen@redhat.com>
The basic idea here is to replace the release name in the ClusterInfo
object with the fine grained version information queried at runtime.
This patch also removes the Name field from the cluster spec, which was
also runtime determined and was redundant with ClusterInfo relase name.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Configure the Ceph rgw daemon completely from the operator a la the
recent changes to the Ceph mon, mgr, and mds operators.
Create the rgw deployment or daemonset first, and then create the
keyring secret for the object store with its owner reference as the
corresponding deployment or daemonset. When the replication controller
is deleted, the secret is also deleted.
The RGW's mime.types file is now stored in a configmap with a different
file created for each object store. This is primarily just a means to
get the mime.types file into the rgw pod, but the added benefit is that
the administrator can modify the configmap, which could reduce
susceptibility to file type execution vulnerabilities (worst case).
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Configure the Ceph mds daemon completely from the operator a la the
recent changes to the Ceph mon and mgr operators.
Create the mds deployments first and then
create the keyring secrets for them with their owner reference as the
corresponding deployment. This will mean that the secrets do not need to
be micromanaged. When the deployment is deleted, the secret is also
deleted. This has not been necessary for the mons or the manager since
the mons share a keyring with a lifespan of the cluster, as does the
mgr, which currently has single-mgr support only.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
All devices detected by the discovery pod were being passed to the OSD provisioning pod
thus not always honoring the desired device list that should be provisioned.
Now the provisioning pod will be given the desired state from the crd,
then apply that state depending on the actual devices detected.
Also added a helper to ensure OSDsPerDevice is always valid.
Signed-off-by: travisn <tnielsen@redhat.com>
Default to 3 mons when no `mon.count` is given.
The documentation and upgrade page has been updated accordingly to reflect the new
default.
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
This introduces a MonSpec to the Ceph Cluster CRD which contains settings for
mons, current options are the count and if more than one mon is allowed
to run per node.
Fixes#1386
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
Sébastien Han
travisn
Blaine Gardner
Alexander Trost
Noah Watkins
Rohan CJ
Stefan Haas
Ben Zieglmeier
Huamin Chen
Jared Watts