Empty deployment list made the operator to crash as the condition check
was checking if the value at the 0th location is nill wich fails in case
of empty list. Updated it to check length of list not equals 0.
Signed-off-by: rohan47 <rohgupta@redhat.com>
Linear raid can be created using `mdadm --create --level=linear`,
By creating a linear raid disk on top of a logical volume,
we can pass the linear device to `ceph-volume lvm batch --prepare`
to provision a new OSD on top of the logical volume since
`ceph-volume` does not take lv as the data device.
Signed-off-by: Ash Wu <hSATAC@gmail.com>
Example of failure scenarios:
- requested PVC 500M = 500,000,000 bytes
- PV is created as:
(500M / 2**20)Mi = "476"Mi
But 476Mi = 476 * 2**20 = 499122176 < 500M
From the k8s documentation:
Claims will remain unbound indefinitely if a matching volume does not
exist. Claims will be bound as matching volumes become available. For
example, a cluster provisioned with many 50Gi PVs would not match a
PVC requesting 100Gi. The PVC can be bound when a 100Gi PV is added
to the cluster.
which is inline with the behavior observed in which a 500M PVC is never
bound. However, the documentation also states:
If a PV was dynamically provisioned for a new PVC, the loop will
always bind that PV to the PVC. Otherwise, the user will always get
at least what they asked for
which suggests that an _explicit_ binding can be created, and perhaps
something has changed with respect to this binding that has caused the
failures in newer versions of k8s.
fixes: #3391fixes: #2922
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
The beast fronted configuration for rgw is quite different than civetweb
so we adapted the construct of the arg line based on that configuration
requirement.
Closes: https://github.com/rook/rook/issues/3411
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit does:
* remove dead code SetCrushTunables()
* stop creating initial crushmap
There is no need to create an initial crushmap, since Ceph natively does
that for us.
We don't delete the existing configmap containing the initial crush map
of already deployed cluster, perhaps people are using for whatever
reason. It does not harm to keep it around.
Closes: https://github.com/rook/rook/issues/3138
Signed-off-by: Sébastien Han <seb@redhat.com>
Use cmd-reporter to detect Ceph version.
The job to detect Ceph version needs to run in a security account context
capable of getting/creating ConfigMaps, so a new security account, role,
and role binding are created for the cmd-runner jobs. In the future, the
Ceph mgr may create jobs in this service account for running
`ceph-volume` on nodes, and this also keeps Rook operators controlling
multiple Ceph clusters independent.
Fixes#2686 - ceph version job not working correctly with graylog
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Add a wrapper for the cmd-reporter function of the 'rook' binary to
k8sutil so that it's easier for operators to use the functionality.
Operators still need to have some idea of how CmdReporter works. It runs
the requested command in a job, but if the calling operator needs the
job to have special configuration, the operator must request the pointer
to the job struct from CmdReporter to modify it before calling the
CmdReporter's Run method.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Add a `DeleteResource` function that accepts new `DeleteOptions` and
`WaitOptions` structs as inputs. This new funciton is reusable and
extensible for any Kubernetes resource.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
cmd-reporter runs a command and puts the results of the command into a
ConfigMap defined by its `--config-map-name` commandline flag.
cmd-reporter is intended to be run as a Kubernetes job where an
operator needs to run a single command and get return information back
from the command which is more than just success/failure.
The ConfigMap returned includes `stdout`, `stderr`, and `retcode` data
from the command. The ConfigMap may be preexisting, in which case
cmd-reporter will overwrite the return information in the ConfigMap.
cmd-reporter labels the ConfigMap with `app=cmd-reporter`, and if the
ConfigMap already has an app label which is different, cmd-reporter
fails without modifying the ConfigMap so as not to overwrite another
application's data in the event of a ConfigMap name+namespace collision.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Previously, Rook Agent and Discovery DaemonSet deployment didn't allow
adding nodeAffinity. This commit adds nodeAffinity spec to daemonSet
deployment, which can be configured through environment variables in
operator deployment yaml.
+ Support multiple LabelKey, each with multiple LabelValue
+ Support multiple LabelKey with no value
Signed-off-by: Umanga Chapagain <chapagainumanga@gmail.com>
If we are running a Ceph cluster that has an unsupported version and if
we have set allowUnsupported: false, a newly bootstrap operator will
fail to start. To fix this, the user will update the CR and set
allowUnsupported: true, however at this point clusterInfo hasn't been
initialized so the operator will panic because we were referencing a nil
pointer.
To fix this, we re-populate clusterInfo if necessary and things go
smoothly.
Closes: https://github.com/rook/rook/issues/3137
Signed-off-by: Sébastien Han <seb@redhat.com>
Somehow during startup the arg got misinterpreted by ceph-osd and was
failing with:
2019-06-19 15:23:09.098967 I | exec: Running command: ceph-osd --foreground --id 0 --osd-uuid feceb7d7-3148-4a0d-af7a-753ba2f21a92 --conf /var/lib/rook/osd0/rook-ceph.config --cluster ceph --default-log-to-file false --ms-learn-addr-from-peer false
2019-06-19 15:23:09.135921 I | unrecognized arg false
failed to start osd. Failed to complete '': exit status 1.
Now we concatenate the whole flag to avoid that.
Signed-off-by: Sébastien Han <seb@redhat.com>
OwnerReferences are now created with the fully qualified apiVersion
such that the references will work properly on OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
The check for whether to set the OwnerReferences is no longer
necessary. When passing the correct value to the apiVersion,
the OwnerReferences can be successfully set for OpenShift.
Signed-off-by: travisn <tnielsen@redhat.com>
This requirement is long gone and was reported in 2017. Clients should
have been updated by then, so there is no reason to set the CRUSH
tunable to such an old client. Actually, we should let Ceph run its own
tunable. They can always be changed later. Also, this was really
restrictive and applied on every orchestration and thus would override
any other config done by an administrator. Even worse, this will trigger
data movement back and forth...
Closes: https://github.com/rook/rook/issues/3138
Signed-off-by: Sébastien Han <seb@redhat.com>
Similarly to #2675, try to find the rook binary
on PATH in case it isn't found in /usr/local/bin/rook.
Distributions generally disallow installing files
into /usr/local, as it is reserved for non-system
installations by the FHS.
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
Signed-off-by: Guy Margalit <guymguym@gmail.com>
Co-Authored-By: Sébastien Han <seb@redhat.com>
Co-Authored-By: Travis Nielsen <tnielsen@redhat.com>
This change is meant to allow running operators locally on a developer machine.
The idea is to allow faster development cycles by reducing the time and complexity of building -> deploying -> debugging on cluster.
For operators that rely only on kubernetes API this works easily - see cockroachdb and minio examples in development-flow doc.
The change includes:
- rook.NewContext() - Refactored to remove repeating initialization code that was copy-pasted in most of the operators in order to create the clusterd.Context and the Clientsets. Also it detects the mode of working in-cluster vs external and sets up the external mode with standard user config (~/.kube/config) and a job executor.
- rook.GetOperatorImage() - Refactor this repeating code in many operators to detect the operator pod image. Also added a global flag --operator-image that developers can use to override this when running locally.
- rook.TerminateOnError() - Added a convenient function.
When the operator first starts, the only operation needed
is to watch for new cephcluster crds to be created and
start the discovery to find available devices. The flexvolume
agent, the csi driver, and the volume provisioning can all be delayed
starting until the first cluster is created.
Signed-off-by: travisn <tnielsen@redhat.com>
Sébastien Han
Kristoffer Grönlund
Blaine Gardner
rohan47
Travis Nielsen
Ash Wu
Noah Watkins
Umanga Chapagain
Dmitry Yusupov
Dmitry Yusupov
Kiefer Chang
Anton Skriptsov
Madhu Rajanna
Alexander Trost
Guy Margalit
xiaorui.zou