Refactor the Ceph NFS (Ganesha) daemon to use the new config pattern.
Use of a custom Rook binary to run NFS has been removed in favor of
instead running the required DBUS process as a sidecar. The effect here
is that the NFS Ganesha container itself may fail and restart a few
times before the DBUS sidecar is running.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
OpenShift environment requires privileged containers when interacting
with the host filesystem, typically via dataDirHostPath.
The init container that changes the ownership of dataDirHostPath was
missing the privileged flag and users were getting permission denied
errors on the chown command.
Fixes: https://github.com/rook/rook/issues/3485
Signed-off-by: Sébastien Han <seb@redhat.com>
At the beginning of every container startup we now change the ownership
of every known ceph directories to ceph:ceph.
This will allow daemons to start properly as well as allowing daemon to
writes their logs on files.
This is needed because ceph daemon now run as 'ceph' instead of 'root'
Signed-off-by: Sébastien Han <seb@redhat.com>
We do not determine the daemon type and name by string parsing anymore
but passing that info directly in the update call, it's more reliable.
Also removing a bunch of unused functions.
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit makes OnAdd() and OnUpdate() consistent in behavior. They
will do their best detecting if we can upgrade, continue an upgrade or
refuse to do one. It also refuses to downgrade to any version.
It also load ClusterInfo earlier in the controller so that we can
compare the spec version versus the cluster running one.
Signed-off-by: Sébastien Han <seb@redhat.com>
If we successfully changed all the Ceph version of all the daemon we
consider the upgrade complete and successfully, thus we display a
message.
Signed-off-by: Sébastien Han <seb@redhat.com>
Do not consider clusterInfo Ceph version since it's the one picked by
the spec image.
The CephVersion provided here needs to be the current running version
of Ceph for when it is later passed to OkToStop and checked with
IsAtLeast. If you are upgrading from a pre 14.2.1 version to a
version 14.2.1 or newer, IsAtLeast will incorrectly return true and
then fail trying to run mon ok-to-stop on a mon pod running a pre 14.2.1 version.
Signed-off-by: Sébastien Han <seb@redhat.com>
These were used on a previous patch as a debug, let's remove them. They
are not intended to be displayed.
Signed-off-by: Sébastien Han <seb@redhat.com>
This method does not start anything but creates the deployment. So let's
rename it to createDeployment() instead.
Signed-off-by: Sébastien Han <seb@redhat.com>
rgw wasn't using UpdateDeploymentAndWait() to update its deployment
configuration. We now use it so that it's consistent with the other
daemons configuration.
Signed-off-by: Sébastien Han <seb@redhat.com>
When a cluster is updated with a different image version, this triggers
a serialized restart of all the pods. Prior to this commit, no safety
check were performed and rook was hoping for the best outcome.
Now before doing restarting a daemon we check it can be restarted. Once
it's restarted we also check we can pursue with the rest of the
platform. For instance, with monitors we check that they are in quorum,
for OSD we check that PGs are clean and for MDS we make sure they are
all active.
Fixes: https://github.com/rook/rook/issues/2889
Signed-off-by: Sébastien Han <seb@redhat.com>
In the 0.9 release rook converted the v1beta1 CRD resources
to v1 resources. This conversion code is no longer necessary
as we will be using v1 resources going forward.
The code paths will not be triggered anymore, thus
removing the dead code.
Signed-off-by: travisn <tnielsen@redhat.com>
Add missing docstrings for container image
functions in pod.go, noting the behavior
when getting a container image from a pod
which only has a single container defined.
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
Follow-up to #3436:
The validation module has a defined constant for
label value max length. It makes sense to use it
here.
Signed-off-by: Kristoffer Grönlund <kgronlund@suse.com>
this patch introduces failure-domain aware scheduling for monitor pods.
previously the only policy was to avoid scheduling monitors on the same
nodes. the new policy tries to spread monitors across zones and nodes.
NOTE: this patch only enforces the well-known k8s "zone"
failure-domain label (i.e. it does not look for region labels). this
decision is made for two reasons. first, eliminating explicit
scheduling is an important goal and will be done as part of upcoming
work. this will enable all supported failure-domain labels to be
handled. second, the "zone" domain is likely more appropriate for a
larger portion of users (as opposed to the geographical distinction
made with region), and supporting multiple failure domain types would
add significant complexity for a temporary resolution to failure
domain scheduling.
at a high-level this patch introduces a data structure `NodeUsage` that
is a pairing of a v1.Node and metadata relevant to scheduling: number of
monitor pods on the node, and its status w.r.t. to being scheduable for
new monitor pods.
each scheduling event computes a unified data structure that organizes
for each node a `NodeUsage` structure into per-failure-zone groups.
subsequent algorithms rely soley on this unified structure to make
scheduling decisions.
there are two entry points to the changes made:
1) during orchestration mon:Cluster:assignMons is invoked to make a
place new monitor pods onto nodes. the core scheduling logic is now
self-contained in mon:scheduleMonitor which implements node and zone
aware scheduling policies.
2) periodic health checks resolve any conflicts that may occur due to
changes to the cluster (e.g. new nodes, configuration changes, etc...).
the two existing checks: mons on invalid nodes, and overloaded nodes are
both handled.
fixes: #2603
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Empty deployment list made the operator to crash as the condition check
was checking if the value at the 0th location is nill wich fails in case
of empty list. Updated it to check length of list not equals 0.
Signed-off-by: rohan47 <rohgupta@redhat.com>
Example of failure scenarios:
- requested PVC 500M = 500,000,000 bytes
- PV is created as:
(500M / 2**20)Mi = "476"Mi
But 476Mi = 476 * 2**20 = 499122176 < 500M
From the k8s documentation:
Claims will remain unbound indefinitely if a matching volume does not
exist. Claims will be bound as matching volumes become available. For
example, a cluster provisioned with many 50Gi PVs would not match a
PVC requesting 100Gi. The PVC can be bound when a 100Gi PV is added
to the cluster.
which is inline with the behavior observed in which a 500M PVC is never
bound. However, the documentation also states:
If a PV was dynamically provisioned for a new PVC, the loop will
always bind that PV to the PVC. Otherwise, the user will always get
at least what they asked for
which suggests that an _explicit_ binding can be created, and perhaps
something has changed with respect to this binding that has caused the
failures in newer versions of k8s.
fixes: #3391fixes: #2922
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
The beast fronted configuration for rgw is quite different than civetweb
so we adapted the construct of the arg line based on that configuration
requirement.
Closes: https://github.com/rook/rook/issues/3411
Signed-off-by: Sébastien Han <seb@redhat.com>
This commit does:
* remove dead code SetCrushTunables()
* stop creating initial crushmap
There is no need to create an initial crushmap, since Ceph natively does
that for us.
We don't delete the existing configmap containing the initial crush map
of already deployed cluster, perhaps people are using for whatever
reason. It does not harm to keep it around.
Closes: https://github.com/rook/rook/issues/3138
Signed-off-by: Sébastien Han <seb@redhat.com>
Use cmd-reporter to detect Ceph version.
The job to detect Ceph version needs to run in a security account context
capable of getting/creating ConfigMaps, so a new security account, role,
and role binding are created for the cmd-runner jobs. In the future, the
Ceph mgr may create jobs in this service account for running
`ceph-volume` on nodes, and this also keeps Rook operators controlling
multiple Ceph clusters independent.
Fixes#2686 - ceph version job not working correctly with graylog
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Add a wrapper for the cmd-reporter function of the 'rook' binary to
k8sutil so that it's easier for operators to use the functionality.
Operators still need to have some idea of how CmdReporter works. It runs
the requested command in a job, but if the calling operator needs the
job to have special configuration, the operator must request the pointer
to the job struct from CmdReporter to modify it before calling the
CmdReporter's Run method.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Blaine Gardner
Travis Nielsen
xieyanker
Sébastien Han
Rohan CJ
Mateusz Gozdek
Dmitry Yusupov
Anton Skriptsov
Alexander Trost
Dinar Valeev
Kristoffer Grönlund
Noah Watkins
rohan47
Umanga Chapagain