Upgrading a Ceph objectstore was failing the upgrade test because the
existing method for handling the case where update errors occurred
because labels and label selectors have changed was not robust enough.
Make this more robust by attempting delete-and-create if a normal update
does not work on an rgw deployment.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Update integration test suite fails on an error with getting the keyring
for the mgr. The mgr's cephx capabilities have changed and need to be
updated before get, create, or get-or-create can be called. Since other
daemons' caps may change in the future, make an attempt to update the
user's caps on any key generation. This makes the key generation more
declarative from within the operator code for individual Ceph daemons.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
The Ceph mon flag `--public-bind-addr` does not keep the same port as
the previous daemon run when the port is left off the flag's value. A
port is undesired since that could interfere with Nautilus' msgr2
protocol, so instead configure the mon's services to forward the default
bind port on pods (6789) to the mon endpoint's port. The endpoint port
will be 6789 for new mons, but it may be 6790 for legacy clusters.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
pass env vars with rook-ceph-system namespace and API version to
ceph-mgr containers to remove hardcoded values from rook orchestrator in
ceph-mgr and updated the unit tests for the same
Signed-off-by: rohan47 <rohgupta@redhat.com>
Allow adding additional required environment variables on a situational
basis to Ceph's spec unit tests.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
(cherry picked from commit 94bf2a6e14b88a8e965b4ad5ce1289b606e5c32d)
The desired number of mons could change depending on the number of nodes in a cluster.
For example, three mons would be the min number of mons in a production cluster with at
least three nodes. If there are five or more nodes, the number of mons could increase
to five in order to increase the failure tolerance to two nodes.
This is accomplished by a new setting in the cluster CRD preferredCount. If the number
of hosts exceeds preferredCount, more mons are added to quorum. If the number
of hosts drops below the preferred count, the operator would reduce the quorum size to
the smaller desired count.
Signed-off-by: travisn <tnielsen@redhat.com>
The basic idea here is to replace the release name in the ClusterInfo
object with the fine grained version information queried at runtime.
This patch also removes the Name field from the cluster spec, which was
also runtime determined and was redundant with ClusterInfo relase name.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Add a new package for parsing and interacting with ceph version
information. Previously ceph versions were limited to the release name,
and this patch supports ceph versions parts X.Y.Z for more fine grained
behavior.
The main structure introduced here in CephVersion which contains the
version parts. This will be used in the next patch to replace the Name
string component of the cluster CRD spec which was populated at runtime.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
If someone sets limits to pod, we want to ensure the possible
experience, so we want to make sure that people do not configure
inapropriate values for certain daemons.
We decide to fail if the memory.limit is too low.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the spec of the OSD daemon contains resource limits, especially
memory.limit we will use that value for setting the osd memory target
value so the bluestore OSD knows the amount of memory available for its
cache.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the spec of the MDS daemon contains resource limits, especially
memory.limit we will use that value for setting the mds cache memory
limit value so the MDS knows the amount of memory available.
Signed-off-by: Sébastien Han <seb@redhat.com>
We now expose resource as environment variable in the pod.
They will be used by Ceph to report as a daemon metadata.
Signed-off-by: Sébastien Han <seb@redhat.com>
Only the args necessary have traditionally been passed to the mon package.
This has led to many properties in the mon struct, whereas we will
now simply pass the cluster crd spec to simplify.
Signed-off-by: travisn <tnielsen@redhat.com>
The orchestrations need to be triggered on any update to the crd.
Users today restart the operator for various scenarios when the
operator could take care of it automatically. If unsupported options
are updated, they should just be ignored.
There are two goroutines that are working with the mons.
The first is an orchestration that is triggered by the operator
at startup, or when the cluster crd is updated. The second
is the health check that triggers periodically by default every
45 seconds. These goroutines must not try to make updates at the
same time. A mutex is added so one will block if the other is
still active. The mutex is only active for the duration of working
with mons, and not an entire orchestration of mgr, osd, etc
Signed-off-by: travisn <tnielsen@redhat.com>
In the form of mon, mgr, mds, and rgw, convert the rbd-mirror to be
configured completely from the operator. This is a straightforward
conversion with one functional addition: the rbd-mirror daemon stores
*no* data and has no default data dir, so the concept of a `Dataless`
daemon is here introduced.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
From ceph mimic (13.x) onwards, `mds deactivate` command is
deprecated. To scale down the number of active MDSes of a filesystem,
it's sufficient to set the desired number of active MDSes using
'max_mds' setting, and is no longer required to individually
deactivate the unwanted MDSes.
Resolves: #2725
Signed-off-by: Ramana Raja <rraja@redhat.com>
Configure the Ceph rgw daemon completely from the operator a la the
recent changes to the Ceph mon, mgr, and mds operators.
Create the rgw deployment or daemonset first, and then create the
keyring secret for the object store with its owner reference as the
corresponding deployment or daemonset. When the replication controller
is deleted, the secret is also deleted.
The RGW's mime.types file is now stored in a configmap with a different
file created for each object store. This is primarily just a means to
get the mime.types file into the rgw pod, but the added benefit is that
the administrator can modify the configmap, which could reduce
susceptibility to file type execution vulnerabilities (worst case).
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
Travis Nielsen
Blaine Gardner
Sébastien Han
rohan47
Dmitry Yusupov
Alexander Trost
Anton Skriptsov
Noah Watkins
Stefan Haas
Jose A. Rivera
Ramana Raja
zhoulin xie