This patch brings a mechanism to define arbitrary environment
variables in OSD containers (both prepare and main ones).
It bases on idea proposed by Sebastien Han to use an optional
`ConfigMap` instance named `aook-ceph-daemon-env-override`
as the source of these settings.
The need for the patch comes from the fact that, although
crimson finally exposes the same CLI interface as the classical
OSD, the broadly used development builds have ASan built in.
As ASan, by default, complains if it isn't the very first loaded
DSO, we need a way to set the `ASAN_OPTIONS` environment variable
to `verify_asan_link_order=0` to mitigate the early aborts.
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
If the pool application is already set, skip setting it again
to avoid a warning message being logged that it is already
set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The certs for accessing TLS enabled RGW is saved as secrets and inject
them if controllers for notification and topics if request is sent to
TLS enabled RGW endpoint.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Signed-off-by: Jiffin Tony Thottan <jthottan@redhat.com>
This commit updates the csi-addons image version
from v0.1.0 to v0.2.1.
Also, this updates the staging path from
{{ .KubeletDirPath }}/plugins/kubernetes.io/csi/pv/ to
{{ .KubeletDirPath }}/plugins/kubernetes.io/csi/.
This changes is done as the reclaimspace on block mode
was not working.
For more details:
https://github.com/csi-addons/kubernetes-csi-addons/pull/97
Signed-off-by: yati1998 <ypadia@redhat.com>
The rook-ceph-csi-config config map now includes CephFS subvolumegroups
when a subvolumegroup CR is created. It is useful for ceph-csi to
understand which subvolumegroup to use for a given cluster.
Signed-off-by: Sébastien Han <seb@redhat.com>
Previously, we were ignoring configuration changes coming from the
operator's pod env variables. We were assuming most users were using the
operator configmap "rook-ceph-operator-config" but most Helm users
don't. Now we will reconcile if a cephcluster is found during a CREATE
event (can be an operator restart or a cephcluster creation) AND no
"rook-ceph-operator-config" is found which means the operator's pod env
var are used.
Also, unit tests have been added (long due) for the predicate!
Closes: #9602, #9487, #9579
Signed-off-by: Sébastien Han <seb@redhat.com>
add annotations to secrets and configmaps to use
different tools to sync secrets and configmap.
Currently, these annotation is applied only on
the `rook-ceph-mon-endpoints` configmap and
the secrets `rook-ceph-mon` and `rook-ceph-admin-keyring`.
Closes: https://github.com/rook/rook/issues/8999
Signed-off-by: subhamkrai <srai@redhat.com>
Below sidecars are updated to the latest releases with this commit.
csi-provisioner : 3.1.0
csi-attacher : 3.4.0
node-driver-registrar: 2.4.0
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
The cluster-wide encryption feature now has a new supported key
management system: IBM Key protect. More information about the backend
can be found in Rook documentation under the
Today's implementation stores OSD encryption keys has "Standard" keys.
Signed-off-by: Sébastien Han <seb@redhat.com>
When the active mgr changes, there may be user-created services
that also need to have their selector updated to point to the
new active mgr. Now the operator will update any service in the
rook operator namespace that has the label app=rook-ceph-mgr
to point to the active mgr.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
the events are used to mark the start/success/failure of the reconciliation
and also to log the bucket names on which the CephbucketNotification is
created
Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Allow specifying daemon startup probes where we also allow configuring
liveness probes. Startup probes allow Rook to tolerate when Ceph daemons
occasionally take a long time to start up while not also making
Kubernetes liveness probes slower to detect runtime failures of daemons.
Startup probes are beta in Kubernetes 1.18, so we should not enable
probes by default for earlier Kubernetes versions.
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
This introduces a new CRD to add the ability to create subvolumegroup
for a given ceph filesystem volume. Typically the name of the volume is
the name of the filesystem created by rook.
Closes: https://github.com/rook/rook/issues/7036
Signed-off-by: Sébastien Han <seb@redhat.com>
This is handling a tricky scenario where the OSD deployment is manually
removed and the OSD never reconvers. This is unlikely to happen, but
still OSD should be able to run after that action. Essentially after a
manual deletion, we need to run the prepare job again to re-hydrate the
OSD information so that the OSD deployment can be deployed.
On encryption, it is a little bit tricky since ceph-volume list again
the main block won't return anything, so we need to target the encrypted
block to list.
There is another case this PR does not handle, which is the removal of
the OSD deployment and then the node is restarted. This means that the
encrypted container is not opened anymore. However, opening it requires
more work like writing the key on the filesystem (if not coming from the
Kubernete secret, eg,. KMS vault) and then run luksOpen. This is an
extreme corner case probably not worth worrying about for now.
Signed-off-by: Sébastien Han <seb@redhat.com>
Current approach is to delete all the existing notifcations from bucket
and re-add the notification from the labels.
Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
Rook does not support running multiple clusters in the same namespace,
so the operator should not reconcile if a new cluster is added.
The scenario where a cluster is added while the operator is down is also
handled. CR updates are also handled.
When the operator detects more than one cluster it will refuse to
reconcile the CephCluster, and child CRDs will block too until the
operator is ready.
The user must remove one of the clusters before can continue to perform
any reconcile.
Closes: https://github.com/rook/rook/issues/9452
Signed-off-by: Sébastien Han <seb@redhat.com>
The mons need to be configured to skip the mds sanity check
during upgrade to 16.2.7 or the mons may crash during that
upgrade.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The built-in pools device_health_metrics and .nfs created by ceph
need to be configured for replicas, failure domain, etc.
To support this, we allow the pool to be created as a CR.
Since K8s does not support underscores in the resource names
the operator must translate this special pool name into
the name expected by ceph.
This also sets the basis for allowing filesystem data
pools to specify the desired pool name instead of requiring
a generated name.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The original PR which added event reporting unnecessarily "optimized" to
prevent spamming the API controller[1].
It is sometimes important to get events as they happen and not hide new
events behind preexisting older events. For example, in integration
tests, we may often want to wait for a controller to finish processing
an update, and the best way to do that is to wait for the
"ReconcileSucceeded" event. In order for this to be useful, the events
must be reported each time.
If we begin having problems with events being reported too often, then
we should fix the underlying issue of reconciles happening too often
instead of relying on a time-based "optimization" that hides recent
event reports that may be useful.
[1]: https://github.com/rook/rook/pull/7222
Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
Sébastien Han
Radoslaw Zarzynski
Travis Nielsen
Blaine Gardner
Jiffin Tony Thottan
Eng Zer Jun
Madhu Rajanna
yati1998
Yuma Ogami
Satoru Takeuchi
subhamkrai
Humble Chirammal
Yuval Lifshitz
Denis Egorenko