Commit Graph
1165 Commits
Author SHA1 Message Date
Madhu Rajanna b0fc7c9b92 namespace: add new CRD
This introduces a new CRD to add the ability
to create rados namespace for a given
ceph block pool. Typically the name of the pool
is the name of the blockpool created by rook.

Closes: #7035

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-05 10:10:04 +05:30
subhamkrai 24802c559e core: fix golangci linter
fix golangci linter

Signed-off-by: subhamkrai <srai@redhat.com>
2022-04-04 20:59:31 +05:30
Sébastien Han 05506e7a68 core: reload go routine after CR is edited
Previously, the struct maintaining the list of cluster was still
initialized with a cluster item. Then the monitoring check will see that
the cluster is part of the struct already and thus won't run the
monitoring go routine again.
Now each time we cancel the context, we also remove the cluster item
from the map so that when the controller runs again, the monitoring
struct is re-populated and the go routine runs and statuses are updated.

Closes: https://github.com/rook/rook/issues/9911
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-03-31 08:49:42 +02:00
Travis Nielsen 6c50530737 Merge pull request #9685 from weirdwiz/context-remove
Remove context.TODO and add context parameter
2022-03-30 12:20:30 -05:00
Divyansh Kamboj 9008409f87 core: add context parameter to functions
This commit adds context parameter to various functions, and remove the
usage of context.TODO.

Closes: https://github.com/rook/rook/issues/8701
Signed-off-by: Divyansh Kamboj <dkamboj@redhat.com>
2022-03-22 08:19:07 +05:30
Travis Nielsen 4edcff04f7 monitoring: create prometheus rules with helm chart
The prometheus rules had been previously created if the cephcluster CR
setting monitoring.enabled was set to true. The rules were not customizable
and therefore not flexible enough. Now the rules are installed by the helm
chart. To customize the rules, a post-processor can be applied to the helm
chart.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-21 14:47:52 -06:00
Travis Nielsen 86eebc7486 mon: set the msgr1 port by default for csi driver
Older kernels still fail to connect for rbd/cephfs with
the msgr2 port due to a difference in http protocol expected.
For now, we keep the default port as msgr1 for the csi driver
and other client connections, unless msgr2 is strictly required
for encryption or compression.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-17 16:34:13 -06:00
Sébastien Han 18b19de639 Merge pull request #9687 from parth-gr/observedGeneration
core: add observedGeneration to CR status
2022-03-17 10:36:11 +01:00
Travis Nielsen 48be000b4f core: add quincy to the supported versions
Ceph Quincy is now supported as of Rook v1.9.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-16 14:26:20 -06:00
parth-gr 2dfd64a97c core: add observedGeneration to CR status
adding observedGeneration field in the cephcluster cr
status for having better control on reconciling,
as observedGeneration field will be updated by the controller

Closes: https://github.com/rook/rook/issues/9673

Signed-off-by: parth-gr <paarora@redhat.com>
2022-03-16 19:58:35 +05:30
subhamkrai 6a9ff434d2 core: add k8s events in controller reconciler
adding k8s event in controller reconciler when
1. when reconciler starts
2. when deletion reconcile triggered.

Closes: https://github.com/rook/rook/issues/9462
Signed-off-by: subhamkrai <srai@redhat.com>
2022-03-14 18:55:26 +05:30
Blaine Gardner afd38a57c5 Merge pull request #9873 from BlaineEXE/report-reconcile-rework
core: rework usage of ReportReconcileResult
2022-03-11 10:02:30 -07:00
Travis Nielsen c580e8f09c Merge pull request #9118 from travisn/network-encryption
mon: Options for enabling msgr2 encryption and compression
2022-03-11 09:49:55 -07:00
Blaine Gardner c92c6fbc36 core: rework usage of ReportReconcileResult
ReportReconcileResult should never be given an object that is nil. It is
impossible to force compile-time checking for this because the object's
type is an interface. We can't force this, but we can rework the
function definition to handle more cases where the object may not be
returned fully-complete, and we can rework callers to return structs
(not pointers-to-structs) so it is less likely a caller will pass nil
to the function.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2022-03-10 16:53:25 -07:00
Travis Nielsen 22dea0a1ef mon: disable startup probe on canary pods
The canary pods are very short-lived pods and are not
running ceph in any form. The startup probe was mistakenly
enabled for the canary pod, which could cause the pod
to fail unexpectedly if the startup probe was triggered
after the delay timeout.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 12:19:55 -07:00
Travis Nielsen 03c7164b15 mon: options for enabling msgr2 encryption and compression
msgr2 allows for encryption and/or compression across the wire.
Settings for enabling the encryption and compression are now
available in the cluster CR to that ceph will be automatically
configured with these settings when desired.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 11:50:42 -07:00
Travis Nielsen 9f6804515c mon: enable msgr2 by default for new mons
New mons all support msgr2, so there is no need to keep the default
msgr1 port for new mons.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 11:48:47 -07:00
Travis Nielsen 2b2231eb0c mon: no need to enable msgr2 since nautilus
Since Nautilus, mgr2 is already enabled by default. No longer
does the operator need to enable msgr2 after the mons are
configured or upgraded.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-03-10 11:48:47 -07:00
parth-gr 2e89ff0fd9 core: sync ceph fsid on cephcluster status
FSID will be updated with cephcluster CR status
when monitoring check get start

Signed-off-by: parth-gr <paarora@redhat.com>
2022-03-09 21:57:37 +05:30
Travis Nielsen b3ca0f1b87 osd: purge job will remove all pvcs for the osd
The purge job is intended to clean up all the resources
related to an OSD that is to be removed. An OSD with
a metadata, wal, and data PVC was only cleaning up the
data PVC. Now the metadata and wal PVCs will also be
cleaned up.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-02-24 12:05:15 -07:00
parth-gr 413662d475 core: change livenessProbe file name to probes
Added unit test for probes changes and
did re-naming of liveness probes file
as it will contain multiple types of probes

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-23 17:20:07 +05:30
parth-gr 1679af4a02 mds: override mds startup liveness probes from CephFilesystem CR
Rook code allows overriding the probes for mds from the CephCluster,
but this should really come from the CephFilesystem CR.

Closes: https://github.com/rook/rook/issues/9519

Signed-off-by: parth-gr <paarora@redhat.com>
2022-02-22 20:36:40 +05:30
Sébastien Han 0143a8f8ee core: remove unnecessary secret validation
We don't need to validate the name of Ceph-CSI secret, simply because
the names can be different. The default example have default names but
since the secret can be created by the administrator we should not
assume anything. This check will fail if the secret names are different
for whatever reason. This is especially true for external mode. In
converged mode Rook creates the secrets on its own if needed to ease the
integration with the default StorageClass examples.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-17 15:33:54 +01:00
Sébastien Han 47b8bb989e core: move clusterInfo discovery in converged cluster
The external code has its own mechanism to look up the clusterInfo so
let's move this to the non-external condition to clarify the code.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-17 15:32:37 +01:00
Sébastien Han fdcad15b87 core: check for canceled context in external mode
This is fixing a potential race where the context is canceled and the
external cluster is trying to fetch the cluster info. If the context is
canceled then the loop will be stuck forever. Now we check for the
canceled context and return appropriately.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-17 15:30:25 +01:00
Travis Nielsen 4570204a1d osd: write log message when node is being skipped
When a node is being skipped for creation of OSDs, log a message
that indicates whether it was from the node being unschedulable,
the node is not ready, or the node does not meet the placement
criteria specified in the cluster CR.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-02-14 12:06:49 -07:00
Sébastien Han 32998cbda4 core: do not dereference a non-pointer
In this case `oldMappingToGo` and `newMappingToGo` were not pointers so
there is nothing to dereference.
This happens when the monitors endpoints configmap is updated, but only
on the pool and file controllers. They are watching for changes on that
configmap so that they can reconcile and upate the bootstrap peers token
when replication is enabled.

Closes: https://github.com/rook/rook/issues/9714
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-02-10 17:11:38 +01:00
Satoru Takeuchi 2d8f9b67c6 core: eliminate unnecessary code
We can eliminate the OSD creation code for Ceph < v15.2.0 because Nautilus(v14.2.z)
is no longer supported. In other words, we can now expect that all newly-created
OSDs on PVC are raw mode.

Please note that the existing lvm mode OSDs on PVCs still be supported after this change.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-02-04 13:59:01 +00:00
Sébastien Han 5404ec13a2 core: dereference pointer before trying to compare with deepequal
Prior to this, we were comparing a pointer (the memory address) with a
struct. This was obviously always failing and returned false. We must
dereference the pointer to access the data contained at that memory
location.

Closes: https://github.com/rook/rook/issues/9544
Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-27 17:41:42 +01:00
Radoslaw Zarzynski 17ca41f557 osd: allow for injecting extra env. variables via ConfigMap
This patch brings a mechanism to define arbitrary environment
variables in OSD containers (both prepare and main ones).
It bases on idea proposed by Sebastien Han to use an optional
`ConfigMap` instance named `aook-ceph-daemon-env-override`
as the source of these settings.

The need for the patch comes from the fact that, although
crimson finally exposes the same CLI interface as the classical
OSD, the broadly used development builds have ASan built in.
As ASan, by default, complains if it isn't the very first loaded
DSO, we need a way to set the `ASAN_OPTIONS` environment variable
to `verify_asan_link_order=0` to mitigate the early aborts.

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2022-01-26 15:17:27 +00:00
Eng Zer Jun ed63571530 test: use T.TempDir to create temporary test directory
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-01-24 23:52:54 +08:00
Sébastien Han 739abf6662 csi: change rook-ceph-csi-config to expose clusterID for subvolume
The rook-ceph-csi-config config map now includes CephFS subvolumegroups
when a subvolumegroup CR is created. It is useful for ceph-csi to
understand which subvolumegroup to use for a given cluster.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-21 10:12:56 +01:00
Blaine Gardner d0d83edc5f Merge pull request #9502 from cybozu-go/core-support-priority-class-for-crashcollector
core: support priority class for crashcollector
2022-01-19 10:38:29 -07:00
Satoru Takeuchi 5465481e0b core: support priority class for crashcollector
Support priorityClass to crashcollectors as mons, mgrs, and osds.

https://rook.io/docs/rook/v1.8/ceph-cluster-crd.html#priority-class-names-configuration-settings

The main use case is applying the high priority to crashcollectors to preempt normal pods
under heavy load. Without this feature, we might lose crash information.

Closes: https://github.com/rook/rook/issues/9500

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2022-01-19 13:54:03 +00:00
subhamkrai 02d43e82e0 mon: add annotation to mon secret and cm
add annotations to secrets and configmaps to use
different tools to sync secrets and configmap.
Currently, these annotation is applied only on
the `rook-ceph-mon-endpoints` configmap and
the secrets `rook-ceph-mon` and `rook-ceph-admin-keyring`.

Closes: https://github.com/rook/rook/issues/8999
Signed-off-by: subhamkrai <srai@redhat.com>
2022-01-19 10:15:17 +05:30
subhamkrai d87be97a50 osd: correct bluestore compression min blob size for ssd
Update bluestore compression min blob size for ssd to 8192.
It was typo which was added long time back.

Signed-off-by: subhamkrai <srai@redhat.com>
2022-01-17 15:13:15 +05:30
Travis Nielsen f3246228d8 Merge pull request #9467 from travisn/update-mgr-service
mgr: Update services when active mgr changes
2022-01-13 12:21:03 -07:00
Sébastien Han a7acf490b6 osd: add ibm key protect kms
The cluster-wide encryption feature now has a new supported key
management system: IBM Key protect. More information about the backend
can be found in Rook documentation under the

Today's implementation stores OSD encryption keys has "Standard" keys.

Signed-off-by: Sébastien Han <seb@redhat.com>
2022-01-13 16:02:10 +01:00
Travis Nielsen 11127f54ca mgr: update services when active mgr changes
When the active mgr changes, there may be user-created services
that also need to have their selector updated to point to the
new active mgr. Now the operator will update any service in the
rook operator namespace that has the label app=rook-ceph-mgr
to point to the active mgr.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2022-01-13 07:25:24 -07:00
Blaine Gardner 5b9ac16670 Merge pull request #9468 from BlaineEXE/startup-probes
core: rgw: allow specifying daemon startup probes
2022-01-04 11:03:12 -07:00
Blaine Gardner c07d89d9ea core: rgw: allow specifying daemon startup probes
Allow specifying daemon startup probes where we also allow configuring
liveness probes. Startup probes allow Rook to tolerate when Ceph daemons
occasionally take a long time to start up while not also making
Kubernetes liveness probes slower to detect runtime failures of daemons.

Startup probes are beta in Kubernetes 1.18, so we should not enable
probes by default for earlier Kubernetes versions.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-12-21 15:12:41 -07:00
Blaine Gardner bb0d0d39f6 Merge pull request #9384 from leseb/fix-7036
subvolumegroup: add new crd
2021-12-21 10:24:23 -07:00
Sébastien Han 6e9eb33782 subvolumegroup: add new crd
This introduces a new CRD to add the ability to create subvolumegroup
for a given ceph filesystem volume. Typically the name of the volume is
the name of the filesystem created by rook.

Closes: https://github.com/rook/rook/issues/7036
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-21 16:28:38 +01:00
Sébastien Han 05775b068d osd: handle removal of encrypted osd deployment
This is handling a tricky scenario where the OSD deployment is manually
removed and the OSD never reconvers. This is unlikely to happen, but
still OSD should be able to run after that action. Essentially after a
manual deletion, we need to run the prepare job again to re-hydrate the
OSD information so that the OSD deployment can be deployed.
On encryption, it is a little bit tricky since ceph-volume list again
the main block won't return anything, so we need to target the encrypted
block to list.
There is another case this PR does not handle, which is the removal of
the OSD deployment and then the node is restarted. This means that the
encrypted container is not opened anymore. However, opening it requires
more work like writing the key on the filesystem (if not coming from the
Kubernete secret, eg,. KMS vault) and then run luksOpen. This is an
extreme corner case probably not worth worrying about for now.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-21 10:51:09 +01:00
Sébastien Han fffb862956 Merge pull request #9457 from leseb/fix-9452
core: disallow multiple clusters in the same namespace
2021-12-20 17:18:10 +01:00
Blaine Gardner 9dc41f5796 Merge pull request #9427 from BlaineEXE/always-report-reconcile-events
operator: always report events
2021-12-20 08:42:08 -07:00
Sébastien Han da76a772f8 core: disallow multiple clusters in the same namespace
Rook does not support running multiple clusters in the same namespace,
so the operator should not reconcile if a new cluster is added.
The scenario where a cluster is added while the operator is down is also
handled. CR updates are also handled.
When the operator detects more than one cluster it will refuse to
reconcile the CephCluster, and child CRDs will block too until the
operator is ready.
The user must remove one of the clusters before can continue to perform
any reconcile.

Closes: https://github.com/rook/rook/issues/9452
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-17 10:56:15 +01:00
Travis Nielsen 0c19f62df2 mds: skip sanity check during upgrades to 16.2.7
The mons need to be configured to skip the mds sanity check
during upgrade to 16.2.7 or the mons may crash during that
upgrade.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-12-16 08:07:02 -07:00
Blaine Gardner da61ac1ae8 operator: always report events
The original PR which added event reporting unnecessarily "optimized" to
prevent spamming the API controller[1].

It is sometimes important to get events as they happen and not hide new
events behind preexisting older events. For example, in integration
tests, we may often want to wait for a controller to finish processing
an update, and the best way to do that is to wait for the
"ReconcileSucceeded" event. In order for this to be useful, the events
must be reported each time.

If we begin having problems with events being reported too often, then
we should fix the underlying issue of reconciles happening too often
instead of relying on a time-based "optimization" that hides recent
event reports that may be useful.

[1]: https://github.com/rook/rook/pull/7222

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-12-15 15:24:17 -07:00
Travis Nielsen 7a6a62f43b Merge pull request #9386 from leseb/mgr-active-side-car-permissions
mgr: run the watch side car config on /etc/ceph
2021-12-15 12:20:28 -07:00