Commit Graph
566 Commits
Author SHA1 Message Date
Travis Nielsen 7a6a62f43b Merge pull request #9386 from leseb/mgr-active-side-car-permissions
mgr: run the watch side car config on /etc/ceph
2021-12-15 12:20:28 -07:00
Sébastien Han 5c1e459a6a mgr: run active-watch as root and privileged
For now, we must run the container with UID 0 and privileged for
multiple reasons:

* the rook binary writes ceph config to /var/lib/rook which is owned by
  root
* it's difficult to use /etc/ceph since it will conflict with the
  rook-ceph-override configmap AND is also owned by root since it's a
  mounted configmap.
* using /etc/ceph might be possible but has other issues with rook's
  exec package since the ceph config is built from /var/lib/rook

Closes: https://github.com/rook/rook/issues/9385
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-12-13 14:02:33 +01:00
Sébastien Han 200fb569f4 Merge pull request #9395 from y1r/add-context-k8sutil-kvstore
core: add context parameter to k8sutil kvstore
2021-12-13 11:44:58 +01:00
Yuichiro Ueno 5352cf58a7 core: add context parameter to k8sutil pvc
This commit adds context parameter to k8sutil pvc functions. By this, we
can handle cancellation during API call of pvc resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-12-11 11:32:46 +09:00
Yuichiro Ueno 0d8c581aa9 core: add context parameter to k8sutil kvstore
This commit adds context parameter to k8sutil kvstore functions. By
this, we can handle cancellation during API call of kvstore resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-12-11 11:17:12 +09:00
Travis Nielsen c953032d09 Merge pull request #8678 from parth-gr/Recommended-labels
core: create rook resources with k8s recommended labels
2021-12-07 11:53:33 -07:00
parth-gr 0a86d26b2e core: create rook resources with k8s recommended labels
Adding Recommended Labels on the resources created by rook
    and using Recommended Labels in the helm chart,
    for better visuals and management of k8s object

Closes: https://github.com/rook/rook/issues/8400
Signed-off-by: parth-gr <paarora@redhat.com>
2021-12-07 18:16:44 +05:30
Travis Nielsen ee83ca74af osd: truncate osd prepare job names further
In K8s 1.22 there is a bug in the job name generation that
the job name is truncated an additional 10 characters. This can cause an issue
in the generated pod name if it then ends in a non-alphanumeric character. In that case,
we more aggressively generate a hashed job name.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-12-06 08:02:04 -07:00
Travis Nielsen 32a884ac18 osd: honor skipUpgradeChecks for osds
Skipping upgrade checks was not being honored for OSDs.
Now the flag will be checked and allow the OSDs to be upgraded
without checking for the ok-to-stop condition.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-12-01 14:41:53 -07:00
parth-gr cbe505d122 osd: update existing OSDs with deviceClass
If we apply useAllNodes to false for the current deployment,
the OSDs should get updated with the individual nodes values and config,
The deviceClass was not updating to the existing OSDs because there was
bug in the check.
The check osdInfo.DeviceClass == "" which should be
checked like this osdInfo.DeviceClass == "None"

Updated the code so OSDs can make use of the devices present

Signed-off-by: parth-gr <paarora@redhat.com>
2021-11-29 21:40:02 +05:30
Travis Nielsen 1afd322650 core: ensure cluster name is available on cluster info
The cluster info is important context for the cluster controller to
create the cluster, and all the fields must be properly set.
A test cluster name was being set temporarily, resulting in
mons incorrectly getting the wrong cluster CR name. There is no
known issue from the temporary value, it was just exposed by
https://github.com/rook/rook/pull/8678 setting the value to a label.

Now the functions are more clearly named so only unit and
integration tests should be using the test value for the cluster
name where it is not important.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-11-18 15:30:26 -07:00
Blaine Gardner cd832e5832 Merge pull request #8579 from thotz/vaultsslsupport
ceph: add support in RGW to communicate vault with TLS
2021-11-18 12:32:38 -07:00
Omar Pakker 8f9055809f osd: add privileged support (back) to blkdevmapper securityContext (work-around)
The blockdevmapper securityContext was changed to request a minimal set of
required capabilities for its operation and drop running as privileged.
While the base change works and is valid in terms of the container's copy operation,
it turns out that OpenShift may require some additional configuration not
currently covered by the limited securityContext and the capabilities granted.

To not break those OpenShift deployments, make the blkdevmapper securityContext
listen to the ROOK_HOSTPATH_REQUIRES_PRIVILEGED flag again to set privileged mode.
This flag is true on OpenShift deployments and running as privileged
works around the (missing) configuration problem for now.
To properly drop privileged completely some additional investigation needs
to be done on OpenShift deployments without relying on privileged execution.

Signed-off-by: Omar Pakker <Omar007@users.noreply.github.com>
2021-11-17 12:25:13 +01:00
Jiffin Tony Thottan aba50d3ca9 object: add support in RGW to communicate vault with TLS
From ceph v16.2.6 onwards the vault TLS suppport in RGW was added,
include similar changes for RGW.

Signed-off-by: Jiffin Tony Thottan <thottanjiffin@gmail.com>
2021-11-17 10:19:28 +05:30
Sébastien Han c5783a77cf Merge pull request #9163 from y1r/add-context-k8sutil-node
core: add context parameter to k8sutil node
2021-11-15 16:25:13 +01:00
Travis Nielsen 716baf7785 Merge pull request #9158 from Omar007/fix/blkdevmapper-capabilities
osd: set blkdevmapper capabilities
2021-11-15 08:05:12 -07:00
Yuichiro Ueno 4cc716a7ca core: add context parameter to k8sutil node
This commit adds context parameter to k8sutil node functions. By this,
we can handle cancellation during API call of node resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-15 22:45:59 +09:00
Yuichiro Ueno 3799542356 core: add context parameter to k8sutil job
This commit adds context parameter to k8sutil job functions. By this, we
can handle cancellation during API call of job resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-15 22:39:08 +09:00
Omar Pakker 4726d39688 osd: set blkdevmapper capabilities
The OSD blkdevmapper init container relies on the MKNOD capability,
which it does not actually request.
As a result, deployments fail on Kubernetes clusters that do not
happen to assign this capability to all containers by default.
Solve this by updating the container spec securityContext to
explicitly request the capability it relies on.

Closes: https://github.com/rook/rook/issues/9156
Signed-off-by: Omar Pakker <Omar007@users.noreply.github.com>
2021-11-15 13:11:19 +01:00
Yuichiro Ueno 0b575703c7 core: add context parameter to k8sutil deployment
This commit adds context parameter to k8sutil deployment functions. By
this, we can handle cancellation during API call of deployment resource.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-11-13 14:58:13 +09:00
Travis Nielsen 427996a7c0 osd: increase wait timeout for osd prepare cleanup
When a reconcile is started for OSDs, the prepare jobs are first
deleted from a previous reconcile. The timeout for the osd prepare
job deletion was only 40s. After that timeout, the reconcile attempts
to continue waiting for the pod, but of course will never complete
since the OSD prepare was not running in the first place, causing the
reconcile to wait indefinitely. In the reported issue, the osd prepare
jobs were actually deleted successfully, the timeout just wasn't long
enough. Pods need at least a minute to be forcefully deleted,
so we increase the timeout to 90s to give it some extra buffer.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-11-05 14:20:43 -06:00
Sébastien Han 3b5bbf28d5 Merge pull request #9003 from leseb/vault-k8s-auth
osd: add support for k8s with vault kms
2021-10-25 18:02:48 +02:00
Blaine Gardner 2f850b6ae6 Merge pull request #8613 from subhamkrai/remove-nautilus
ceph: remove ceph nautilus, ceph octopus to default
2021-10-25 09:09:18 -06:00
Yuichiro Ueno 3fd86f83ae core: add context parameter to opcontroller
This commit adds context parameter to utilities in opcontroller to
remove context.TODO use in opcontroller. By this, we can handle
cancellation of reconcilers in a fine-grained way.

Signed-off-by: Yuichiro Ueno <y1r.ueno@gmail.com>
2021-10-25 20:45:06 +09:00
Sébastien Han 18a4047679 osd: add support for k8s with vault kms
Rook cluster-wide encryption can now use the native Kubernetes
authentication to interact with vault KMS instead of using the token
method.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-21 13:59:28 +02:00
subhamkrai 0150966024 ceph: remove ceph nautilus, ceph octopus to default
since rook 1.8, ceph nautilus no longer supported,
ceph octopus will be the minimum ceph version.

Closes: https://github.com/rook/rook/issues/7908
Signed-off-by: subhamkrai <srai@redhat.com>
2021-10-20 14:45:12 +05:30
Sébastien Han 4017f94464 osd: use rook binary to fetch key encryption key
When deploying a cluster-wde encrypted cluster we now use the rook
binary to execute some code to fetch the key encryption key.

Using cURL all the time to fetch the key has its limitations. The
incoming integration with Kubernetes Authentication through service
accounts is leading the usage of cURL to its end.
The logic is really complex and prone to errors. Re-implementing the
Go logic into Bash is not a viable option.

Also, using the lib in a binary allows us to keep a consistent behavior
throughout the life cycle of our code.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-10-19 17:29:25 +02:00
Sébastien Han 121c2987e3 ceph: stop using tini
We don't need to use tini.
We don't have anything in the rook operator that would
either create zombie processes (no threads) or use
exec (to fork). The Go binary has a really good
signal handling mechanism.

Closes: https://github.com/rook/rook/issues/8794
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-27 10:54:16 +02:00
Travis Nielsen 0a0b9c98bd build: remove obsolete flex driver
The flex driver has been fully deprecated and thus removed from Rook.
Before upgrading to v1.8, users will need to convert existing flex volumes
from flex to csi volumes.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-09-23 16:17:20 -06:00
Sébastien Han b89730d895 ceph: refactor operator initialization sequence
This commit is a large refactor on how the operator starts, stops and
how it starts various sub-components such as the ceph-csi driver. It
also refines the way we cancel orchestrations. We don't use breakpoints
anymore but send our self a SIGUP to reload our controller runtime
manager.
The reload will happen under different circonstances like:

* a new adminission controller secret is created/deleted/changed
* a CephCluster CR is edited

As mentioned earlier, the csi driver now has its own controller, just
like flex. It reacts to change in the operator config map for particular
ROOK_CSI_ fields.

A second new controller for the operator's general config has been
created, it manages:

* the logging level
* the ceph CLI command timeout
* the discovery daemon

The operator reacts much more rapidly to cancellation events by stopping
the manager's context and reloading it.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-09-17 16:57:12 +02:00
Hiroya Onoe 2ff5413b75 ceph: fix error message in UpdateNodeStatus
The error message in UpdateNodeStatus regards the second argument
as node name. However, it is a PVC name in OSD on PVC.

Signed-off-by: Hiroya Onoe <onoehiroya@gmail.com>
2021-09-02 02:43:16 +00:00
Travis Nielsen 42717d2fca Merge pull request #8566 from subhamkrai/merge-tolerations
ceph: merge toleration for osd/prepareOSD pod if specified both places
2021-08-26 08:55:27 -06:00
subhamkrai e1f232eeb7 ceph: merge toleration for osd/prepareOSD pod if specified both places
earlier, `ApplyToPodSpec()` was only taking one toleration and ignoring
tolerations from `placement.ALL()`.

this commit merge toleration for Mgr,Mon,Osd pod
example, for osd it will merge
spec.placement.all and
storageDeviceClassSets.Placement(in case of pvc) or
spec.placement.osd(in case of non-pvc's).

Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-25 19:34:28 +05:30
parth-gr 77ba1ef11b core: convert util.NewSet() to sets.NewString()
Converting util.NewSet() instance to use sets.NewString() instance

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-25 13:13:05 +05:30
parth-gr 1601ba1c1c core: convert util.NewSet() to sets.NewString()
Converting util.NewSet() instance to use sets.NewString() instance

Closes: https://github.com/rook/rook/issues/8479
Signed-off-by: parth-gr <paarora@redhat.com>
2021-08-20 18:56:25 +05:30
subhamkrai 9838313398 ceph: add an option to enable/disable merge all placement
earlier, we had the issue for osd placement regarding
whether we should merge the placements or not.

Now, we have option `skipApplyAllPlacement` to
enable/disable osd placement. By default it is false
which means it will merge both placement.All() and
storageClassDeviceSets.Placement, when true
it will only apply storageClassDeviceSets.Placement.

Also, adding unit test.

Closes: https://github.com/rook/rook/issues/8135
Signed-off-by: subhamkrai <srai@redhat.com>
2021-08-03 19:47:40 +05:30
Sébastien Han d55fb8d13b Merge pull request #8351 from leseb/refact-exec-helpers
ceph: remove unnecessary exec helpers
2021-07-23 18:12:57 +02:00
Sébastien Han 6d77a9976c ceph: remove unnecessary exec helpers
Both `ExecuteCommandWithOutputFileTimeout()` and
`ExecuteCommandWithOutputFile()` generate unnecessary system calls by
creating/reading/removing files where the stream output of the command
can simply be used. So sticking with `ExecuteCommandWithOutput()` and
`ExecuteCommandWithCombinedOutput()` for reading outputs is sufficient.

Closes: https://github.com/rook/rook/issues/8343
Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-23 09:16:33 +02:00
Sébastien Han 6bce1ff3e9 ceph: move all of our docker.io reference to quay.io
Recently, the builds of `ceph/ceph` image moved to quay.io, see
https://github.com/ceph/ceph-build/pull/1883 for more details.
Current images will remain but new builds will happen on quay.io only.

This means that tags such as `v14.2`, `v15.2`,`v16.2` will need to
switch to quay.io to get updates.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-07-22 11:17:05 +02:00
subhamkrai d254c935e8 ceph: create new keyring for osd
when osd pods removed, they are not able
to add back due to missing `ceph auth`
or different `ceph auth`.

this commit mounts the secret `rook-ceph-admin-keyring`
inside the osd activate initcontainer which has
admin keyring.

Signed-off-by: subhamkrai <srai@redhat.com>
2021-07-19 19:26:06 +05:30
Satoru Takeuchi 763b076c75 ceph: mount dev directory in lvm mode osd on osd on lv-backed pvc
"/dev/" is only mounted on OSD on device. However, this directory
should also be mounted on OSD on LV-backed PVC.

Closes: https://github.com/rook/rook/issues/8264

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2021-07-13 12:21:41 +00:00
Blaine Gardner c22f545ebf ceph: block delete object store when buckets exist
Block deletion of CephObjectStore resources when buckets exist in the
object store.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-29 14:31:39 -06:00
Sébastien Han bcd2511a72 ceph: do not leak key encryption key
When evaluating the potential "warning" key from the json output we were
not hiding stdout (1) from the output, so the key was leaked. Also the
warning check for the key was not perfect, it only checked for the
command to succeed, but we should introspect it since the "warning" key
is always present unlike the "error" one.
We now only print the warning or the error is present.

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-22 15:04:13 +02:00
Sébastien Han e6929ce308 Merge pull request #7913 from abursavich/extract-kk
Remove dependency on k8s.io/kubernetes
2021-06-22 09:22:41 +02:00
Andy Bursavich 11f0add706 build: update golangci-lint version and nolint comment format
Signed-off-by: Andy Bursavich <abursavich@gmail.com>
2021-06-21 08:50:04 -07:00
Sébastien Han eb0c986196 ceph: use cacert if no client cert/key are present
We have seen issues with fullchain.pem files containing one self-signed
certificate. At the same time, the failing configuration was not
providing any client certificates or keys.

The error from curl was `"self signed certificate in the chain"`.

The openssl binary is not available so we cannot even c_rehash the
certificate directory when using `--capath`, however this worked already
with a fully signed fullchain.pem along with a client certificate and
key. So we cannot only use `--cacert` which will break that use case.

This is becoming really obscur on why using `--cacert` solves this
problem. I just can't wait to use the `vault` binary, that's all I can
say...

Signed-off-by: Sébastien Han <seb@redhat.com>
2021-06-21 16:06:25 +02:00
Travis Nielsen dd3b4738bd ceph: activate osd container should use correct dataDirHostPath
The active osd container was always using /var/lib/rook for the
configuration of the OSD. If the dataDirHostPath is anything other
than the default, the config would show up in the wrong place.
Changing this back to the dataDirHostPath does not appear to affect
any upgrade scenarios as the activate container will simply start
extracting the config to the correct place.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2021-06-17 06:59:31 -06:00
Denis Egorenko 6dbb902674 ceph: set device class for already present osd deployments
Add ability to set device class for existing osd deployments,
based on its crush map device class.

Related-Issue: rook#8028
Signed-off-by: Denis Egorenko <degorenko@mirantis.com>
2021-06-15 13:31:37 +04:00
Blaine Gardner 21e290e003 ceph: implement dependencies for CephCluster
Implement the first step of `design/ceph/resource-dependencies.md` to
add dependency checking when deleting a CephCluster.

Signed-off-by: Blaine Gardner <blaine.gardner@redhat.com>
2021-06-10 10:06:10 -06:00
Travis Nielsen 06fbf29b3c Merge pull request #8028 from degorenko/device-classes-resource-limits
ceph: add ability to set resource limit for OSDs based on device classes
2021-06-08 09:01:14 -06:00