Commit Graph
761 Commits
Author SHA1 Message Date
Sébastien Han b49b6393bc ceph: tune osd tcmalloc thread cache bytes
We can now tune the value of TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES by
setting an annotation on the CephCluster CRD like so:

```
annotation:
  osd:
    TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES: "134217728"
```

Alternatively, if nothing is specified Rook will read the value of the
file `/etc/sysconfig/ceph`.

Closes: https://github.com/rook/rook/issues/6134
Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-07 18:49:18 +02:00
subhamkrai 364df7793f ceph: remove suffix _sdd for tuneFastDeviceClass
this commit remove _ssd  suffix from
tuneFastDeviceClass setting as with _ssd
won't affect disk.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-10-06 20:54:52 +05:30
Travis Nielsen 3eddcf5477 Merge pull request #6283 from sp98/rook-IPv6-support
ceph: support IPv6 single-stack
2020-10-06 07:46:25 -06:00
Raghavendra TalurandTravis Nielsen 0c21abd535 ceph: prevent closing of channel more than once
Once we close the channel for the monitoring daemons, we update the
monitoring status so as to not call them again.

This is to prevent "close of closed channel" panic.

Co-authored-by: Travis Nielsen <tnielsen@redhat.com>
Signed-off-by: Nitin Goyal <nigoyal@redhat.com>
Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
2020-10-05 22:14:16 +05:30
subhamkrai c2c59ec47b ceph: tune fast device class for OSD on PVC in the Azure
on Azure, all SSDs are reporting as a rotational disk.
This commit will add `tuneFastDeviceClass` flag which will
tune for an SSD even if the cloud provider is reporting
rotational. We have a flag `tuneDeviceClass`
which works vice-versa.

Signed-off-by: subhamkrai <srai@redhat.com>
2020-10-05 20:19:55 +05:30
Sébastien Han 95ac992e48 ceph: check underlaying block status
During the initialization encrypted OSD initialization sequence, we check
for the presence of the encrypted container. If it exists we don't try
  to open it again since this will result in an error.
However, there is another case we need to handle, when the underlying
device is gone. For instance, if the pod/PV couple was drained and move
back, living the orphan dm. Once the pod comes back, the dm is still
present and perfectly matches. Unfortunately, the underlying disk is
different and thus the dm must be removed and the disk re-opened.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-10-05 15:47:29 +02:00
Santosh Pillai 70c5f70701 ceph: support IPv6 single-stack for ceph
Add --ms-bind-ipv6 true args to mom, osd, mgr, object, mds, rbd daemons.

Signed-off-by: Santosh Pillai <sapillai@redhat.com>
2020-10-02 01:03:03 +05:30
subhamkrai 0fddfcf307 ceph: handle golangci-lint linter errcheck error
this commit handle golangci-lint linter errcheck.

`errcheck` - Errcheck is a program for checking for
unchecked errors in go programs. These unchecked errors
can be critical bugs in some cases

To see only staticcheck linter output
`golangci-lint run --disable-all -E errcheck`

Signed-off-by: subhamkrai <srai@redhat.com>
2020-09-30 22:24:34 +05:30
Sébastien Han 61b2c6186d Merge pull request #6350 from leseb/fix-luks2-resize
ceph: disable keyring after opening encrypted block
2020-09-30 16:42:51 +02:00
Sébastien Han a7e78858ce ceph: disable keyring after opening encrypted block
On LUKS2 format, the resize operation requires the KEK to be passed. To
maintain a single simple init container for resize, we now open the
encrypted block with  "--disable-keyring" so that further operations
won't require the keyring.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-09-30 15:53:36 +02:00
Travis Nielsen caa51e4cb8 ceph: detect osd deployments on pvcs in a batch
OSD prepare jobs only need to be run if the OSD deployments on PVCs were
not already started. For efficiency, we now query them in a batch instead
of querying them one-by-one with a label selector, which proves to be very
slow in large clusters.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-28 12:52:17 -06:00
Travis Nielsen c73022060a ceph: detect osd pvcs in a batch
The OSD PVCs were being detected one by one with a label selector.
This proves to be slow in large clusters. Instead, we now make this
more efficient by querying for all existing PVCs and looking up their
existing in memory with a map.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-28 12:52:17 -06:00
subhamkrai de8dbbcdcc ceph: handle golangci-lint linter staticcheck error
this commit handle golangci-lint linter staticcheck error.

`staticcheck` - Staticcheck is a go vet on steroids,
applying a ton of static analysis checks.

To see only `staticcheck` linter output

`golangci-lint run --disable-all -E staticcheck`

Signed-off-by: subhamkrai <srai@redhat.com>
2020-09-24 15:04:55 +05:30
Travis Nielsen 2b9e7a5335 ceph: allow osd pvc template name set to any value
The pvc template name has either been require to be empty, or one of the
expected values such as data and metadata. Those who upgrade from 1.1
may find that if they changed it to something else, their upgrade will
fail when not one of these values. If there is only a single template,
always treat it as the data template.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-22 15:20:54 -06:00
subhamkrai 829778f251 ceph: handle golangci-lint linter ineffassign
this commit will enable one more linter ineffassign
in golangci-lint.

This linter throws an error when variable is assigned and never used.

`golangci-lint run --disable-all -E ineffassign` is used detects ineffassign
errors only.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-09-21 14:31:18 +05:30
subhamkrai 1e9bb8e6e2 ceph: handle golangci-lint linter deadcode
this commit will enable one more linter `deadcode`
in golangci-lint .

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-09-18 16:53:49 +05:30
Sébastien Han d118bd38a6 Merge pull request #6278 from subhamkrai/golanci-lint-unused
ceph: handle golangci-lint linter unused
2020-09-18 12:44:37 +02:00
Sébastien Han 204ffd4065 Merge pull request #6250 from leseb/mirroring-config
ceph: add rbd-mirror configuration
2020-09-18 09:16:08 +02:00
Travis Nielsen 524362f9e4 ceph: avoid duplicating env vars in the osd deployments
The OSD deployments have some env vars duplicated in the pod spec for
the daemon container, which may cause the detection for patch changes
to fail. Now we ensure the env vars are not duplicated in the OSD pod
spec.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-17 13:43:05 -06:00
subhamkrai f9fafe62d4 ceph: handle golangci-lint linter unused
this commit will enable one more linter
in golangci-lint.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-09-17 22:53:24 +05:30
Sébastien Han 451622a955 ceph: add rbd-mirror configuration
Rook is now capable of configuring mirroring between sites. The
implementation works at different levels:

* CephBlockPool: which introduces a new `mirroring` configuration as well
as `statusCheck`. When turned on, Rook will enable mirroring on the
pool. It will also create a bootstrap peer token and store it in a
Kubernetes Secret. The name of that Secret can be found in the Status
field of the CephBlockPool CRD. This token can be fetched and used by
other clusters to configure the site as a peer. Mirroring can be
configured either at the pool or the image level.

* CephRBDMirror: which introduces a new `peers` configuration allowing
Rook to connect to peers by passing a Secret name. The administrator will
create a Kubernetes Secret with 2 keys: 'token' for the bootstrap peer
token and 'pool' for the name of pool. Once detected the rbd-mirror
controller will go ahead and import the peer configuration.

Pool mirroring status example:

```
status:
  info:
    rbdMirrorBootstrapPeerSecretName: pool-peer-token-test
  mirroringInfo:
    lastChanged: "2020-09-17T14:47:27Z"
    lastChecked: "2020-09-17T14:48:27Z"
    summary:
      summary:
        mode: image
        peers:
        - client_name: client.rbd-mirror-peer
          direction: rx-tx
          mirror_uuid: ""
          site_name: rhcs
          uuid: c50522a4-28a4-4bd3-ba68-e11780308882
        site_name: 91eae0dd-06b1-4d2c-91f3-1311c9df382b-rook-ceph
  mirroringStatus:
    lastChecked: "2020-09-17T14:48:27Z"
    summary:
      summary:
        daemon_health: OK
        health: OK
        image_health: OK
        states:
          replaying: 1
```

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-09-17 19:05:04 +02:00
Sébastien Han a9aad94570 Merge pull request #6276 from leseb/mgr-check-version
ceph: only run version check for monitoring
2020-09-17 16:53:25 +02:00
Sébastien Han 6f894a6de8 ceph: only run version check for monitoring
We only need to read the ceph cluster version if the cluster external
has external monitoring enabled.
This avoids any issue during upgrades from 1.3 to 1.4 when monitoring is
enabled and ceph caps have not been updated.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-09-17 15:20:46 +02:00
subhamkrai 25c116a4bd ceph: handle golangci-lint linter gosimple
this commit handles all the errors  check for
golangci-lint linter gosimple.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-09-17 15:10:27 +05:30
Varsha Rao 195dee9bc2 ceph: set ROOK_CEPH_CLUSTER_CRD_NAME with cluster name
ROOK_CEPH_CLUSTER_CRD_NAME is incorrectly set to cluster namespace value

Signed-off-by: Varsha Rao <varao@redhat.com>
2020-09-16 23:53:52 +05:30
Travis Nielsen 00e6398fe8 Merge pull request #6235 from fanjunwei/user_truncated_node_name
ceph: Use truncated node name
2020-09-15 13:19:49 -06:00
Travis Nielsen da863e4156 ceph: do not change selector labels during upgrade
The selector labels used for mons cannot change during upgrade
or else the upgrade will hang when the cluster is checking
for mon status immediately after the upgrade begins.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-14 21:43:25 -06:00
fanjunwei f013add62e ceph: use truncated node name
Signed-off-by: fanjunwei <fanjunwei@t2cloud.net>
2020-09-11 19:48:38 +08:00
subhamkrai 6d08d94948 ceph: add storageClassDeviceSet label to osd pods
this commit will add a new flag to the osd pods
which will be named according to the name of the
storageClassDeviceSet. This will help in differentiate
pods immediately.

Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
2020-09-11 07:26:40 +05:30
Travis Nielsen 7a6620f2c0 ceph: remove orphaned mon pvc after failed failover
If the mon failover fails, if the mon is on a pvc, the pvc could remain
orphaned. This is likely to happen if there is not a node available
for scheduling the mon. Therefore, the operator will ensure after a failover
that any orphaned mon pvc is cleaned up.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-10 10:21:33 -06:00
Travis Nielsen 2e36913dad ceph: set the pvc name label on mon deployments
When the mons are based on pvcs, the pvc_name label was not being set
as expected. Now the label will always be set on both the canary and
the daemon pods.

The mon pvcs were also never being deleted during canary cleanup. This
code is now removed since we must be very careful when the pvcs are
removed.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-09-09 16:08:07 -06:00
Sébastien Han 152ebf618d ceph: only enable init container if dash is enabled
If the dashboard module is not enabled we don't need to run the
"dashboard-server-addr" init container.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-09-01 11:42:40 +02:00
Sébastien Han af72502fe4 ceph: do not use label selector on external svc
When the cluster is external there are no running pods so setting a
Selector on the Service Spec does not make sense, even worse, it makes
Kubernetes reconciling and overriding the configured endpoint.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-09-01 09:32:15 +02:00
Sébastien Han b68f9a7a41 ceph: fix the service monitor port name
When the cluster is external the service monitor port name should match
with the name of the external metrics one.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-09-01 09:32:15 +02:00
Raghavendra TalurandNitin Goyal 42a30ff5f8 ceph: skip check for existing PVs in forced uninstall
If AllowUninstallWithVolumes is set in the cleanup policy then we don't
check and wait for the deletion of existing volumes from the
cephCluster.

Co-authored-by: Nitin Goyal <nigoyal@redhat.com>
Signed-off-by: Raghavendra Talur <raghavendra.talur@gmail.com>
2020-08-31 14:07:32 -04:00
Travis Nielsen 24b0bdd119 Merge pull request #6156 from cloudical-io/first_part_6111
ceph: operator should only "watch" OSD placement nodes
2020-08-26 07:20:30 -06:00
Alexander Trost a87c00ddee ceph: allow user to add labels to pods
This allows users to specify additional labels to be added to the components
created by the Rook Ceph operator.
Additionally the rbd mirrors are now getting their annotations and
labels set.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-08-25 19:38:51 +02:00
Alexander Trost 94d5196439 ceph: operator should only "watch" OSD placement nodes
This change is the first part to improve the watch OSD nodes logic in
the Rook Ceph operator.
This commit changes the "is node valid check" to use the OSD placement
instead of the all placement. The OSD placement is "merged" with the
all placement already.

Second step will be to skip the first "iteration" of the "nodes added"
events and then start the actual watch logic.

Partially resolves #6111.

Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2020-08-25 18:16:20 +02:00
Sébastien Han ee893eda5e Merge pull request #6126 from leseb/encryption-expansion-pvc
ceph: add expansion support for encrypted osd on pvc
2020-08-19 21:40:46 +02:00
Travis Nielsen ce92249725 ceph: continue with memory limits below min settings
The operator will now allow the resource limits to be applied below the
recommended minimums. In small clusters, even the recommended minimums
may not be necessary. A warning is still printed to the operator log,
but we allow the configuration to continue.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-08-19 11:07:16 -06:00
Sébastien Han 321e8e0b5a ceph: add more pvc dep unit tests
Added more unit tests scenario for osd on pvc such as:

* block/db/wal
* block encryption
* block/db encryption
* block/db/wal encryption

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-19 18:45:31 +02:00
Sébastien Han 803462ceca ceph: add encrypted block status container
This init container displays the status of the encrypted pvc. This can
be really useful for debugging purpose.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-19 17:13:46 +02:00
Sébastien Han cb81ee7d1d ceph: add pvc expansion support for encrypted osds
We now support the expansion of encrypted osds. We added a new init
container to run "cryptsetup resize".

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-19 17:13:46 +02:00
Sébastien Han 3f42db5ce0 ceph: support osd deployment update on encrypted pvc
If the deployment spec was updated or restarted, the "encryption-open"
init container would fail to open the device since it was already
opened on the node.
Because they are no reliable post stop hook on Kubernetes, the trick is
to use a small bash init container.

Signed-off-by: Sébastien Han <seb@redhat.com>
2020-08-19 17:13:25 +02:00
Sébastien Han d5b1e89ce2 Merge pull request #6118 from travisn/osd-prepare-resources
ceph: Set the OSD prepare resources
2020-08-19 15:38:06 +02:00
Travis Nielsen 18d39d909e ceph: set the osd prepare resources
The osd prepare resources were only being set on the main container
while the init containers either did not have resources set, or were
inheriting the resource settings from the device set resources for osds.
Now the osd prepare resources will be consistently set based on the
prepareosd resources in the main section of the cluster CR.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-08-18 22:58:26 -06:00
Travis Nielsen 26f59c0382 ceph: remove redundant check for encrypted osds
This is simply a code simplification to remove an extra check for
whether the OSDs are encrypted. There is no change in functionality.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-08-18 22:50:32 -06:00
Travis Nielsen 0ffa0348b5 ceph: separate placement for osd prepare jobs from osd daemons
The osd prepare jobs and osd daemons may have different placement
requirements. In particular, the topology spread constraints
or pod anti-affinity should only work on either the prepare job or
osd daemon, but does not work effectively on both types at the same
time. Therefore, we split out the osd prepare placement to be separate
from the osd daemon placement. If the osd prepare placement is not set
the operator will fall back to the osd daemon placement for
backward compatibility.

Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
2020-08-18 15:24:04 -06:00
Travis Nielsen 98db380237 Merge pull request #6062 from leseb/wal-encryption
ceph: add support for wal encrypted device on pvc
2020-08-17 14:07:40 -06:00
Sébastien Han 1390335f98 Merge pull request #6063 from leseb/fix-crash-collector
ceph: check for gone node on crash collector
2020-08-13 17:02:30 +02:00