feedc0de/my-rook-config
1
0
Fork 0
forked from rook/rook
Code Pull Requests Activity
Files
9120791e24c2f911dc5935f2a78fdcbe7f8ea014
my-rook-config/PendingReleaseNotes.md
T
Sébastien Han 900abbc967 ceph: run ceph processes with the 'ceph' user 
Ceph daemons will run as the 'ceph' user instead of the 'root' user.
Existing data store (osd and mon) owned by root will continue to run as
'root'.
ALl CLI calls also inherate from this and will use them when fetching a key
for instance. If we run on a version of ceph that does not know these
  flags they will simply get ignored.

Consider the following:

[leseb@tarox~/go/src/github.com/rook/rook][use-ceph-user !] kubectl get pods -n rook-ceph
NAME                                     READY     STATUS      RESTARTS   AGE
rook-ceph-mds-myfs-a-67cff7df69-fq4l5    1/1       Running     0          3m
rook-ceph-mds-myfs-b-85bd5f59f5-d8hdn    1/1       Running     0          3m
rook-ceph-mgr-a-7d88b9457b-jzln6         1/1       Running     2          4m
rook-ceph-mon-a-64bf697496-r94tc         1/1       Running     0          4m
rook-ceph-osd-0-c5fdcd74c-9kvb4          1/1       Running     0          3m
rook-ceph-osd-prepare-minikube-shz9l     0/2       Completed   0          3m
rook-ceph-rbd-mirror-a-5cbff8469-xv64c   1/1       Running     0          3m
rook-ceph-rgw-my-store-f7645b44-vlbp4    1/1       Running     0          3m

MDS daemon:
ceph         1  0.2  0.1 449508 29200 ?        Ssl  15:10   0:00 ceph-mds --fsid=bf359e2c-a1df-4a80-9137-8134b41a5459 --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug  --mon-host=10.110.204.228:6789 --mon-initial-members=a --name=mds.myfs-a --setuser=ceph --setgroup=ceph --foreground --mds-standby-for-fscid=1 --mds-standby-replay=true

MGR daemon:
ceph         1  0.8  0.6 867176 104732 ?       Ssl  15:10   0:02 ceph-mgr --fsid=bf359e2c-a1df-4a80-9137-8134b41a5459 --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug  --mon-host=10.110.204.228:6789 --mon-initial-members=a --name=mgr.a --setuser=ceph --setgroup=ceph --foreground

MON daemon:
ceph         1     0  0 15:10 ?        00:00:02 ceph-mon --fsid=bf359e2c-a1df-4a80-9137-8134b41a5459 --keyring=/etc/ceph/keyring-store/keyring --log-to-stderr=true --err-to-stderr=true --mon-cluster-log-to-stderr=true --log-stderr-prefix=debug  --mon-host=10.110.204.228:6789 --mon-initial-members=a --name=mon.a --setuser=ceph --setgroup=ceph --foreground --public-addr=10.110.204.228 --public-bind-addr=172.17.0.6 --setuser-match-path=/var/lib/ceph/mon/ceph-a

OSD daemon:
ceph     30985  0.9  0.5 885736 81236 ?        Ssl  15:10   0:04      |   \_ ceph-osd --foreground --id 0 --conf /var/lib/rook/osd0/rook-ceph.config --osd-data /var/lib/rook/osd0 --setuser-match-path /var/lib/rook/osd0 --setuser ceph --setgro
up ceph --keyring /var/lib/rook/osd0/keyring --cluster rook-ceph --osd-uuid 98b4527d-f7a8-44f0-ac47-f8b55ff5ee05 --osd-journal=/var/lib/rook/osd0/journal

RBD mirror daemon:
ceph         1  0.0  0.1 914500 21848 ?        Ssl  15:10   0:00 rbd-mirror --setuser ceph --setgroup ceph --foreground -n client.rbd-mirror.a --conf /etc/ceph/ceph.conf --keyring /etc/ceph/keyring

RGW daemon:
ceph         1  0.2  0.3 5179452 52244 ?       Ssl  15:11   0:01 radosgw --setuser ceph --setgroup ceph --foreground --name=client.radosgw.gateway --rgw-mime-types-file=/var/lib/rook/rgw/mime.types

Even though Ceph processes are running as Ceph, the container itself runs as root.
We haven't reached the situation where we can runAsUser != "root" and enable SecurityContext to RunAsNonRoot, but that's our ultimate goal.

Resolves: https://github.com/rook/rook/issues/2664
Signed-off-by: Sébastien Han <seb@redhat.com>
2019-07-19 11:13:42 +02:00

3.6 KiB
Raw Blame History

Major Themes

Action Required

Notable Features

  • Creation of storage pools through the custom resource definitions (CRDs) now allows users to optionally specify deviceClass property to enable distribution of the data only across the specified device class. See Ceph Block Pool CRD for an example usage
  • Added K8s 1.15 to the test matrix and removed K8s 1.10 from the test matrix.
  • OwnerReferences are created with the fully qualified apiVersion such that the references will work properly on OpenShift.
  • Linear disk device can now be used for Ceph OSDs.
  • The integration tests can be triggered for specific storage providers rather than always running all tests. See the dev guide for more details.

Ceph

  • Rook can now be configured to read "region" and "zone" labels on Kubernetes nodes and use that information as part of the CRUSH location for the OSDs.
  • Rgw pods have liveness probe enabled
  • Rgw is now configured with the Beast backend as of the Nautilus release
  • OSD: newly updated cluster from 0.9 to 1.0.3 and thus Ceph Nautilus will have their OSDs allowing new features for Nautilus
  • Rgw instances have their own key and thus are properly reflected in the Ceph status
  • The Rook Agent pods are now started when the CephCluster is created rather than immediately when the operator is started.
  • Ceph CRUSH tunable are not enforced to "firefly" anymore, Ceph picks the right tunable for its own version, to read more about tunable see the Ceph documentation
  • NodeAffinity can be applied to rook-ceph-agent DaemonSet with AGENT_NODE_AFFINITY environment variable.
  • NodeAffinity can be applied to rook-discover DaemonSet with DISCOVER_AGENT_NODE_AFFINITY environment variable.
  • Rook does not create an initial CRUSH map anymore and let Ceph do it normally
  • Ceph monitor placement will now take failure zones into account see the documentation for more information.
  • The cluster CRD option to allow multiple monitors to be scheduled on the same node---spec.Mon.AllowMultiplePerNode---is now active when a cluster is first created. Previously, it was ignored when a cluster was first installed.
  • Upgrades have drastically improved, Rook intelligently checks for each daemon state before and after upgrading. To learn more about the upgrade workflow see Ceph Upgrades
  • Rook Operator now supports 2 new environmental variables: AGENT_TOLERATIONS and DISCOVER_TOLERATIONS. Each accept list of tolerations for agent and discover pods accordingly.
  • Ceph daemons now run under 'ceph' user and not 'root' anymore (monitor or osd store already owned by 'root' will keep running under 'root')

Breaking Changes

Known Issues

Deprecations

Ceph

  • For rgw, when deploying an object store with object.yaml, using allNodes is not supported anymore, a transition path has been implemented in the code though. So if you were using allNodes: true, Rook will replace each daemonset with a deployment (one for one replacement) gradually. This operation will be triggered on an update or when a new version of the operator is deployed. Once complete, it is expected that you edit your object CR with kubectl -n rook-ceph edit cephobjectstore.ceph.rook.io/my-store and set allNodes: false and instances with the current number of rgw instances.

View Git Blame Copy Permalink