Commit Graph
82 Commits
Author SHA1 Message Date
Noah Watkins b29ba26b12 exec: add timeout variant of exec with output file
the existing exec interface with timeout is effectively the same as
ExecuteCommandWithOutput plus a timeout. this patch adds a variant of
ExecuteCommandWithOutputFile that uses a timeout.

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-05-06 10:10:05 -07:00
Sébastien Han 871054e887 ceph: implement pod restriction on memory
If someone sets limits to pod, we want to ensure the possible
experience, so we want to make sure that people do not configure
inapropriate values for certain daemons.
We decide to fail if the memory.limit is too low.

Signed-off-by: Sébastien Han <seb@redhat.com>
2019-03-15 17:35:44 +01:00
travisn bdc3cf8146 osd: fix the device filter and improve device provisioning reliability
All devices detected by the discovery pod were being passed to the OSD provisioning pod
thus not always honoring the desired device list that should be provisioned.
Now the provisioning pod will be given the desired state from the crd,
then apply that state depending on the actual devices detected.
Also added a helper to ensure OSDsPerDevice is always valid.

Signed-off-by: travisn <tnielsen@redhat.com>
2019-02-26 16:11:25 -07:00
Noah Watkins fbb56c41ed exec: interleave stdout/stderr logging
The io.MultiReader(r1, r2) reads r1 until EOF before moving on to r2.
When used for stdout/stderr stderr will not be written to the log until
stdout reaches EOF. This patch reads stderr in a go routine so that the
two streams can be interleaved properly.

Fixes: #2479

Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
2019-01-24 13:10:46 -08:00
travisn f7c266f7ec skip sgdisk detection if not found in nautilus image
Signed-off-by: travisn <tnielsen@redhat.com>
2018-12-06 21:28:57 -07:00
Alexander Trost b12d1e82cf Use truncated node name "everywhere"
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
2018-11-23 10:02:35 +01:00
Angus Lees 9c95ed5f6f Don't misinterpret command output as string format
Previous code tried to interpret the string output as a `fmt.Sprintf`
string formatter, mangling `%` into `%!(MISSING)`, etc.

Thanks to go, I don't belive these are exploitable (unlike the similar
error in C).

Since this seemed to be a common error in the codebase, I did a quick
audit by visually inspecting the results of `git grep 'f([^"]'`.  I
don't have a good suggestion for automated tests to prevent this in
future :(

Example error (look for `(MISSING)`):
```
E0927 05:31:07.618429   11227 driver-call.go:237] Failed to unmarshal output for command: unmount, output: "2018-09-27 05:31:07.191711 I | exec: Running command: df --type ceph /var/lib/kubelet/pods/95461479-c216-11e8-bcf0-02030782ac80/volumes/ceph.rook.io~rook/oe-scratch\n2018-09-27 05:46:43.808596 I | Filesystem                                                 1K-blocks     Used Available Use%!M(MISSING)ounted on\n2018-09-27 05:46:43.808659 I | 10.107.25.147:6790,10.109.173.79:6790,10.104.85.255:6790:/ 151678976 49410048 102268928  33%!/(MISSING)var/lib/kubelet/pods/95461479-c216-11e8-bcf0-02030782ac80/volumes/ceph.rook.io~rook/oe-scratch\n{\"status\":\"Success\"}\n", error: invalid character '-' after top-level value
```

Signed-off-by: Angus Lees <gus@inodes.org>
2018-10-15 15:59:13 +11:00
Ganesh Maharaj Mahalingam 30dcd26da9 Enable glog logging
Change enables glog logging and also make default flag values
configurable.

Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2018-09-24 13:17:25 -07:00
Ganesh Maharaj Mahalingam 92f91ab156 golang 1.11 support
Signed-off-by: Ganesh Maharaj Mahalingam <ganesh.mahalingam@intel.com>
2018-09-12 10:53:53 -07:00
Dennis Chen 9cfafaa0a7 Enhancement of Ceph volume manager kmod detection
Current Ceph volume manager code doesn't take the builtin
kernel module into the account, so the `modinfo` will fail
in this case. Also, we need to add error handling code to
make sure the volume manager does work when proceeding to
continue, otherwise we will encounter the `rbd` related issue
in the following step even we have an artificial `agent` pod
in running state.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-07-31 16:56:40 +08:00
Blaine Gardner fe5fb394c7 Fix spellcheck and trailing space/newline issues
Fix some basic spellcheck errors. Also remove trailing spaces and make
sure files have a newline (my editor does automatically).

Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
2018-07-26 11:36:09 -06:00
Huamin Chen dc1e827cce get partlabel from udev info PARTNAME
Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-07-05 16:09:21 -06:00
travisn f1ad88e0ac tests: enable osds on devices
Signed-off-by: travisn <tnielsen@redhat.com>
2018-07-05 16:09:20 -06:00
Huamin Chen 82425aafcc prepare osd in a job per node. Once all osds are prepared, store osd info in orchestration configmap.
Operator watches the configmap, starts one osd replica set per osd.

Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-07-05 16:02:27 -06:00
Evgeniy Klemin 759ec55f64 Fix retrieve device label
Signed-off-by: Evgeniy Klemin <evgeniy.klemin@gmail.com>
2018-06-27 13:31:46 +03:00
Jared Watts 48b879e0ad Merge pull request #1769 from ucloudlab/ipv6-support
support ipv6
2018-06-19 20:33:12 -07:00
travisn d5d3540497 osd: detect disk uuid with sgdisk
Signed-off-by: travisn <tnielsen@redhat.com>
2018-06-19 13:05:51 -07:00
Aayush Sarva 80f3e8ebf4 Replace custom Awk and remove related tests
Signed-off-by: Aayush Sarva <checkaayush@gmail.com>

Return error if field not found

Signed-off-by: Aayush Sarva <checkaayush@gmail.com>
2018-06-19 05:03:28 +05:30
Zhang Miaolei 100e80250f support ipv6 host-port pair
deprecate NetworkInfo.PublicAddrIPv4, use NetworkInfo.PublicAddr instead
deprecate NetworkInfo.ClusterAddrIPv4, use NetworkInfo.ClusterAddr instead
deprecate command flag public-ipv4, use public-ip instead
deprecate command flag private-ipv4, use private-ip instead
change ROOK_PUBLIC_IPV4 to ROOK_PUBLIC_IP
change ROOK_PRIVATE_IPV4 to ROOK_PRIVATE_IP

Signed-off-by: Zhang Miaolei <zmlcc@outlook.com>
2018-06-15 14:26:42 +08:00
Huamin Chen 6c11ff52d4 add device discovery daemon to operator:
run "rook discover" on storage nodes and discover devices on each node. The discovered disks are saved in a per node configmap, local-device-nodename.
Device information consits of name and persistent names, uuid, partition, filesystem, rotational, readonly, size, etc.

Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-05-07 17:49:02 +00:00
Jared Watts 6a2e20aaa8 include stderr output in CommandError.Error() output
Signed-off-by: Jared Watts <jbw976@gmail.com>
2018-03-28 13:26:00 -07:00
David González Ruiz 093d881b9f [OSD] Add crypt type to the array of supported block devices
[skip ci]

Signed-off-by: David González Ruiz <godboole@gmail.com>
2018-01-31 17:41:22 +01:00
Jared Watts 202650f6f5 Add and remove storage nodes 2018-01-24 14:31:20 -08:00
Travis Nielsen 9b94111dd2 consolidate the kvstore interface 2018-01-03 08:36:43 -08:00
Pavel Snagovsky 1eb6d5bb65 WIP: the first path of OSD monitoring
Re: #667
2017-11-01 13:45:31 -07:00
Travis Nielsen 4e6774c063 remove legacy standalone code 2017-10-17 12:23:57 -07:00
Steve Leon 724a78cc6b Moving FindRBDMappedFile to ceph/util/
Fixes https://github.com/rook/rook/issues/1056
2017-10-12 15:51:59 -07:00
Steve Leon b4279e06fb Rook plugin for Kubernetes implemented as flexvolume
- Operator is deployed under namespace rook-system
- Operator deploys rook-agent on all nodes as daemonset also on
  rook-system
- Rook-agent installs flexvolume driver on hosts
- Rook-agent listens on unix socket for driver requests
- Rook-agent perform attach/detach on its node
- Rook-agent creates/delete CRD volumeattach objects
- Added fencing to support ROX and RWO
- Added unit and integration tests
- Updated examples and docks

fixes #432
2017-10-09 09:14:08 -07:00
Jared Watts 113e861020 osd: Allow OSDs to recover from losing all their metadata and config 2017-10-03 15:54:02 -07:00
Jared Watts caa9fdf96a log version, arguments and flags at start up 2017-08-24 14:02:53 -07:00
Travis Nielsen ec181c0390 logs: stop writing the frequent health check 2017-08-07 10:58:52 -07:00
Bassam Tabbara cdc26ef162 enable 'gofmt -s' and fix issues 2017-07-16 21:24:13 -07:00
Jared Watts 299ce8f62f proc: make go-ps a dependency instead of using in-tree copy 2017-07-13 18:14:16 -07:00
Travis Nielsen 059987e6cf build: unit tests pass on mac 2017-06-23 16:00:21 -07:00
Travis Nielsen cb5741989b log all output of commands executed with an output file 2017-06-21 11:29:14 -07:00
Jared Watts e4045eb74b Use blkid to get PARTLABEL because lsblk depends on udev which isn't available in containers 2017-06-21 10:02:41 -07:00
Jared Watts 585501b925 use an output file when invoking ceph commands to isolate the payload from any logging 2017-06-20 20:57:46 -07:00
Travis Nielsen 0a035ed402 support for RBD images, all api tests passing
RBD image support fixes, ensure /etc/ceph is created in toolbox image, return output string in exec even for errors because the ceph tools return useful information there upon error
2017-06-20 20:57:46 -07:00
Travis Nielsen fab106a16c Unit tests passing except api after the refactor 2017-06-20 20:57:46 -07:00
Travis Nielsen c122562a7d Rook refactor to call external ceph tools instead of embedded ceph
Remove all cgo from the code base, and start the transformation to calling external ceph tools to generate config and start daemons.
2017-06-20 20:57:45 -07:00
Travis Nielsen 58aa72a254 remove dependency on bash 2017-05-05 12:25:31 -07:00
Jared Watts d185e6b2cf For bluestore OSDs look at the data partition size to determine CRUSH weight 2017-04-14 15:12:25 -07:00
Steve Leon c46c076096 Updating dependency to support K8S 1.6 2017-04-12 13:27:55 -07:00
Jared Watts e5cca80654 Operator and cephmgr support for specifying storage resources and configuration 2017-04-04 15:32:49 -07:00
Michael Goff 4870f951f0 cli and rgw: Added support for list, create, update, get, and delete users as well as listing buckets. Updated connection info to be for a user. 2017-02-08 09:50:19 -08:00
Travis Nielsen a6b4430d8a start the rgw service 2017-02-01 09:50:02 -08:00
Travis Nielsen 3c56abd279 factor out rgw daemon to run both in k8s and rook orchestration 2017-02-01 09:48:53 -08:00
Jared Watts d383fc7d45 Backoff exponentially restarting processes that start but crash right away 2017-01-25 14:44:36 -08:00
Travis Nielsen 5d76276f29 detect rook partitions
(cherry picked from commit 0f5552389eba6dad8babab027a05a5466517177f)
2017-01-20 16:48:17 -08:00
Travis Nielsen e355cdb68d write ceph daemon conf files to the log 2017-01-03 08:15:44 -08:00