OSD on PVC does not upgrade when the user upgrades the ceph version on cluster-on-pvc yaml.
The solution incudes:
- upgrade osd prepare and daemon pods on upgrade
- skip c-v prepare if filesystem is already present on the pvc device.
- skip lvm release in case of upgrade.
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
When both PARTNAME and ID_PART_ENTRY_NAME are specified, rook takes
PARTNAME in account, overriding ID_PART_ENTRY_NAME. As PARTNAME is not
always updated by the kernel after sgdisk --change-name, it's better to
do the opposite. It also makes GetDevicePartitions coherent with
parsePartLabel (called by GetPartitionLabel).
Signed-off-by: Mikaël Cluseau <mikael.cluseau@gmail.com>
- Added code to support StorageClassDeviceSet spec provided in the cluster-on-pvc.yaml
- The code reads the StorageClassDeviceSet spec and creates pvc based on the ‘count’ field for each device set.
- OSD prepare job is started for each PVC which activates the ceph-volume on each PVC
- Finally OSD is started on each of the PVC device.
Co-authored-by: rohan47 <rohgupta@redhat.com>
Co-authored-by: Ashish Ranjan <aranjan@redhat.com>
Signed-off-by: Santosh Pillai <sapillai@redhat.com>
Linear raid can be created using `mdadm --create --level=linear`,
By creating a linear raid disk on top of a logical volume,
we can pass the linear device to `ceph-volume lvm batch --prepare`
to provision a new OSD on top of the logical volume since
`ceph-volume` does not take lv as the data device.
Signed-off-by: Ash Wu <hSATAC@gmail.com>
Signed-off-by: Guy Margalit <guymguym@gmail.com>
Co-Authored-By: Sébastien Han <seb@redhat.com>
Co-Authored-By: Travis Nielsen <tnielsen@redhat.com>
This change is meant to allow running operators locally on a developer machine.
The idea is to allow faster development cycles by reducing the time and complexity of building -> deploying -> debugging on cluster.
For operators that rely only on kubernetes API this works easily - see cockroachdb and minio examples in development-flow doc.
The change includes:
- rook.NewContext() - Refactored to remove repeating initialization code that was copy-pasted in most of the operators in order to create the clusterd.Context and the Clientsets. Also it detects the mode of working in-cluster vs external and sets up the external mode with standard user config (~/.kube/config) and a job executor.
- rook.GetOperatorImage() - Refactor this repeating code in many operators to detect the operator pod image. Also added a global flag --operator-image that developers can use to override this when running locally.
- rook.TerminateOnError() - Added a convenient function.
the existing exec interface with timeout is effectively the same as
ExecuteCommandWithOutput plus a timeout. this patch adds a variant of
ExecuteCommandWithOutputFile that uses a timeout.
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
If someone sets limits to pod, we want to ensure the possible
experience, so we want to make sure that people do not configure
inapropriate values for certain daemons.
We decide to fail if the memory.limit is too low.
Signed-off-by: Sébastien Han <seb@redhat.com>
All devices detected by the discovery pod were being passed to the OSD provisioning pod
thus not always honoring the desired device list that should be provisioned.
Now the provisioning pod will be given the desired state from the crd,
then apply that state depending on the actual devices detected.
Also added a helper to ensure OSDsPerDevice is always valid.
Signed-off-by: travisn <tnielsen@redhat.com>
The io.MultiReader(r1, r2) reads r1 until EOF before moving on to r2.
When used for stdout/stderr stderr will not be written to the log until
stdout reaches EOF. This patch reads stderr in a go routine so that the
two streams can be interleaved properly.
Fixes: #2479
Signed-off-by: Noah Watkins <noahwatkins@gmail.com>
Previous code tried to interpret the string output as a `fmt.Sprintf`
string formatter, mangling `%` into `%!(MISSING)`, etc.
Thanks to go, I don't belive these are exploitable (unlike the similar
error in C).
Since this seemed to be a common error in the codebase, I did a quick
audit by visually inspecting the results of `git grep 'f([^"]'`. I
don't have a good suggestion for automated tests to prevent this in
future :(
Example error (look for `(MISSING)`):
```
E0927 05:31:07.618429 11227 driver-call.go:237] Failed to unmarshal output for command: unmount, output: "2018-09-27 05:31:07.191711 I | exec: Running command: df --type ceph /var/lib/kubelet/pods/95461479-c216-11e8-bcf0-02030782ac80/volumes/ceph.rook.io~rook/oe-scratch\n2018-09-27 05:46:43.808596 I | Filesystem 1K-blocks Used Available Use%!M(MISSING)ounted on\n2018-09-27 05:46:43.808659 I | 10.107.25.147:6790,10.109.173.79:6790,10.104.85.255:6790:/ 151678976 49410048 102268928 33%!/(MISSING)var/lib/kubelet/pods/95461479-c216-11e8-bcf0-02030782ac80/volumes/ceph.rook.io~rook/oe-scratch\n{\"status\":\"Success\"}\n", error: invalid character '-' after top-level value
```
Signed-off-by: Angus Lees <gus@inodes.org>
Current Ceph volume manager code doesn't take the builtin
kernel module into the account, so the `modinfo` will fail
in this case. Also, we need to add error handling code to
make sure the volume manager does work when proceeding to
continue, otherwise we will encounter the `rbd` related issue
in the following step even we have an artificial `agent` pod
in running state.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Fix some basic spellcheck errors. Also remove trailing spaces and make
sure files have a newline (my editor does automatically).
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
deprecate NetworkInfo.PublicAddrIPv4, use NetworkInfo.PublicAddr instead
deprecate NetworkInfo.ClusterAddrIPv4, use NetworkInfo.ClusterAddr instead
deprecate command flag public-ipv4, use public-ip instead
deprecate command flag private-ipv4, use private-ip instead
change ROOK_PUBLIC_IPV4 to ROOK_PUBLIC_IP
change ROOK_PRIVATE_IPV4 to ROOK_PRIVATE_IP
Signed-off-by: Zhang Miaolei <zmlcc@outlook.com>
run "rook discover" on storage nodes and discover devices on each node. The discovered disks are saved in a per node configmap, local-device-nodename.
Device information consits of name and persistent names, uuid, partition, filesystem, rotational, readonly, size, etc.
Signed-off-by: Huamin Chen <hchen@redhat.com>
- Operator is deployed under namespace rook-system
- Operator deploys rook-agent on all nodes as daemonset also on
rook-system
- Rook-agent installs flexvolume driver on hosts
- Rook-agent listens on unix socket for driver requests
- Rook-agent perform attach/detach on its node
- Rook-agent creates/delete CRD volumeattach objects
- Added fencing to support ROX and RWO
- Added unit and integration tests
- Updated examples and docks
fixes#432
RBD image support fixes, ensure /etc/ceph is created in toolbox image, return output string in exec even for errors because the ceph tools return useful information there upon error
Santosh Pillai
Mikaël Cluseau
rohan47andAshish Ranjan
Ash Wu
Guy Margalit
Noah Watkins
Sébastien Han
travisn
Alexander Trost
Angus Lees
Ganesh Maharaj Mahalingam
Dennis Chen
Blaine Gardner
Huamin Chen
Evgeniy Klemin
Jared Watts
Aayush Sarva
Zhang Miaolei
David González Ruiz
Travis Nielsen
Pavel Snagovsky
Steve Leon
Bassam Tabbara
Michael Goff