The user creds should not be passed to the ceph helper methods
since the clusterInfo already has the needed context. Removing
a final reference that was missed in the refactor.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph-file-controller gets stuck in loops, this happens when it tires to delete the filesystem which doesn't exist.Filesystem should only be deleted when it exist.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
If the healthcheck goroutine cannot check for the ceph status/times out
(if the mons are not in quorum) this should still be reflected in the
CRD status field.
Example:
```
status:
ceph:
details:
error:
message: 'failed to get status. . timed out: exit status 1'
severity: Urgent
health: HEALTH_ERR
lastChanged: "2020-07-20T14:11:05Z"
lastChecked: "2020-07-20T14:11:05Z"
previousHealth: HEALTH_WARN
```
Signed-off-by: Sébastien Han <seb@redhat.com>
A long-standing issue that was only solved via a documentation note. Now,
during the prepare pod instantiation, we check for the presence of the
lvm binary on the host. Success will indicate that we can bootstrap
OSDs where failure will refuse to prepare the OSD.
Closes: https://github.com/rook/rook/issues/5627
Signed-off-by: Sébastien Han <seb@redhat.com>
Name for requeue time variables were the same for
the object-zone and the object-zonegroup.
This commit makes them more descriptive and
accurate
Signed-off-by: Ali Maredia <amaredia@redhat.com>
The masterArg flag cannot be "" so a different
command without masterArg must be run for zone
and zone group creation
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Make it so that zones & zone groups can be
modified with more than one endpoint so that
more than one object store can be in a zone
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Pass desired devices to the OSD provisioning container by
JSON-marshalling/-unmarshalling a disk ID with StorageConfig settings.
This will allow disks to be specified that contain special characters.
Notably, this will support /dev/disk/by-path/pci-HHHH:HH:HH.H devices
that were unsupported previously due to the format which separated
devices from params with colons (:).
Fixes#5056Fixes#5535
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
We can now connect an external prometheus exporter to Rook to collect
metrics and generate alerts from Prometheus.
Just enable this in the CephCluster CR:
```
spec:
external:
enable: true
monitoring:
enabled: true
rulesNamespace: rook-ceph
externalMgrEndpoints:
- ip: 192.168.39.182
```
Closes: https://github.com/rook/rook/issues/5516
Signed-off-by: Sébastien Han <seb@redhat.com>
We currently don't use the module, so let's speed up the bootstrap
progress as well as the reconcile by not always enabling it.
It can always be enabled via the spec mgr section.
Signed-off-by: Sébastien Han <seb@redhat.com>
The external creds have been loaded from their own key. For
backward compatibility we need to check that key if the
new key is not found.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Health checks should log to debug rather than info
or else the operator log fills with messages as the health
checks trigger periodically.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph config and client packages are conceptually the same.
To avoid circular dependencies in some cases, we simplify by combining
the packages into the client package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator should only connect to ceph with a single set of creds.
In a converged cluster this will be the admin creds and in an external
cluster it will be lower-privileged creds. Independent clusters were
implemented with a separate set of creds. To simplify the code these
are now merged to a single set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When a PodAntiAffinity is set to mon, detect-version may not be scheduled because detect-version has the same PodAntiAffinity with mon.
This commit fixes this issue.
Closes#5810
Signed-off-by: Hiroshi Muraoka <h.muraoka714@gmail.com>
Most of the CRs expect CephCluster and CephBlockPool were not
reconciling on updates. The predicate must return true on CR diff.
Signed-off-by: Sébastien Han <seb@redhat.com>
adding method to generate secret name in
cephobjectstoreuser status field in order
to programmatically retrieve the secret name
generated by the cephobjectstoreuser crd.
Also added unit test for the method.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
The output of the bucket policy change should not be logged in case
there is sensitive information in the output.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add the ability to provision Ceph OSDs with Drive Groups.
This adds Drive Groups to the CephCluster CRD, and it sets code
in place for propagating this config to the OSD provisioning pod.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
moves the admission-controller server to use controller-runtime library for better support, maintainibility of code.
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
This commit allows us to configure status check for each daemon:
* "mon": health check on the ceph monitors (quorum)
* "osd": health check on the ceph osds
* "status": ceph health status check
Each check is controlled by the following settings:
* disabled: whether to disable the check (default: false)
* internal: interval to run the check
* timeout: only valid for mons, is the timeout for unresponsive mon
before failling over.
Example to disable the status health check:
```yaml
healthCheck:
daemonHealth:
status:
disabled: true
```
As part of that, pod's livenessprobe can now be configured via the
following settings:
* disabled: whether to enable or not
* probe: override the current probe in place by a new one
```yaml
healthCheck:
livenessProbe:
mon:
disabled: true
```
Closes: https://github.com/rook/rook/issues/5772
Signed-off-by: Sébastien Han <seb@redhat.com>
Now, when an S3 user gets created, Rook will add the S3 endpoint to the
Secret along with the credentials.
Signed-off-by: Sébastien Han <seb@redhat.com>
For the CephObjectStore, the Status field of the CR has a new property
called "info", which contains a couple of information about the CR.
The first new addition is the endpoint of the object store.
Signed-off-by: Sébastien Han <seb@redhat.com>
The OSD daemon has been missing critical flags for logging to stderr
where k8s can capture the logs. Without the --log-to-stderr=true,
all the OSD logging was essentially lost until now.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Update the base operator image and cluster examples to use the latest
octopus release v15.2.4. Also cleanup some old examples and unit tests
that were still based on mimic.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
remove csi drivers and delete k8s
services that drivers create,when
the default setting of drivers changed
to false.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
We have introduced a new goroutine to check the state of the rgw
endpoint. It will run every minute and perform operations on a bucket.
The success or failure will be reported as part of the status field of
the CephObjectStore CR.
A good status will look like:
status:
endpointStatus:
lastChanged: "2020-06-25T13:47:45Z"
lastChecked: "2020-06-25T13:48:46Z"
phase: Connected
A failed status:
status:
endpointStatus:
details: |-
error creating bucket "rook-ceph-internal-s3-bucket-checker": RequestError: send request failed
caused by: Put http://rook-ceph-rgw-my-store.rook-ceph:8080/rook-ceph-internal-s3-bucket-checker: dial tcp 10.108.189.148:8080: connect: connection refused
health: ERROR
This check works for both converged and external modes. Note that the
CephObjectStore CRD has a new field called "externalRgwEndpoints" which
allows you to define a list of IP addresses pointing to rgws.
Closes: https://github.com/rook/rook/issues/5692
Signed-off-by: Sébastien Han <seb@redhat.com>
removed CSI_ENABLE_SNAPSHOTTER from the configuration
to enable or disable csi-snapshotter as snapshot
is moved to beta and enabled by default in kubernetes
1.17+
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as we need kubernetes v1.17+ for snapshotter
to work,Added code changes to disable snapshotter
sidecar deployment by default if the kubernetes version
is less than v1.17+
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Travis Nielsen
RAJAT SINGH
Sébastien Han
subhamkrai
Ali Maredia
Blaine Gardner
Jiffin Tony Thottan
Hiroshi Muraoka
Ahmad Nurus S
Vineet Badrinath
Madhu Rajanna