The name generation utils need to provide consistent outputs across
versions so that names generated based on values are consistent and
older generated keys are not lost.
Signed-off-by: Rohan CJ <rohantmp@gmail.com>
At startup there is a race condition between the goroutine checking
for mon health and the first reconcile of the mons. There is a lock
that ensures only one of them will check the mon health at a time.
The issue is that if the wrong one gets there first, it will not
have the desired mon count initialized from the cluster CR as expected
and the healther checker will use a desired mon count of 0.
The desired count of 0 leads the health checker to remove
a mon since there are extra mons. Only one mon is allowed
to be removed at a time, and adjusting to fewer than two mons is
disallowed once there are at least two mons.
After the health checker is completed, the main reconcile of the mons
will proceed with the correct desired mon count of 3 and attempt
to restore to the real desired quorum size.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The mon was starting with a partial spec. There was a race condition
that was causing incorrect initialization in the health check goroutine
that could remove a mon upon operator restart. Now the mon will be
properly initialized with the full spec.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
- Add realm/zone group/zone to Object Context,
so that any call to runAdminCommand has the
same realm, zone group, and zone as the object
store that the call is going to.
- Also modify the delete code for the object-store
using multisite to remove the endpoints of an
object store from a zone instead of the normal
clean-up
- Added more debug logging all around the object-store
code related to multisite
- files generated by rerun of `make codegen`
- change back the edit on the Copyright in object.go
Signed-off-by: Ali Maredia <amaredia@redhat.com>
When zones are created the ceph RGWs inside the
those zones should be using pools with the zones
name not the object-store's name
Signed-off-by: Ali Maredia <amaredia@redhat.com>
This commit:
- adds the pull section on the CephObjectRealm spec
to enables realms to be pulled instead of created.
- creates the system user and generates the access
key and secret key for the user and zones in a realm.
- adds "omitempty" to fields in multisite CRs that
are not required
Signed-off-by: Ali Maredia <amaredia@redhat.com>
During the deletion sequence of the CR, we now set a success flag at the
end, right before removing the finalizer.
This avoids the case where we fail to remove the finalizer.
2020-07-21 13:29:46.334816 E | ceph-object-controller: failed to reconcile failed to remove finalizer: failed to remove finalizer "cephobjectstore.ceph.rook.io" on "teststore": Operation cannot be fulfilled on cephobjectstores.ceph.rook.io "teststore": the object has been modified; please apply your changes to the latest version and try again
Signed-off-by: Sébastien Han <seb@redhat.com>
Now, we can not only cleanup monitor data, logs and crashes but the
disks too. As part of the cleanupPolicy CR spec, we have a new setting
called sanitizeDisk which holds more details:
* method: indicates if the entire disk should be sanitized or simply ceph's metadata.
Possible choices are 'complete' or 'quick' (default)
* dataSource: indicate where to get random bytes from to write on the disk.
Possible choices are 'zero' (default) or 'random'
Using random sources will consume entropy from the system and will take much more time then the zero source
* iteration: overwrite N times instead of the default (1). Takes an integer value
See the documentation for more details.
Signed-off-by: Sébastien Han <seb@redhat.com>
The label generated to apply to a node are simplified with the
backtick string delimeter to avoid the escaping of quotes.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
All assignments made to errors or other variables should be
used instead of ignored. This checks for errors in a couple
places that were previously ignored, and also initializes
a variable such that it won't be always overwritten by the
various if else statements.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The user creds should not be passed to the ceph helper methods
since the clusterInfo already has the needed context. Removing
a final reference that was missed in the refactor.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
ceph-file-controller gets stuck in loops, this happens when it tires to delete the filesystem which doesn't exist.Filesystem should only be deleted when it exist.
Signed-off-by: RAJAT SINGH <rajasing@redhat.com>
If the healthcheck goroutine cannot check for the ceph status/times out
(if the mons are not in quorum) this should still be reflected in the
CRD status field.
Example:
```
status:
ceph:
details:
error:
message: 'failed to get status. . timed out: exit status 1'
severity: Urgent
health: HEALTH_ERR
lastChanged: "2020-07-20T14:11:05Z"
lastChecked: "2020-07-20T14:11:05Z"
previousHealth: HEALTH_WARN
```
Signed-off-by: Sébastien Han <seb@redhat.com>
A long-standing issue that was only solved via a documentation note. Now,
during the prepare pod instantiation, we check for the presence of the
lvm binary on the host. Success will indicate that we can bootstrap
OSDs where failure will refuse to prepare the OSD.
Closes: https://github.com/rook/rook/issues/5627
Signed-off-by: Sébastien Han <seb@redhat.com>
Name for requeue time variables were the same for
the object-zone and the object-zonegroup.
This commit makes them more descriptive and
accurate
Signed-off-by: Ali Maredia <amaredia@redhat.com>
The masterArg flag cannot be "" so a different
command without masterArg must be run for zone
and zone group creation
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Make it so that zones & zone groups can be
modified with more than one endpoint so that
more than one object store can be in a zone
Signed-off-by: Ali Maredia <amaredia@redhat.com>
Pass desired devices to the OSD provisioning container by
JSON-marshalling/-unmarshalling a disk ID with StorageConfig settings.
This will allow disks to be specified that contain special characters.
Notably, this will support /dev/disk/by-path/pci-HHHH:HH:HH.H devices
that were unsupported previously due to the format which separated
devices from params with colons (:).
Fixes#5056Fixes#5535
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
We can now connect an external prometheus exporter to Rook to collect
metrics and generate alerts from Prometheus.
Just enable this in the CephCluster CR:
```
spec:
external:
enable: true
monitoring:
enabled: true
rulesNamespace: rook-ceph
externalMgrEndpoints:
- ip: 192.168.39.182
```
Closes: https://github.com/rook/rook/issues/5516
Signed-off-by: Sébastien Han <seb@redhat.com>
We currently don't use the module, so let's speed up the bootstrap
progress as well as the reconcile by not always enabling it.
It can always be enabled via the spec mgr section.
Signed-off-by: Sébastien Han <seb@redhat.com>
The external creds have been loaded from their own key. For
backward compatibility we need to check that key if the
new key is not found.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Health checks should log to debug rather than info
or else the operator log fills with messages as the health
checks trigger periodically.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
To provide more context for executing commands in a ceph cluster,
the full clusterInfo is now passed to the ceph execution commands.
All information about the cluster will now be available throughout
all the areas of the operator. The namespace, ceph credentials,
mon endpoints, and other info is a core part of that cluster info.
Arguments passed through the controllers are also simplified for
mons, mgr, osds, and other daemons where the parameters had
become too complex.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ceph config and client packages are conceptually the same.
To avoid circular dependencies in some cases, we simplify by combining
the packages into the client package.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The operator should only connect to ceph with a single set of creds.
In a converged cluster this will be the admin creds and in an external
cluster it will be lower-privileged creds. Independent clusters were
implemented with a separate set of creds. To simplify the code these
are now merged to a single set.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
When a PodAntiAffinity is set to mon, detect-version may not be scheduled because detect-version has the same PodAntiAffinity with mon.
This commit fixes this issue.
Closes#5810
Signed-off-by: Hiroshi Muraoka <h.muraoka714@gmail.com>
Most of the CRs expect CephCluster and CephBlockPool were not
reconciling on updates. The predicate must return true on CR diff.
Signed-off-by: Sébastien Han <seb@redhat.com>
adding method to generate secret name in
cephobjectstoreuser status field in order
to programmatically retrieve the secret name
generated by the cephobjectstoreuser crd.
Also added unit test for the method.
Signed-off-by: subhamkrai <subhamkumarrai03@gmail.com>
The output of the bucket policy change should not be logged in case
there is sensitive information in the output.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Add the ability to provision Ceph OSDs with Drive Groups.
This adds Drive Groups to the CephCluster CRD, and it sets code
in place for propagating this config to the OSD provisioning pod.
Signed-off-by: Blaine Gardner <blaine.gardner@suse.com>
moves the admission-controller server to use controller-runtime library for better support, maintainibility of code.
Signed-off-by: Vineet Badrinath <vbadrina@redhat.com>
Rohan Joseph
Travis Nielsen
Sébastien Han
Ali Maredia
RAJAT SINGH
Blaine Gardner
Jiffin Tony Thottan
Hiroshi Muraoka
subhamkrai
Ahmad Nurus S
Vineet Badrinath