We still see the CI failing with:
```
Traceback (most recent call last):
File "/etc/ceph/create-external-cluster-resources.py", line 801, in <module>
raise err
File "/etc/ceph/create-external-cluster-resources.py", line 798, in <module>
rjObj.main()
File "/etc/ceph/create-external-cluster-resources.py", line 779, in main
generated_output = self.gen_json_out()
File "/etc/ceph/create-external-cluster-resources.py", line 613, in gen_json_out
self._gen_output_map()
File "/etc/ceph/create-external-cluster-resources.py", line 598, in _gen_output_map
self.out_map['MONITORING_ENDPOINT_PORT'] = self.get_active_and_standby_mgrs()
File "/etc/ceph/create-external-cluster-resources.py", line 389, in get_active_and_standby_mgrs
self.endpoint_dial(monitoring_endpoint)
File "/etc/ceph/create-external-cluster-resources.py", line 245, in endpoint_dial
"unable to connect to endpoint: {}".format(endpoint_str))
__main__.ExecutionFailureException: unable to connect to endpoint: 172.17.0.7:9283
command terminated with exit code 1
```
Signed-off-by: Sébastien Han <seb@redhat.com>
corrected the component in mirroring yaml and added
appropriate mirroring related affinity example.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The rook.io/v1 package was only an internal implementation detail and
does not have any CRDs that rely on it. The CRD deserialization should
handle the change in internal types without any issue. This separation
gives more flexibility for the storage providers to implement exactly
what is needed for their storage provider instead of forcing to use the
same types and risk affecting another storage provider.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
The ingress kind is found in apiversion networking.k8s.io/v1 since K8s
1.19. We now update the examples to use v1 instead of v1beta1.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
With this change, the Rook ceph cluster can have only a public network or
only a cluster network or both can be specified. Prior to this commit if
only a public network was specified, it was copied to the cluster network.
Signed-off-by: rohan47 <rohgupta@redhat.com>
`CephObjectStore->gateway->type` is not used. Rook has only supported s3-like
interface and hasn't had no code which handles `type` field.
In addition, this field was removed from CRD in the following commit.
ceph: auto-gen crds
31db03fece
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Enabling rbac is a legacy setting from way back when the operator
was creating the rbac internally, instead of declaring it explicitly
in the common.yaml. There is no need for this setting now.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
After mon failover is initiated, there was a time window where if the operator
was restarted, the new mon is started and has joined quorum, but the operator
does not believe the mon should be in quorum after the operator restart.
The operator was mistakenly removing the extra mon prematurely, sometimes
causing quorum to be lost if another mon was also down at the same time.
If the mon does not come back online, steps to recover quroum would need
to be followed from the disaster guide. Now the expected list of mons
will be updated immediately during mon failover if the operator successfully
created the new mon deployment, thus removing the window where restarting
the operator can cause quorum loss.
Signed-off-by: Travis Nielsen <tnielsen@redhat.com>
Before running the external script test let's make sure the prometheus
exporter is loaded. We often see in the CI errors like:
```
kubectl -n rook-ceph exec $(kubectl get pod -l app=rook-ceph-tools -n rook-ceph -o jsonpath='{.items[0].metadata.name}') -- python3 /etc/ceph/create-external-cluster-resources.py --rbd-data-pool-name replicapool
shell: /bin/bash -e {0}
env:
GOROOT: /opt/hostedtoolcache/go/1.16.4/x64
MINIKUBE_HOME: /home/runner/work/_temp
Execution Failed: unable to connect to endpoint: 172.17.0.10:9283
Traceback (most recent call last):
File "/etc/ceph/create-external-cluster-resources.py", line 801, in <module>
raise err
File "/etc/ceph/create-external-cluster-resources.py", line 798, in <module>
rjObj.main()
File "/etc/ceph/create-external-cluster-resources.py", line 779, in main
generated_output = self.gen_json_out()
File "/etc/ceph/create-external-cluster-resources.py", line 613, in gen_json_out
self._gen_output_map()
File "/etc/ceph/create-external-cluster-resources.py", line 598, in _gen_output_map
self.out_map['MONITORING_ENDPOINT_PORT'] = self.get_active_and_standby_mgrs()
File "/etc/ceph/create-external-cluster-resources.py", line 389, in get_active_and_standby_mgrs
self.endpoint_dial(monitoring_endpoint)
File "/etc/ceph/create-external-cluster-resources.py", line 245, in endpoint_dial
"unable to connect to endpoint: {}".format(endpoint_str))
__main__.ExecutionFailureException: unable to connect to endpoint: 172.17.0.10:9283
command terminated with exit code 1
```
but then when we log into the runner, we can see the command works.
So let's wait a little bit for the module to be up.
Signed-off-by: Sébastien Han <seb@redhat.com>
Changing the livenessprobe handler is not possible anymore for the
simple reason that it does not make sense. Practicly speaking we just
cannot change its value since it is propapaged to all the daemons. The
exec handler contains the socket name of the daemon, because daemons
have different names we cannot possibly make it generic since it will
apply to all.
Closes: https://github.com/rook/rook/issues/7842
Signed-off-by: Sébastien Han <seb@redhat.com>
A directory is created under dataDirHostPath for each OSD. In host-based cluster,
its name is constructed from kernel device name like _dev_sdb though user specifies
udev persistent device name. We should use unique name among all OSDs.
Closes: https://github.com/rook/rook/issues/7878
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
The preparation script does not require any manual inputs anymore. The
cluster/examples/kubernetes/ceph/create-external-cluster-resources.sh
will create and set the env variable needed for the import script.
Hopefully this will reduce the friction in some installation, especially
for the mon ports.
Signed-off-by: Sébastien Han <seb@redhat.com>
If the cluster is external we want to periodically rehydrate the mgr
endpoint. This handles the scenarion where the active manager changes,
so we need to update the endpoint with the new IP address.
The create-external-cluster-resources.py script now requires an extra
permission to query the manager service so Rook can discover the active
one and its IP.
Testing:
```
[leseb@tarox~/go/src/github.com/rook/rook][external-active-mgr-change] minikube kubectl -- exec -n rook-ceph deploy/rook-ceph-tools -ti -- ceph mgr stat
{
"epoch": 37,
"available": true,
"active_name": "b",
"num_standby": 1
}
[leseb@tarox~/go/src/github.com/rook/rook][external-active-mgr-change] kubectl -n rook-ceph-external get ep
NAME ENDPOINTS AGE
rook-ceph-mgr-external 172.17.0.12:9283 3m10s
[leseb@tarox~/go/src/github.com/rook/rook][external-active-mgr-change] k scale --replicas=0 deployment rook-ceph-mgr-b
deployment.apps/rook-ceph-mgr-b scaled
[leseb@tarox~/go/src/github.com/rook/rook][external-active-mgr-change] minikube kubectl -- exec -n rook-ceph deploy/rook-ceph-tools -ti -- ceph mgr stat
{
"epoch": 40,
"available": true,
"active_name": "a",
"num_standby": 0
}
[leseb@tarox~/go/src/github.com/rook/rook][external-active-mgr-change] kubectl -n rook-ceph-external get ep
NAME ENDPOINTS AGE
rook-ceph-mgr-external 172.17.0.13:9283 3m55s
```
Signed-off-by: Sébastien Han <seb@redhat.com>
Blaine Gardner
Sébastien Han
Shachar Sharon
subhamkrai
Travis Nielsen
Madhu Rajanna
Nitin Goyal
rohan47
Satoru Takeuchi
Diego Abelenda