Sébastien Han
bd58790c31
When the CephCluster is configured with Multus and multiple networks are used to deploy Ceph some commands are failing to be executed from the Operator. These commands, in particular, `radosgw-admin` ones need access to the "ceph public network" to talk to OSDs. Unfortunately, the Rook-Ceph Operator does not have the network annotations and thus doesn't have the networks available and cannot reach OSDs. So the commands end up hanging and eventually time out. Applying the annotations to the Operator pod is possible but will result in restarting the operator too and this should be avoided at all costs. Also, applying the annotations beforehand is not possible since the Multus declaration is in the CephCluster specification. So we would have no idea what to do. So the current approach runs a new sidecar container in the mgr pod to act as a proxy for "some" ceph commands, only the `radosgw-admin` ones for multi-site setup. This is a small container with admin access running idle waiting for commands to be executed. In a sense, it is similar to the toolbox but we didn't want to clearly expose it, so running as a sidecar is quite nice. Proxying command is obviously not always recommended since we add an extra hop in the network path. Now each request has to go from the operator pod to the API server to the remote pod to Ceph. Previously, the command only goes from the operator to Ceph. It's worth noting that external mode is not impacted since no rgw pod is configured. This scenario is flexible and allows us to scale pretty well since any CephCluster with Multus will see its mgr sidecar deployed and can then talk to Ceph. We are not limited. Signed-off-by: Sébastien Han <seb@redhat.com>
81 lines
3.3 KiB
AMPL
81 lines
3.3 KiB
AMPL
module github.com/rook/rook
|
|
|
|
go 1.16
|
|
|
|
require (
|
|
github.com/aws/aws-sdk-go v1.35.24
|
|
github.com/banzaicloud/k8s-objectmatcher v1.1.0
|
|
github.com/ceph/go-ceph v0.9.1-0.20210607162346-8179bd4437f9
|
|
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f
|
|
github.com/csi-addons/volume-replication-operator v0.1.1-0.20210525040814-ab575a2879fb
|
|
github.com/davecgh/go-spew v1.1.1
|
|
github.com/ghodss/yaml v1.0.1-0.20190212211648-25d852aebe32
|
|
github.com/go-ini/ini v1.51.1
|
|
github.com/google/go-cmp v0.5.5
|
|
github.com/google/uuid v1.1.2
|
|
github.com/hashicorp/vault/api v1.0.5-0.20200902155336-f9d5ce5a171a
|
|
github.com/k8snetworkplumbingwg/network-attachment-definition-client v1.1.0
|
|
github.com/kube-object-storage/lib-bucket-provisioner v0.0.0-20210311161930-4bea5edaff58
|
|
github.com/libopenstorage/secrets v0.0.0-20201006135900-af310b01fe47
|
|
github.com/openshift/cluster-api v0.0.0-20191129101638-b09907ac6668
|
|
github.com/openshift/machine-api-operator v0.2.1-0.20190903202259-474e14e4965a
|
|
github.com/pkg/errors v0.9.1
|
|
github.com/prometheus-operator/prometheus-operator v0.43.0
|
|
github.com/prometheus-operator/prometheus-operator/pkg/apis/monitoring v0.43.0
|
|
github.com/spf13/cobra v1.1.1
|
|
github.com/spf13/pflag v1.0.5
|
|
github.com/stretchr/testify v1.7.0
|
|
github.com/tevino/abool v1.2.0
|
|
github.com/yanniszark/go-nodetool v0.0.0-20191206125106-cd8f91fa16be
|
|
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
|
|
gopkg.in/ini.v1 v1.57.0
|
|
gopkg.in/yaml.v2 v2.4.0
|
|
k8s.io/api v0.21.2
|
|
k8s.io/apiextensions-apiserver v0.21.1
|
|
k8s.io/apimachinery v0.21.2
|
|
k8s.io/apiserver v0.21.1
|
|
k8s.io/client-go v0.21.2
|
|
k8s.io/cloud-provider v0.21.1
|
|
k8s.io/component-helpers v0.21.1
|
|
k8s.io/kube-controller-manager v0.21.1
|
|
k8s.io/utils v0.0.0-20210527160623-6fdb442a123b
|
|
sigs.k8s.io/controller-runtime v0.9.0
|
|
sigs.k8s.io/kustomize/kyaml v0.10.17
|
|
sigs.k8s.io/sig-storage-lib-external-provisioner/v6 v6.1.0
|
|
)
|
|
|
|
replace (
|
|
github.com/googleapis/gnostic => github.com/googleapis/gnostic v0.4.1
|
|
github.com/kubernetes-incubator/external-storage => github.com/libopenstorage/external-storage v0.20.4-openstorage-rc3
|
|
github.com/portworx/sched-ops => github.com/portworx/sched-ops v0.20.4-openstorage-rc3
|
|
google.golang.org/grpc => google.golang.org/grpc v1.27.1
|
|
)
|
|
|
|
exclude (
|
|
// This tag doesn't exist, but is imported by github.com/portworx/sched-ops.
|
|
github.com/kubernetes-incubator/external-storage v0.20.4-openstorage-rc2
|
|
// Exclude pre-go-mod kubernetes tags, because they are older
|
|
// than v0.x releases but are picked when updating dependencies.
|
|
k8s.io/client-go v1.4.0
|
|
k8s.io/client-go v1.5.0
|
|
k8s.io/client-go v1.5.1
|
|
k8s.io/client-go v1.5.2
|
|
k8s.io/client-go v10.0.0+incompatible
|
|
k8s.io/client-go v11.0.0+incompatible
|
|
k8s.io/client-go v11.0.1-0.20190409021438-1a26190bd76a+incompatible
|
|
k8s.io/client-go v12.0.0+incompatible
|
|
k8s.io/client-go v2.0.0+incompatible
|
|
k8s.io/client-go v2.0.0-alpha.1+incompatible
|
|
k8s.io/client-go v3.0.0+incompatible
|
|
k8s.io/client-go v3.0.0-beta.0+incompatible
|
|
k8s.io/client-go v4.0.0+incompatible
|
|
k8s.io/client-go v4.0.0-beta.0+incompatible
|
|
k8s.io/client-go v5.0.0+incompatible
|
|
k8s.io/client-go v5.0.1+incompatible
|
|
k8s.io/client-go v6.0.0+incompatible
|
|
k8s.io/client-go v7.0.0+incompatible
|
|
k8s.io/client-go v8.0.0+incompatible
|
|
k8s.io/client-go v9.0.0+incompatible
|
|
k8s.io/client-go v9.0.0-invalid+incompatible
|
|
)
|