# MikroTik operator This is an initial Kubernetes operator that reconciles annotated `LoadBalancer` Services into RouterOS IPv4 `dst-nat` rules. The controller reads the internal destination address from `Service.status.loadBalancer.ingress[].ip`. In this cluster MetalLB publishes and announces that address over L2. The design does not depend on MetalLB specifically, so the same contract works with Cilium LB IPAM/L2 announcements. ## API contract Routers are cluster-scoped because Services in several namespaces may use the same physical router. Credentials are stored in a Kubernetes Secret and only a reference is placed in the `MikroTikRouter` resource. Every router must explicitly list the namespaces permitted to create Internet-facing rules; an empty allowlist permits none. For a Service with one public port, the compact annotations create one rule: ```yaml metadata: annotations: mikrotik.brunner.ninja/router: newgw mikrotik.brunner.ninja/service-port: game-udp mikrotik.brunner.ninja/dst-port: "34197" mikrotik.brunner.ninja/in-interface-list: WAN ``` `service-port` selects a Service port by name or number and is optional when the Service has exactly one port. `dst-port` is the public port and defaults to the selected Service port. For multiple public ports, use the `forwardings` annotation with uniquely named entries: ```yaml metadata: annotations: mikrotik.brunner.ninja/router: newgw mikrotik.brunner.ninja/forwardings: >- [ {"name":"http","servicePort":"web","dstAddressList":"WANs"}, {"name":"https","servicePort":"websecure","dstAddressList":"WANs"} ] ``` Each entry accepts `servicePort`, `dstPort`, `dstAddress`, `dstAddressList`, `inInterface`, `inInterfaceList`, `srcAddress`, and `srcAddressList`. Protocol is inherited from the selected Service port. A contiguous range can be mapped without creating one RouterOS rule per port when every port in the range exists on the Service and uses the same protocol: ```yaml mikrotik.brunner.ninja/forwardings: >- [{"name":"relay","servicePortRange":"49160-49200","dstAddressList":"WANs"}] ``` `dstPort` defaults to `servicePort` or `servicePortRange`; if overridden, its range must contain the same number of ports. The generated RouterOS properties are: | Source | RouterOS property | | --- | --- | | annotation `dst-port` | `dst-port` | | annotation `dst-address` | `dst-address` | | annotation `dst-address-list` | `dst-address-list` | | annotation `in-interface` | `in-interface` | | annotation `in-interface-list` | `in-interface-list` | | annotation `src-address` | `src-address` | | annotation `src-address-list` | `src-address-list` | | selected Service protocol | `protocol` | | LoadBalancer status IPv4 | `to-addresses` | | selected Service `port` | `to-ports` | RouterOS calls the incoming/WAN matcher `in-interface`, not `src-interface`. `src-address` refers to the remote client's address. Single managed rules have comments of the form `k8s-mikrotik//`. Named forwardings append `/`. The controller never modifies rules without the `k8s-mikrotik/` prefix. It creates and updates desired rules, deletes stale owned rules, removes duplicate owned rules, and preserves existing rules if the corresponding Service temporarily has no LoadBalancer IP or invalid annotation JSON. Deleting a `MikroTikRouter` removes all of its managed rules before the CR is deleted. If RouterOS cannot be reached, the cleanup finalizer deliberately keeps the CR in `Terminating` rather than silently leaving public ports open. ## RouterOS preparation The implementation uses the RouterOS v7 REST API. Enable `www-ssl`, configure a certificate, and create a dedicated user rather than using `admin`. RouterOS 7.24 on `newgw` also requires the `api` policy for REST operations, so the verified group policies are `read`, `write`, `api`, and `rest-api`: ```routeros /user/group/add name=kubernetes-nat policy=read,write,api,rest-api /user/add name=kubernetes-nat group=kubernetes-nat password= address= /ip/service/enable www-ssl ``` Restrict `www-ssl` with both `/ip service ... address=` and firewall rules to the actual operator egress address or CIDR. Use a trusted certificate or put its issuing CA in a Secret. `insecureSkipVerify` exists only to make the first lab test easier. Plain HTTP sends the Basic Auth credentials without transport encryption and should not be used. The operator manages NAT only; it does not create RouterOS filter rules. The router's forward-chain policy must already allow dst-natted traffic. New rules are appended by RouterOS, so check that existing earlier NAT rules do not shadow them. Rule positioning is a good candidate for the next revision after testing against the actual router configuration. ## Install from the OCI Helm chart Gitea CI publishes the operator image and chart to the local Quay registry. A release tag such as `v0.1.0` produces these matched artifacts: - `registry.brunner.ninja/feedc0de/mikrotik-operator:0.1.0` - `oci://registry.brunner.ninja/feedc0de/charts/mikrotik-operator:0.1.0` Install that release directly from the registry: ```sh helm registry login registry.brunner.ninja helm upgrade --install mikrotik-operator \ oci://registry.brunner.ninja/feedc0de/charts/mikrotik-operator \ --version 0.1.0 \ --namespace mikrotik-operator-system \ --create-namespace ``` The chart installs the CRD, Deployment, ServiceAccount, and RBAC. It deliberately does not contain credential Secrets or `MikroTikRouter` instances. Keep those as separate cluster configuration. Helm installs CRDs from `crds/` but does not upgrade or delete them; apply a changed CRD explicitly before a future chart upgrade: ```sh kubectl apply -f config/crd/mikrotikrouters.yaml ``` For a local development build and chart install: ```sh make IMAGE_TAG=dev push make IMAGE_TAG=dev install ``` The original plain-manifest path remains available as `make install-manifests`. ## Gitea CI releases The workflow in `.gitea/workflows/ci.yml` runs Go race tests and vet, verifies that the standalone and chart CRDs are identical, lints and renders the chart, and builds the container on every push and pull request. Publishing runs only for `main` and `v*` tags: - `main` publishes the image under the full commit SHA and a development chart such as `0.0.0-dev.sha0123456789ab` whose `appVersion` is that SHA. - `v0.1.0` publishes image tag `0.1.0` and chart version `0.1.0`. The Deployment template defaults the image tag to `Chart.appVersion`. CI renders the packaged chart and verifies the exact expected image reference before pushing either artifact. Configure these Gitea repository Actions secrets: - `QUAY_USERNAME` - `QUAY_TOKEN` The Quay account or robot token needs push access to both `feedc0de/mikrotik-operator` and `feedc0de/charts/mikrotik-operator`. Make the repositories public if clusters should pull without an image pull Secret. Create credentials without committing them. This keeps the password itself out of shell history: ```sh read -rsp 'Router password: ' ROUTER_PASSWORD echo kubectl -n mikrotik-operator-system create secret generic newgw-credentials \ --from-literal=username=kubernetes-nat \ --from-literal=password="${ROUTER_PASSWORD}" unset ROUTER_PASSWORD ``` Edit and apply the examples: ```sh kubectl apply -f examples/router.yaml kubectl apply -f examples/service.yaml kubectl get mikrotikrouter newgw kubectl describe mikrotikrouter newgw kubectl -n mikrotik-operator-system logs deployment/mikrotik-operator -f ``` For a CA-verified self-signed setup, create a Secret with a `ca.crt` key and replace `insecureSkipVerify` in the router with: ```yaml tls: caSecretRef: namespace: mikrotik-operator-system name: router-ca ``` ## Current limits and next revisions - RouterOS destination NAT is IPv4-only; IPv6 LoadBalancer addresses are ignored. - There is no admission webhook yet. Validation errors are reported in the router's `Ready` condition while the previous rule is preserved. - Metrics use controller-runtime's standard endpoint; domain-specific counters and Kubernetes Events can be added once live reconciliation is proven. The chart remains intentionally small while the API is `v1alpha1`; new router and forwarding fields can evolve through `0.x` chart releases without treating the current contract as stable.