Elastic network-observability stack

Elasticsearch, Kibana, a dedicated MikroTik IPFIX collector, and a Traefik access-log collector managed by Elastic Cloud on Kubernetes (ECK). Everything runs in the elastic namespace.

Architecture

MikroTik newgw -- IPFIX/UDP 2055 --> MetalLB 192.168.0.20
                                     |
                                     v
                              Filebeat NetFlow module
                                     |
                            GeoIP, ASN, ECS and 30-day ILM
                                     |
                                     v
                         Elasticsearch <--> Kibana Maps

Traefik JSON access logs --> Elastic Agent on each node
                              |
                    request path, real client IP,
                    GeoIP, /16 + /24 and response bytes
                              |
                              v
                         Elasticsearch

• ECK operator 3.5.0, restricted to managing the elastic namespace.
• Elasticsearch 9.5.2: three nodes, 12 GiB memory each, and three expandable 200 GiB rook-ceph-block volumes.
• Kibana 9.5.2: two 4 GiB replicas at https://kibana.brunner.ninja.
• Filebeat 9.5.2: one stateful UDP collector. Keeping a single collector ensures all IPFIX templates and their flow packets reach the same process.
• Elastic Agent 9.5.2: one pod per Traefik node, reading container access logs into the logs-traefik.access-default data stream.
• All Stack components use Elastic's hardened Wolfi images. The standard UBI 10 images require the x86-64-v3 CPU baseline, which the Odroid H3 CPUs do not provide; the Wolfi images were verified on an Odroid before deployment.
• Traefik serves Kibana with the shared *.brunner.ninja certificate from its default TLSStore. Elasticsearch transport and HTTP certificates remain ECK-managed and private.
• Flow indices are deleted after 30 days. Traefik access indices roll over daily or at 30 GiB per primary shard and are deleted after 90 days.

Install

From a host with cluster-admin kubectl and Helm access:

./install.sh
./configure-router.sh
./check.sh

From the current remote location, copy the directory to dahoam, deploy from there, and configure the router through the SSH jump host:

rsync -av elastic/ dahoam:elastic/
ssh dahoam 'cd elastic && ./install.sh'
JUMP_HOST=dahoam \
ROUTER_HOST=admin@192.168.0.254 \
ROUTER_HOST_KEY_ALIAS=newgw \
./configure-router.sh

The router exports IPFIX only on the oja Internet-facing interface. Both WAN directions are retained, while inter-VLAN and collector-to-router LAN traffic are excluded. Override TRAFFIC_FLOW_INTERFACES only if the WAN interface is renamed.

NetFlow is UDP telemetry on the trusted LAN and does not use TLS. Kibana uses public HTTPS; Elasticsearch has no Ingress or LoadBalancer.

Sign in and view maps

Open https://kibana.brunner.ninja. The ingress currently uses Kibana's native authentication; Authentik ForwardAuth is not enabled.

Use username elastic. Retrieve its ECK-generated password without writing it to a file:

kubectl get secret elasticsearch-es-elastic-user \
  --namespace elastic \
  --output=go-template='{{.data.elastic | base64decode}}{{"\n"}}'

Filebeat loads the supplied NetFlow dashboards automatically. The installer also imports Elastic's official [Logs Traefik] Access logs dashboard and the local Traefik Traffic by Client, Subnet and Path dashboard. In Kibana open Analytics -> Dashboards and search for NetFlow. Useful starting points include:

• [Filebeat NetFlow] Overview
• [Filebeat NetFlow] Geo Location
• [Filebeat NetFlow] Autonomous Systems
• [Filebeat NetFlow] Conversation Partners
• [Filebeat NetFlow] Traffic Analysis

Search for Traefik to open the access-log dashboards. The local dashboard ranks Internet response bytes by exact client, /24, /16, and requested path; private cluster/LAN probes are filtered out. The documents include source.ip, source.geo.location, source.network_16, source.network_24, url.path, and http.response.body.bytes, which support maps and traffic totals per client, subnet, URL, country, or ASN. Query strings are deliberately not logged because they frequently contain tokens.

Run ./check.sh to inspect readiness, recent collector logs, and the number of stored netflow.log events.

Gitea Actions CD

The ECK operator and initial resources must be bootstrapped once with install.sh. CI subsequently updates only the committed stack resources; it cannot read Secrets, modify RBAC, or change the operator.

Create the restricted deployer and generate its kubeconfig:

KUBECTL_SSH_HOST=dahoam \
SSH_CONFIG="$HOME/.ssh/config" \
./create-ci-kubeconfig.sh

Create one repository Actions secret:

• KUBE_CONFIG_BASE64: the single base64 line printed by the command above.

No Elasticsearch, Kibana, router, DNS, or certificate secret belongs in Gitea. ECK and cert-manager create and rotate the live credentials inside the cluster.

Removal

This destroys the indices, Kibana saved objects, credentials, and all three PVCs:

./uninstall.sh --yes

The script removes the Helm release and namespace but intentionally retains the cluster-scoped ECK CRDs. Removing CRDs is unsafe if another ECK deployment is ever added to the cluster.