homelab-gitops/docs/memory-sizing.md

Memory request review

Review on 2026-11-04

This uses the CPU review's p95 plus 20% headroom method for memory requests. Prometheus was queried over the previous 14 days, sampled every 15 minutes:

quantile_over_time(0.95,
  (max by(namespace,pod,container)
    (container_memory_working_set_bytes{container!="",container!="POD"})
  )[14d:15m])

The largest matching replica/pod-history p95 is used for each current workload container, including previous ReplicaSets. The inner max deduplicates scrape series; it does not add replicas together. History coverage was checked using count_over_time on the same subquery. The longest series covers about 13.8 days; individual replacement pods have shorter histories. Working set is a practical scheduling estimate, not a guaranteed upper bound on allocation or peak demand.

Requests are rounded upward to a 64 MiB boundary after multiplying by 1.2. Existing higher requests and startup/recovery floors are retained. CPU values, memory limits, images, replicas, storage and placement rules are unchanged. Requests account for scheduling capacity; they do not cap memory consumption. The scheduler considers init-container requirements as well as normal containers.

The accompanying memory-sizing-2026-11-04.csv is a dated measurement and decision record, not an install-time configuration source. Owning values files remain canonical. It records both changes and exclusions rather than claiming every container was automatically resized.

Scope and exclusions

The GitOps changes cover 49 measured container settings across 30 chart applications. Some chart settings also affect init containers; the Prometheus operator's config-reloader request applies to both Prometheus and Alertmanager. The main gaps include Quay, Immich, Paperless, Prometheus, Argo CD, and services with no previous memory reservation. InfluxDB retains its existing request, which is above the history-based recommendation.

The legacy Matrix authentication Helm release has four corresponding request increases in matrix/authentication/values.yaml in the homelab workspace. It requires its existing Helm installation path; it is not managed by this repo. Unchanged workers retain their shared resource anchor and existing floors.

• Ceph daemons and CSI are excluded to avoid an unreviewed storage-wide rollout.
• Network/control-plane components require a coordinated capacity review.
• Operator-owned databases, game/GPU servers and bursty jobs retain their reservations; idle measurements are not sufficient peak-demand evidence.
• Copyparty has very short history dominated by OOM failures under its former limit. Review again after representative operation with the new limit.
• Recommendations above existing memory limits are recorded for investigation, not silently capped or paired with unsolicited limit changes.
• Diun's published chart declares a resources value but does not render it. Loops/Pixelfed also need chart-interface work for hard-coded resources. Unsupported values have not been added.
• Small existing reservations and unrelated legacy deployments are retained.

Validation and rollout

Render and lint the exact pinned charts using the same values as Argo CD. Compare resources before/after and assert intended requests are actually rendered; check the Prometheus operator's config-reloader argument separately. Validate changed workloads with server-side dry-run. Do not delete PVCs or force-delete stateful pods to make a new reservation fit.

At preparation time intel-xeon was unreachable and its Argo application controller pod was terminating. Helm rendering does not establish successful deployment. After controller recovery, verify Argo sync/health, pending pods, node memory and request totals, application responses and unchanged PVC volumes. If a workload cannot fit, review capacity/placement rather than reducing its request merely to hide the scheduling failure.