Deploy read-only Copyparty with guarded arschrock media storage

apps/copyparty/README.md

@@ -0,0 +1,61 @@
+# Komposthaufen file browser
+
+Copyparty replaces arschrock's host Apache directory listing at
+`https://brunner.ninja/komposthaufen/`. The maintained community chart from
+https://github.com/erNail/copyparty-chart owns the Deployment, configuration,
+Service, ServiceAccount and index PVC. No separate homelab chart repository or
+image build is needed. `bootstrap/copyparty.yaml` pins the published chart.
+The temporary primary-image tag override in `values.yaml` follows the root
+policy for a latest chart that trails its application. Remove it when the chart
+catches up; keep its official image repository unchanged.
+
+Anonymous users receive only `r` (read/list/download) permission. No account
+has upload, move, delete or admin permission. The non-root container also mounts
+`/komposthaufen` recursively read-only. Symlinks cannot escape the exported
+volume. HTML is served as text and embedded directory prologues are disabled.
+The Traefik route preserves the URL prefix; Copyparty's `rp-loc` handles it.
+Direct downloads, HEAD and HTTP byte ranges retain their normal URL shape.
+
+The writable RBD PVC contains rebuildable SQLite search indexes, thumbnails and
+session state. Indexing scans on startup and hourly; file contents are not hashed
+and media tag extraction is not enabled. Search covers path/name, size and date.
+The authoritative files stay on the existing bcachefs bind mount; nothing is
+copied, renamed, chowned or deleted. The current backup owner in `apps/backup`
+does not cover this host library. **The library has an existing backup coverage
+gap.** The disposable index needs no historical backup: recreate it and let
+Copyparty rescan. Preserve the host data independently; Ceph replication and
+its existing backup mirrors are not historical backups of this library.
+
+## Host storage maintenance
+
+`../../scripts/media-storage-{offline,online}.sh` are the shared Plex, Jellyfin,
+Copyparty and Paper maintenance helpers. The existing scripts at
+`~/kubernetes/media-storage-{offline,online}.sh` on arschrock forward to these
+versioned helpers. `~/unmount.sh --stop` uses `bcachefs-maintenance.py` to cordon,
+stop filesystem holders, invoke the offline helper and verify mount namespaces
+before unmounting. `~/mount_binds.sh` restores and checks the bind mounts, calls
+the online helper to recreate consumers and restore the storage-ready labels,
+and restores scheduling. Copyparty uses the same component selector as Plex
+and Jellyfin. Never tolerate the unschedulable taint or bypass the ready label.
+
+After changing GitOps, update arschrock's checkout before storage maintenance:
+`git -C ~/kubernetes/homelab-gitops pull --ff-only`.
+
+The chart has readiness/liveness hooks but no startup-probe template; liveness
+has a startup grace period and does not wait for the background index scan.
+Configuration changes require a Deployment restart because this chart does not
+hash its ConfigMap into the pod template.
+
+## Recovery and legacy Apache
+
+Reconcile the Argo Application and its retained PVC to recover the service.
+For an index-only failure, stop Copyparty before rebuilding its disposable
+index; never remove anything under `/komposthaufen` to repair search.
+Apache's host unit is `httpd.service`; it is disabled/stopped after cutover.
+Its configuration remains intact, but currently has a missing hiredis library
+required by mod_auth_openidc. An Apache rollback would first require repairing
+that dependency, then enabling the service and reverting only the komposthaufen
+route to `legacy-apache-service`. The `youtube.brunner.ninja` route is intentionally
+retained for a separate migration and remains unavailable while Apache is off.
+No old dedicated source repository exists for this host Apache service; remote
+repository archival is not applicable.

apps/copyparty/values.yaml

@@ -0,0 +1,106 @@
+# Latest chart still bundles an older application. Remove this tag-only
+# override when a newer chart supplies this release or later.
+image:
+  tag: "1.20.26"
+strategy:
+  type: Recreate
+extraPodLabels:
+  app.kubernetes.io/component: server
+nodeSelector:
+  kubernetes.io/hostname: arschrock
+  media.brunner.ninja/multimedia-ready: "true"
+tolerations:
+- key: node-role.kubernetes.io/control-plane
+  operator: Exists
+  effect: NoSchedule
+serviceAccount:
+  name: copyparty
+  automount: false
+containerSecurityContext:
+  allowPrivilegeEscalation: false
+  runAsNonRoot: true
+  capabilities:
+    drop: [ALL]
+podSecurityContext:
+  seccompProfile:
+    type: RuntimeDefault
+resources:
+  requests:
+    cpu: 100m
+    memory: 256Mi
+    ephemeral-storage: 128Mi
+  limits:
+    cpu: null
+    memory: 2Gi
+    ephemeral-storage: 1Gi
+app:
+  env:
+  - name: XDG_CONFIG_HOME
+    value: /state
+  args: []
+configMap:
+  data:
+    config.conf: |
+      [global]
+        p: 3923
+        http-only
+        rp-loc: /komposthaufen
+        xff-src: 10.0.0.0/16
+        xff-hdr: x-real-ip
+        rproxy: 1
+        hist: /state/hist
+        e2dsa
+        no-hash: .*
+        re-maxage: 3600
+        no-del
+        no-mv
+        unpost: 0
+        xvol
+        no-logues
+        no-readme
+        no-html
+      [/]
+        /komposthaufen
+        accs:
+          r: *
+volumeConfig:
+  appData:
+    volumeMount:
+      mountPath: /state
+    persistence:
+      enabled: true
+      pvcName: copyparty-index
+      size: 10Gi
+      storageClass: rook-ceph-block
+      volume:
+        persistentVolumeClaim:
+          claimName: copyparty-index
+  userData:
+    volumeMount:
+      mountPath: /komposthaufen
+      readOnly: true
+      recursiveReadOnly: Enabled
+    volume:
+      emptyDir: null
+      hostPath:
+        path: /komposthaufen
+        type: Directory
+  copypartyConfig:
+    volumeMount:
+      readOnly: true
+    volume:
+      secret: null
+      configMap:
+        name: copyparty-config
+readinessProbe:
+  httpGet:
+    path: /komposthaufen/?h
+  timeoutSeconds: 5
+livenessProbe:
+  httpGet:
+    path: /komposthaufen/?h
+    port: http
+  initialDelaySeconds: 90
+  periodSeconds: 30
+  timeoutSeconds: 10
+  failureThreshold: 5

bootstrap/copyparty-namespace.yaml

@@ -0,0 +1,6 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: copyparty
+  annotations:
+    argocd.argoproj.io/sync-wave: "-2"

bootstrap/copyparty.yaml

@@ -0,0 +1,26 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: copyparty
+  namespace: argocd
+spec:
+  project: media
+  sources:
+  - repoURL: ghcr.io/ernail/charts
+    chart: copyparty
+    targetRevision: 2.0.0
+    helm:
+      releaseName: copyparty
+      valueFiles:
+      - $values/apps/copyparty/values.yaml
+  - repoURL: https://code.brunner.ninja/feedc0de/homelab-gitops.git
+    targetRevision: main
+    ref: values
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: copyparty
+  syncPolicy:
+    automated:
+      enabled: true
+      prune: false
+      selfHeal: true

bootstrap/kustomization.yaml

@@ -153,3 +153,5 @@ resources:
 - oldipmi.yaml
 - descheduler-project.yaml
 - descheduler.yaml
+- copyparty-namespace.yaml
+- copyparty.yaml

bootstrap/media-project.yaml

@@ -4,16 +4,21 @@ metadata:
   name: media
   namespace: argocd
 spec:
-  description: Plex and Jellyfin media servers pinned to arschrock
+  description: Media and read-only file servers pinned to arschrock
   sourceRepos:
+  - ghcr.io/ernail/charts
   - https://brunner.ninja/charts
   - https://code.brunner.ninja/feedc0de/homelab-gitops.git
   destinations:
+  - namespace: copyparty
+    server: https://kubernetes.default.svc
   - namespace: plex
     server: https://kubernetes.default.svc
   - namespace: jellyfin
     server: https://kubernetes.default.svc
   namespaceResourceWhitelist:
+  - group: ""
+    kind: ServiceAccount
   - group: ""
     kind: ConfigMap
   - group: ""

scripts/media-storage-offline.sh

@@ -0,0 +1,78 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+readonly expected_context="kubernetes-admin@kubernetes"
+readonly node="arschrock"
+readonly dynmap_ready_label="minecraft.brunner.ninja/dynmap-ready"
+readonly library_services=(lidarr.service readarr.service radarr.service sonarr.service bazarr.service)
+readonly whisparr_service="whisparr.service"
+
+current_context="$(kubectl config current-context)"
+if [[ "${current_context}" != "${expected_context}" ]]; then
+  echo "Refusing to modify ${node}: context is ${current_context}, expected ${expected_context}." >&2
+  exit 1
+fi
+
+# These host-native library managers must never inspect the empty Btrfs
+# directory underneath /komposthaufen when its Bcachefs mount is offline.
+sudo -n systemctl stop "${library_services[@]}" "${whisparr_service}"
+for service in "${library_services[@]}" "${whisparr_service}"; do
+  if sudo -n systemctl is-active --quiet "${service}"; then
+    echo "${service} is still active; do not unmount /komposthaufen." >&2
+    exit 1
+  fi
+done
+
+kubectl get node "${node}" >/dev/null
+if ! kubectl label node "${node}" media.brunner.ninja/multimedia-ready- 2>/dev/null; then
+  labels=$(kubectl get node "${node}" --output=jsonpath='{.metadata.labels}')
+  if [[ "${labels}" == *"media.brunner.ninja/multimedia-ready"* ]]; then
+    echo "Could not remove the media-ready label from ${node}; do not unmount." >&2
+    exit 1
+  fi
+fi
+
+for namespace in plex jellyfin copyparty; do
+  scheduled_pods=$(kubectl --namespace "${namespace}" get pods \
+    --selector="app.kubernetes.io/component=server" \
+    --field-selector="spec.nodeName=${node}" \
+    --output=name)
+  if [[ -n "${scheduled_pods}" ]]; then
+    mapfile -t pod_names <<< "${scheduled_pods}"
+    kubectl --namespace "${namespace}" delete "${pod_names[@]}" --wait=false
+    for pod in "${pod_names[@]}"; do
+      kubectl --namespace "${namespace}" wait --for=delete "${pod}" --timeout=5m
+    done
+  fi
+done
+
+if ! kubectl label node "${node}" "${dynmap_ready_label}-" 2>/dev/null; then
+  labels=$(kubectl get node "${node}" --output=jsonpath='{.metadata.labels}')
+  if [[ "${labels}" == *"${dynmap_ready_label}"* ]]; then
+    echo "Could not remove the Dynmap-ready label from ${node}; do not unmount." >&2
+    exit 1
+  fi
+fi
+
+mapfile -t paper_pods < <(kubectl --namespace mcserver get pods \
+  --selector="app.kubernetes.io/name=minecraft-paper,app.kubernetes.io/instance=paper" \
+  --field-selector="spec.nodeName=${node}" --output=name)
+if ((${#paper_pods[@]})); then
+  kubectl --namespace mcserver delete "${paper_pods[@]}" --wait=false
+  for pod in "${paper_pods[@]}"; do
+    kubectl --namespace mcserver wait --for=delete "${pod}" --timeout=15m
+  done
+fi
+
+for namespace in plex jellyfin copyparty; do
+  remaining_pods=$(kubectl --namespace "${namespace}" get pods \
+    --selector="app.kubernetes.io/component=server" \
+    --field-selector="spec.nodeName=${node}" \
+    --output=name)
+  if [[ -n "${remaining_pods}" ]]; then
+    echo "A ${namespace} Pod is still assigned to ${node}; do not unmount." >&2
+    exit 1
+  fi
+done
+
+echo "Plex, Jellyfin, Copyparty and Paper are stopped on ${node}. Replacements stay Pending until their storage-ready labels return."

scripts/media-storage-online.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+readonly expected_context="kubernetes-admin@kubernetes"
+readonly node="arschrock"
+readonly media_path="/komposthaufen/multimedia"
+readonly dynmap_tiles_path="/home/feedc0de/mcserver/plugins/dynmap/web/tiles"
+readonly dynmap_ready_label="minecraft.brunner.ninja/dynmap-ready"
+readonly library_services=(lidarr.service readarr.service radarr.service sonarr.service bazarr.service)
+readonly whisparr_service="whisparr.service"
+readonly whisparr_library_path="/mnt/msdownld.tmp/_hardware/whisparr"
+
+current_context="$(kubectl config current-context)"
+if [[ "${current_context}" != "${expected_context}" ]]; then
+  echo "Refusing to modify ${node}: context is ${current_context}, expected ${expected_context}." >&2
+  exit 1
+fi
+
+if ! mountpoint -q /komposthaufen; then
+  echo "/komposthaufen is not a mountpoint; storage remains offline." >&2
+  exit 1
+fi
+if [[ "$(findmnt -T "${media_path}" -n -o FSTYPE)" != bcachefs ]]; then
+  echo "${media_path} is not backed by bcachefs; storage remains offline." >&2
+  exit 1
+fi
+if [[ ! -d "${media_path}/Videos" ]]; then
+  echo "Required directory ${media_path}/Videos is missing; storage remains offline." >&2
+  exit 1
+fi
+if ! mountpoint -q "${dynmap_tiles_path}" || [[ "$(findmnt -T "${dynmap_tiles_path}" -n -o FSTYPE)" != bcachefs ]]; then
+  echo "${dynmap_tiles_path} is not mounted from bcachefs; Paper remains stopped." >&2
+  exit 1
+fi
+
+for attempt in $(seq 1 60); do
+  if kubectl get node "${node}" >/dev/null 2>&1; then
+    # Stale readiness after a reboot may have mounted the empty system-disk
+    # directory. Drop both gates and recreate every hostPath consumer first.
+    kubectl label node "${node}" media.brunner.ninja/multimedia-ready- 2>/dev/null || true
+    if ! kubectl label node "${node}" "${dynmap_ready_label}-" 2>/dev/null; then
+      labels="$(kubectl get node "${node}" --output=jsonpath='{.metadata.labels}')"
+      if [[ "${labels}" == *"${dynmap_ready_label}"* ]]; then
+        echo "Could not clear the stale Dynmap-ready label on ${node}." >&2
+        exit 1
+      fi
+    fi
+    for namespace in plex jellyfin copyparty; do
+      mapfile -t pods < <(
+        kubectl --namespace "${namespace}" get pods \
+          --selector="app.kubernetes.io/component=server" \
+          --output=name 2>/dev/null || true
+      )
+      if ((${#pods[@]})); then
+        kubectl --namespace "${namespace}" delete "${pods[@]}" --wait=false
+        for pod in "${pods[@]}"; do
+          kubectl --namespace "${namespace}" wait --for=delete "${pod}" --timeout=5m
+        done
+      fi
+    done
+    mapfile -t paper_pods < <(
+      kubectl --namespace mcserver get pods \
+        --selector="app.kubernetes.io/name=minecraft-paper,app.kubernetes.io/instance=paper" \
+        --output=name 2>/dev/null || true
+    )
+    if ((${#paper_pods[@]})); then
+      kubectl --namespace mcserver delete "${paper_pods[@]}" --wait=false
+      for pod in "${paper_pods[@]}"; do
+        kubectl --namespace mcserver wait --for=delete "${pod}" --timeout=15m
+      done
+    fi
+    kubectl label node "${node}" media.brunner.ninja/multimedia-ready=true --overwrite
+    kubectl label node "${node}" "${dynmap_ready_label}=true" --overwrite
+    sudo -n systemctl start "${library_services[@]}"
+    if mountpoint -q /mnt \
+      && [[ "$(findmnt -T /mnt -n -o FSTYPE)" == bcachefs ]] \
+      && [[ -d "${whisparr_library_path}" ]]; then
+      sudo -n systemctl start "${whisparr_service}"
+    else
+      echo "Whisparr remains stopped; its /mnt Bcachefs library is not ready."
+    fi
+    echo "${node} is media-ready; Plex, Jellyfin, Copyparty and Paper may now be scheduled."
+    exit 0
+  fi
+  sleep 2
+done
+
+echo "Kubernetes API did not become available; ${node} remains media-offline." >&2
+exit 1