From 5572d60a751b0a98b9943fd4bad4eccb18745a33 Mon Sep 17 00:00:00 2001 From: 0xFEEDC0DE64 Date: Wed, 28 Oct 2026 14:11:19 +0100 Subject: [PATCH] Resume Paperless with persistent broker and staging storage --- apps/paperless/README.md | 40 +++++++++++++++++++++++++-------- apps/paperless/values.yaml | 2 +- bootstrap/paperless-valkey.yaml | 4 ++-- 3 files changed, 34 insertions(+), 12 deletions(-) diff --git a/apps/paperless/README.md b/apps/paperless/README.md index ac61d76..f614ddb 100644 --- a/apps/paperless/README.md +++ b/apps/paperless/README.md @@ -13,15 +13,37 @@ The `paperless-storage` Application owns the pre-existing `paperless-data` and recreates them. The scanner action copies scans to the `consume` subdirectory of `paperless-data`; it does not use email or a recipient address. -PostgreSQL currently remains the existing manual Helm release `postgresql` -at chart 1.6.8 with the PostgreSQL 17.11 image. Its Argo declaration is staged -but intentionally not activated: a server-side dry-run of the current chart -against the live StatefulSet reported an immutable-spec/field-ownership -conflict. Do not delete or recreate its StatefulSet or PVC to force adoption. -The `postgres-data-postgresql-0` PVC and `paperless-postgres` Secret must be -preserved. Valkey remains the existing ephemeral Deployment for now; its -upstream chart changes the Deployment selector, so its staged Application is -also intentionally not activated. Resolve these as separate, verified cutovers. +PostgreSQL is the dedicated CloudNativePG cluster `paperless-cnpg`, reached +through the existing `postgresql` Service alias. Preserve its PVC and the +retired `postgres-data-postgresql-0` rollback claim. + +`paperless-valkey` now owns the upstream Valkey chart and retains the `valkey` +Service address. Its AOF is stored in `paperless-data/valkey` with every-second +fsync. Paperless stages uploads and PDF edits in `paperless-data/scratch`, not +an ephemeral container filesystem. Both paths belong to the existing CephFS +backup selection. The scheduled file mirror is not an atomic queue snapshot; +for queue recovery, quiesce Paperless and capture a consistent Valkey RDB as +well as the staged files, database and application Secret. Do not call an +unmounted `/mnt/backup` a successful backup destination. + +Paperless has no hostPath or dependency on `/mnt` or `/komposthaufen`. Its +preference for `arschrock` is soft; a cordon permits scheduling on another node. +Four import workers limit database connection bursts. Drain active/reserved +Celery tasks before replacements: durable storage does not replay every task +that was already acknowledged by a worker. + +### Stale task history (2026-10-28) + +Seventy old in-progress imports already had real documents. Original SHA-256, +OCR content and thumbnail checks were used before reconciling their results; +do not blindly mark pending tasks successful or re-upload duplicates. Historical +failures were retained and acknowledged only after verifying successful retries +or complete replacement splits. Two old bulk metadata jobs lacked their broker +payloads and were cancelled, not falsely reported successful. + +For a future incident, compare task records with Celery active/reserved work, +broker queues, staged files and existing documents before retrying. Back up +the database and task records first; never delete the task list to hide errors. `paperless-secret-key`, `paperless-postgres`, and `paperless-oidc` remain cluster-managed Secrets referenced by name. They are not copied into this diff --git a/apps/paperless/values.yaml b/apps/paperless/values.yaml index 7bce410..283641f 100644 --- a/apps/paperless/values.yaml +++ b/apps/paperless/values.yaml @@ -1,4 +1,4 @@ -replicaCount: 0 +replicaCount: 1 url: https://paper.brunner.ninja timeZone: Europe/Vienna ocrLanguage: deu+eng diff --git a/bootstrap/paperless-valkey.yaml b/bootstrap/paperless-valkey.yaml index f9159a5..975c9d0 100644 --- a/bootstrap/paperless-valkey.yaml +++ b/bootstrap/paperless-valkey.yaml @@ -20,6 +20,6 @@ spec: namespace: paperless-ngx syncPolicy: automated: - enabled: false + enabled: true prune: false - selfHeal: false + selfHeal: true