Install collectors with symlinks and share paste_log helper

README.md

@@ -2,9 +2,56 @@
 
 Telegraf bcachefs monitoring
 
-Add the following config section to your telegraf.yml:
+## Installation
 
+Run once from the checkout:
+
+```sh
+sudo ./install.sh
 ```
+
+This links `/etc/telegraf/bcachefs*.sh` to this checkout. Existing files are
+preserved in a printed `/etc/telegraf/bcachefs-backup.*` directory. Re-running
+is safe; unchanged links are skipped. You can pass a different destination
+directory as the first argument. Keep the checkout at the same path and readable
+and executable by the Telegraf service. Edits take effect on the next collection;
+no copy or service restart is needed. To roll back, replace the links with the
+files in the backup directory (remove any newly installed links separately).
+
+On the original host, Telegraf runs as root with `ProtectHome=no`. Thus edits
+here become code executed as root. On another host, check service permissions
+and sandbox settings before installing into a home directory.
+
+The original host loads its configuration from InfluxDB using `TELEGRAF_OPTS`
+and `INFLUX_TOKEN` in `/etc/default/telegraf`. Keep these credentials outside Git.
+The existing hosted commands can remain unchanged with this installation;
+a new systemd environment variable is unnecessary.
+
+## Collectors
+
+The scripts retain the deployed host's `/mnt` mount point and, in several cases,
+its filesystem UUID. Adapt those values before using them on another host.
+`bcachefs_reconcile.sh` already accepts `FSPATH` (or a positional mount path) and
+`BCACHEFS_UUID` overrides.
+
+- `bcachefs.sh`: filesystem usage; imported version selects usage sections explicitly.
+- `bcachefs_fs_top.sh`: parses the collector's `/fs_top` endpoint (`/s` format).
+- `bcachefs_options.sh`: sysfs options.
+- `bcachefs_perf.sh`: collector's `/perf_json` endpoint.
+- `bcachefs_reconcile.sh`: reconcile status.
+- `bcachefs_snapshot_delete_status.sh`: snapshot deletion progress.
+- `bcachefs_time_stats.sh`: sysfs time statistics.
+- `bcachefs_timestats.sh`: JSON time statistics from bcachefs-tools; different measurement/schema.
+- `bcachefs_device_time_stats.sh`: compatibility symlink to `bcachefs_time_stats.sh`;
+  the old workspace file was identical and did not collect extra device metrics.
+  Do not configure both aliases, as that would duplicate collection.
+
+The old `_not_finished` human-readable prototype was removed; Git retains its
+history. `_not_maintained_anymore` remains as an archive.
+
+Add the desired inputs to your Telegraf TOML configuration (local or hosted):
+
+```toml
 [[inputs.exec]]
   commands = ["/etc/telegraf/bcachefs.sh"]
   data_format = "influx"
@@ -25,3 +72,44 @@ Add the following config section to your telegraf.yml:
 ![example disks table](disks-table.png)
 ![example disks graphs](disks-graphs.png)
 ![example disks states](disks-states.png)
+
+## Support reports and pastes
+
+Run `./kent_report.sh` from the directory where you want `kent_report.log` and
+trace output saved. It performs the existing lengthy diagnostics and uploads
+the report to the paste server, printing a URL for manually sharing on IRC.
+It does not send an IRC message. The diagnostics retain the original host's
+Arch packages, bcachefs source checkout, device and encryption-file paths;
+review those and the collected information before running on another host.
+
+`paste_log.sh` contains the shared function and can also be run directly:
+
+```sh
+./paste_log.sh path/to/log
+```
+
+To make `paste_log` available in interactive Bash, add:
+
+```bash
+source "$HOME/telegraf-bcachefs-input/paste_log.sh"
+```
+
+`kent_report.sh` sources the same file relative to its own location; it does not
+rely on interactive shell functions being exported. Uploads use the current
+kubectl context/namespace and require exactly one pod named `brunner-ninja*`.
+Failed uploads return an error without printing a misleading URL. Temporary
+name reservations are cleaned up, and function errors do not exit your shell.
+
+## Building the performance collector
+
+Requires Qt 6 Core, Network, WebSockets and HttpServer development packages:
+
+```sh
+mkdir -p /tmp/bcachefs-collector-build
+cd /tmp/bcachefs-collector-build
+qmake6 /path/to/telegraf-bcachefs-input/perf-collector/perf-collector.pro
+make
+```
+
+The collector listens on localhost:8000 and launches `bcachefs fs top /mnt`
+and `sudo perf trace`. Building does not deploy or restart it.

install.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+# Keep existing Telegraf command paths pointing at this checkout.
+set -euo pipefail
+repo=$(cd -- "$(dirname -- "$(readlink -f -- "${BASH_SOURCE[0]}")")" && pwd -P)
+target=${1:-/etc/telegraf}
+[[ $# -le 1 ]] || { echo "Usage: $0 [telegraf-directory]" >&2; exit 2; }
+[[ -d $target && -w $target ]] || {
+    echo "Directory must exist and be writable: $target (use sudo for /etc/telegraf)" >&2
+    exit 1
+}
+backup=
+staging=$(mktemp -d "$target/.bcachefs-install.XXXXXXXX")
+trap 'rm -rf -- "$staging"' EXIT
+for source in "$repo"/bcachefs*.sh; do
+    name=${source##*/}
+    destination=$target/$name
+    [[ -x $source ]] || { echo "Not executable: $source" >&2; exit 1; }
+    if [[ -L $destination && $(readlink -- "$destination") == "$source" ]]; then
+        continue
+    fi
+    [[ ! -d $destination ]] || { echo "Refusing to replace directory: $destination" >&2; exit 1; }
+    if [[ -e $destination || -L $destination ]]; then
+        if [[ -z $backup ]]; then
+            backup=$(mktemp -d "$target/bcachefs-backup.XXXXXXXX")
+            echo "Backup: $backup"
+        fi
+        cp -a -- "$destination" "$backup/$name"
+    fi
+    ln -s -- "$source" "$staging/$name"
+    mv -Tf -- "$staging/$name" "$destination"
+    echo "$destination -> $source"
+done

kent_report.sh

@@ -1,22 +1,9 @@
 #!/bin/bash
 #set -euo pipefail
 
-function paste_log()
-{
-    if [[ -z "$1" ]] ; then echo "Usage: $0 <filename>" ; exit 1 ; fi
-    if [[ ! -f "$1" ]] ; then echo "Usage: $0 <filename>" ; echo ; echo "File does not exist!" ; exit 2 ; fi
-
-    POD_NAME="$(kubectl get pods | grep -Po 'brunner-ninja[^ ]+')"
-    if [[ -z "$POD_NAME" ]]
-    then
-        echo no pod
-        exit 1
-    fi
-
-    NAME="$(mktemp XXXXXXXX.log)"
-    kubectl cp "$1" "${POD_NAME}:/srv/http/pastes/$NAME"
-    echo https://brunner.ninja/pastes/$NAME
-}
+# Resolve the checkout even when invoked through a symlink.
+REPORT_DIR=$(dirname -- "$(readlink -f -- "${BASH_SOURCE[0]}")")
+source "$REPORT_DIR/paste_log.sh" || exit 1
 
 (
 

paste_log.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+# Source this file for paste_log(), or execute it with a filename.
+paste_log()
+{
+    if [[ $# -ne 1 || ! -f ${1:-} ]]; then
+        echo 'Usage: paste_log <existing-file>' >&2
+        return 2
+    fi
+
+    local pods pod name temporary status
+    pods=$(kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}') || return
+    local -a matches=()
+    while IFS= read -r pod; do
+        [[ $pod == brunner-ninja* ]] && matches+=("$pod")
+    done <<< "$pods"
+    if [[ ${#matches[@]} -ne 1 ]]; then
+        echo "Expected one brunner-ninja pod, found ${#matches[@]}" >&2
+        return 1
+    fi
+
+    temporary=$(mktemp "${TMPDIR:-/tmp}/XXXXXXXXXXXX.log") || return
+    name=${temporary##*/}
+    # Resolve local paths so kubectl does not interpret a colon as a pod path.
+    local source
+    source=$(realpath -- "$1") || { rm -f -- "$temporary"; return 1; }
+    if kubectl cp "$source" "${matches[0]}:/srv/http/pastes/$name"; then
+        status=0
+    else
+        status=$?
+    fi
+    rm -f -- "$temporary"
+    ((status == 0)) || return "$status"
+    printf 'https://brunner.ninja/pastes/%s\n' "$name"
+}
+
+if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
+    paste_log "$@"
+fi

tests/test_helpers.py

@@ -0,0 +1,29 @@
+import os, pathlib, subprocess, tempfile
+repo=pathlib.Path(__file__).resolve().parents[1]
+with tempfile.TemporaryDirectory() as tmp:
+ p=pathlib.Path(tmp); target=p/'telegraf'; target.mkdir()
+ (target/'bcachefs.sh').write_text('original')
+ subprocess.run([str(repo/'install.sh'),str(target)],check=True,stdout=subprocess.DEVNULL)
+ assert (next(target.glob('bcachefs-backup.*'))/'bcachefs.sh').read_text()=='original'
+ for script in repo.glob('bcachefs*.sh'): assert (target/script.name).resolve()==script.resolve()
+ subprocess.run([str(repo/'install.sh'),str(target)],check=True)
+ assert len(list(target.glob('bcachefs-backup.*')))==1
+ stub=p/'kubectl'; stub.write_text('''#!/bin/bash
+if [[ $1 == get ]]; then
+ printf '%s\\n' "${MOCK_PODS-brunner-ninja-test}"
+else
+ exit "${MOCK_CP_STATUS:-0}"
+fi
+'''); stub.chmod(0o755)
+ log=p/'test log'; log.write_text('test')
+ env=dict(os.environ,PATH=tmp+':'+os.environ['PATH'],TMPDIR=tmp)
+ def run(extra={}):
+  return subprocess.run([str(repo/'paste_log.sh'),str(log)],env=env|extra,text=True,capture_output=True)
+ result=run(); assert result.returncode==0 and result.stdout.startswith('https://brunner.ninja/pastes/'),result
+ result=run({'MOCK_CP_STATUS':'7'}); assert result.returncode==7 and not result.stdout,result
+ for pods in ['', 'brunner-ninja-a\nbrunner-ninja-b']:
+  result=run({'MOCK_PODS':pods}); assert result.returncode==1 and not result.stdout,result
+ assert not list(p.glob('*.log'))
+ result=subprocess.run(['bash','-c','source "$1"; paste_log /nonexistent; echo survived','bash',str(repo/'paste_log.sh')],capture_output=True,text=True)
+ assert result.returncode==0 and result.stdout.strip()=='survived'
+ print('PASS: installer backup/idempotency, upload success/failure, pod selection, cleanup, sourced-shell survival')