Files
wolfssl/.github/workflows/check-source-text.yml
T
Tobias Frauenschläger 641c39dbf3 CI: catch workflows that GitHub silently fails to load
A workflow file GitHub cannot load does not fail loudly. Its runs end
within 0s with zero jobs, no logs, no annotations and no check runs, and
the workflow re-registers under its bare path instead of its `name:`
field. Among the few hundred checks on a PR that reads as unrelated
flake, so the coverage just disappears: os-check.yml was in this state on
master for ten days in July 2026 before anyone noticed, and no open PR
reported a problem the whole time.

Add two guards.

Pre-merge, check-workflows.py measures every `run:` step against
GitHub's 21000 character cap and fails the build past it, with a warning
from 18000 so a growing step is noticed while there is still runway.
Sizes come from the parsed YAML, which is what the Actions service
evaluates, so block-scalar indentation needs no guessing. It runs from
check-source-text.yml over every workflow and composite action rather
than only PR-changed files: the cap applies per file, the whole sweep
takes well under a second, and a file can be pushed over the line by a
change elsewhere in the PR. Note that this cap is enforced by the
service and not by the workflow schema, so neither a YAML validator nor
actionlint reports it.

Post-merge, workflow-health.yml runs check-workflow-health.py daily and
looks for the symptom rather than any particular cause, so a workflow
that stops loading for a reason nobody anticipated is still caught. Two
signals: an active workflow whose registered name equals its path, and a
completed run that failed with zero jobs (prefiltered on
created_at == updated_at, so only a handful need a jobs lookup). Against
the live repository the first signal flags os-check.yml and nothing else
across 107 workflows, and reports clean on wolfTPM and wolfMQTT. It
exits 1 on a finding and 2 when the check could not be carried out at
all, because a missing token and a broken workflow call for different
responses.

Findings go into a single reused issue rather than another red check
that would blend into the noise: the body is rewritten on each run, a
comment is posted only when the set of affected workflows changes, and
the issue closes itself once everything loads again.

Finding that issue reliably turned out to be the fiddly part, and the
approach here is the one that survived testing against a live
repository. The issue is identified by both a dedicated label and its
title, and looked up through the REST issues endpoint. Both halves of
that identity matter: the label alone is a normal repository label that
anyone can apply, and an adopted issue has its body overwritten and is
then closed, so matching on the label alone would destroy a mislabelled
issue. Searching by title instead is unusable, because search ignores
--state and returns closed issues, which had the monitor re-closing an
already closed issue on every clean run. `gh issue list` reads a GraphQL
replica that can lag. The REST endpoint lags too, by about 2.4s for a
newly created issue, so the lookup re-checks a few times before
concluding nothing is open - without that, consecutive runs each open a
duplicate, and a clean run right after an outage fails to close the
issue it just opened.

Verified against the commit that caused the outage: check-workflows.py
fails on acff4d62a (21813 characters) and passes on its parent
f5ace71dd, which it flags at 20662 - already inside the warning band,
338 characters short of breaking. The full issue lifecycle (open,
repeat with no comment, comment on change, close, stay closed, reopen a
fresh issue for a new outage) was exercised end to end against a live
repository.
2026-08-03 17:05:29 +02:00

123 lines
4.5 KiB
YAML

name: Check Source Text
# Source-hygiene + shell-script lint. Runs on drafts too - fast feedback.
#
# Checks:
# * check-source-text.sh: trailing whitespace, hard tabs in C/H, CRLF,
# BOM / non-ASCII.
# * bash -n + shellcheck (warning level) on shell scripts.
# * check-workflows.py: every `run:` step against GitHub's 21000
# character cap, past which GitHub stops loading the workflow file
# altogether and its runs fail in 0s with zero jobs.
#
# Scope:
# * pull_request: only files changed in the PR (catches new violations
# without failing on historical debt).
# * push: scan the full tree (baseline guard on master).
on:
push:
branches: [ master, main ]
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
branches: [ master, main ]
concurrency:
group: check-source-text-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
check:
# Only run from the wolfssl org to avoid burning forks' CI minutes.
if: github.repository_owner == 'wolfssl'
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- uses: actions/checkout@v5
with:
fetch-depth: 0
# python3-yaml backs check-workflows.py, which measures run: steps
# from the parsed YAML rather than from the raw text.
- name: Install shellcheck
uses: ./.github/actions/install-apt-deps
with:
packages: shellcheck python3-yaml
ghcr-debs-tag: ubuntu-24.04-full
- name: Collect files to check
id: files
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
BASE_SHA="${{ github.event.pull_request.base.sha }}"
HEAD_SHA="${{ github.event.pull_request.head.sha }}"
git diff --name-only --diff-filter=ACMR "$BASE_SHA" "$HEAD_SHA" \
> changed.txt || true
grep -E '\.sh$' changed.txt > changed-sh.txt || true
echo "Files changed in PR:"
cat changed.txt
echo "Shell scripts changed:"
cat changed-sh.txt
echo "count=$(wc -l < changed.txt)" >> "$GITHUB_OUTPUT"
echo "sh_count=$(wc -l < changed-sh.txt)" >> "$GITHUB_OUTPUT"
else
: > changed.txt
git ls-files '*.sh' > changed-sh.txt
echo "count=0" >> "$GITHUB_OUTPUT"
echo "sh_count=$(wc -l < changed-sh.txt)" >> "$GITHUB_OUTPUT"
fi
# Always over the whole set, not just PR-changed files: the cap is a
# property of each file on its own, the check takes well under a
# second for the ~110 of them, and a workflow can be pushed over the
# line by a change to a file the PR does not otherwise touch.
- name: Lint workflow files
run: ./.github/scripts/check-workflows.py
- name: Run check-source-text (PR changed files)
if: github.event_name == 'pull_request' && steps.files.outputs.count != '0'
run: |
# shellcheck disable=SC2046
./.github/scripts/check-source-text.sh $(cat changed.txt)
- name: Run check-source-text (full tree)
if: github.event_name != 'pull_request'
run: ./.github/scripts/check-source-text.sh
- name: bash -n (syntax check)
if: steps.files.outputs.sh_count != '0'
run: |
fail=0
while IFS= read -r f; do
[ -f "$f" ] || continue
if ! bash -n "$f"; then
echo "::error file=$f::bash -n syntax error"
fail=1
fi
done < changed-sh.txt
exit "$fail"
- name: shellcheck (warning level)
if: steps.files.outputs.sh_count != '0'
run: |
# Mirrors the internal multi-test check-shell-scripts subtest:
# --severity=warning
# -e SC2226,SC2166,SC2164,SC2046,SC2034,SC2188,SC2043
# SC2226 (no ln destination), SC2166 ([ p -a q ]), SC2164 (cd ||),
# SC2046 (word splitting), SC2034 (unused var), SC2188 (redirect
# w/o command), SC2043 (loop runs once) - common in this codebase,
# suppressed in the internal multi-test for the same reason.
fail=0
while IFS= read -r f; do
[ -f "$f" ] || continue
if ! shellcheck --severity=warning \
--exclude=SC2226,SC2166,SC2164,SC2046,SC2034,SC2188,SC2043 \
--format=gcc "$f"; then
fail=1
fi
done < changed-sh.txt
exit "$fail"