Commit Graph

10 Commits

Author SHA1 Message Date
Juliusz Sosinowicz 634ac9b6da CI: align branch-introduced actions with master's Node.js 24 bump
Rebasing onto master (which migrated JS actions to Node.js 24 runtimes)
left a few action refs that this branch added in new steps still on the
old major versions. Bring them in line with master:

- ccache-setup read-only restore:   actions/cache/restore@v4 -> @v5
- smoke-test / os-check ccache save: actions/cache/save@v4    -> @v5
- ci-deps-image checkout:            actions/checkout@v4       -> @v5
2026-06-15 22:39:56 +00:00
Juliusz Sosinowicz b8c008f3ac CI: address Skoll review (reseed coverage, ghcr owner, restore key)
- os-check.yml linux shard: add a schedule-gated CCACHE_RECACHE=1 step so
  the weekday seed reseeds from clean compiles rather than only accumulating
  deltas. This shard manages ccache directly (its own restore/save) and so
  was not covered by the ccache-setup composite's reseed.
- install-apt-deps: hardcode the ghcr bundle owner to wolfssl. The bundle is
  only published under ghcr.io/wolfssl by ci-deps-image, so fork PRs now read
  the public upstream image instead of a nonexistent ghcr.io/<fork>/wolfssl-ci-debs.
- ccache-setup: document that the read-only restore key reuses the save
  key shape for symmetry and is never an exact hit by design.

Skoll F3 (a packages-subset-of-bundle CI guard) is deferred to a follow-up;
F4 (release-branch ccache saves) is left as the intended seed-on-schedule /
everything-else-reads model.
2026-06-15 22:36:35 +00:00
Juliusz Sosinowicz 93b1e35a15 CI: address Copilot review (ccache-setup if:, smoke-test concurrency)
- ccache-setup: gate the scheduled-reseed step with
  `if: github.event_name == 'schedule'` again. The github context IS
  available in a composite action's step-level if: (install-apt-deps
  already relies on it), so the earlier $GITHUB_EVENT_NAME workaround and
  its comment were based on a wrong premise. The real load failure was the
  ${{ }} expression in the read-only input description, fixed separately.

- smoke-test.yml: include github.event_name in the concurrency group. The
  workflow pushes to master/main and now also runs on a weekday schedule;
  both share github.ref on the default branch, so under
  cancel-in-progress a seed run and a master push could cancel each other.
2026-06-15 22:36:35 +00:00
Juliusz Sosinowicz 86e5df754e CI: drop github context from ccache-setup input description
The read-only input description embedded `${{ github.event_name ==
'pull_request' }}` as example text. GitHub validates ${{ }} expressions in
an action's input definitions at manifest-load time, where the github
context is not available, so the action failed to load ("Unrecognized
named-value: 'github'", action.yml line 27) and every ccache-setup
consumer died at "Set up ccache" (build library, make check, Compiler
test, Multi-arch test, ...). Describe the expression in prose instead of
embedding it as a live ${{ }} template; the github.* references that
remain are in step with:/run: blocks, where the context is available.
2026-06-15 22:36:35 +00:00
Juliusz Sosinowicz 8754aee447 CI: fix ccache-setup load failure (github context in composite if:)
The scheduled-reseed step gated CCACHE_RECACHE with
`if: github.event_name == 'schedule'`, but the github context is not
available in a composite action's step-level if:. The action manifest
therefore failed to load ("Unrecognized named-value: 'github'"), and
every workflow using ccache-setup broke at the "Set up ccache" step
(build library, make check, Compiler test, Multi-arch test, ...).

Gate on the built-in $GITHUB_EVENT_NAME env var in the shell instead,
which keeps the schedule-only reseed behaviour with no caller changes.
2026-06-15 22:36:35 +00:00
Juliusz Sosinowicz 146e1c3d34 CI: reseed ccache from clean compiles on scheduled runs
The scheduled (cron) refresh restored the prior ccache and recompiled
only the translation units that changed, so unchanged objects were never
rebuilt and the shared cache could drift indefinitely. Set
CCACHE_RECACHE=1 on schedule events - gated inside the ccache-setup
action, so none of the calling workflows change - to force fresh
compiles that re-store every result. PR and push runs are unaffected and
keep their warm hits; only the scheduled jobs pay the full recompile.
2026-06-15 22:36:35 +00:00
Juliusz Sosinowicz dd2f9d3ab8 CI: offload ccache/apt/buildx caches off the GitHub Actions cache
The 10 GB, LRU-evicted, PR-scoped Actions cache was being thrashed - the
docker simulator buildx layers (~6 GiB), plus per-PR ccache and apt-archive
writes whose keys never hit - which kept evicting the shared ccache, while
the apt mirror timed out often enough to break PR CI. Move the heavy caches
to ghcr (free, separate pool) and make PR runs read-only against the Actions
cache.

apt dependencies from prebuilt ghcr .deb bundles
  - ci-deps-image.yml resolves each package list under .github/ci-deps/ into
    its .deb closure and publishes ghcr.io/<owner>/wolfssl-ci-debs:<tag> in
    two tiers: <ver>-minimal (make-check family) and <ver>-full (interop
    superset), for ubuntu-22.04 and 24.04.
  - install-apt-deps gains a ghcr-debs-tag input: pull the bundle and install
    offline (--no-download) so the apt mirror is never on the PR critical
    path. Any failure (bundle missing/not public/incomplete) falls through to
    the existing apt path, so it is always safe to set.

sim-test buildx layers to a shared ghcr registry cache
  - the 7 docker simulator workflows switch from cache-to: type=gha to
    ghcr.io/wolfssl/wolfssl-sim-cache:<scope>. cache-from reads on every run
    (anonymous); cache-to writes only on the weekend cron and manual
    workflow_dispatch. Per-distinct-image tags and de-duplicated writers keep
    parallel matrix jobs from racing on one ref.

ccache: PRs read, the schedule writes
  - ccache-setup gains read-only: PR runs restore the shared master-scoped
    cache but never upload; schedule/push runs refresh it. Wired across
    os-check (linux + macOS), pq-all, smoke-test and the 12 small make-check
    workflows.
  - parallel-make-check.py gains --build-only (compile every config, skip the
    test phase) so weekday-morning seed crons warm the cache PR runs consume.

artifact retention capped at 7 days on the failure-log/result uploads that
previously defaulted to 90.

ONE-TIME SETUP: after their first publish, make the ghcr packages
wolfssl-ci-debs and wolfssl-sim-cache PUBLIC so anonymous pulls work from PR
(including fork) runs; until then everything falls back cleanly.
2026-06-15 22:36:35 +00:00
Juliusz Sosinowicz 844852202b .github: bump JavaScript actions to Node.js 24 runtimes
GitHub Actions now emits "Node.js 20 actions are deprecated" warnings:
actions are forced to Node.js 24 by default starting 2026-06-16, and
Node.js 20 is removed from the runners on 2026-09-16. Update every
JavaScript action referenced by the workflows and the local composite
actions to the lowest release that runs on Node.js 24:

  actions/checkout              v4     -> v5
  actions/checkout (SHA pin)    v4.1.7 -> v5
  actions/upload-artifact       v4     -> v6   (v5 still Node.js 20)
  actions/download-artifact     v4     -> v7   (v5/v6 still Node.js 20)
  actions/cache[/restore|/save] v4     -> v5
  actions/setup-python          v5     -> v6
  actions/github-script         v7     -> v8
  docker/setup-buildx-action    v3     -> v4
  docker/build-push-action      v5     -> v7   (v6 still Node.js 20)
  docker/login-action           v3     -> v4
  microsoft/setup-msbuild       v2     -> v3
  open-watcom/setup-watcom      v0     -> v1

Actions already running on Node.js 24 (jwlawson/actions-setup-cmake,
shogo82148/actions-setup-perl, msys2/setup-msys2, dorny/paths-filter)
are left unchanged. These bumps are runtime-only; no workflow uses an
input or output removed by the new majors, and v4-format artifacts
remain compatible across the upload v6 / download v7 backends.
2026-06-15 18:09:04 +00:00
Juliusz Sosinowicz 3a6c31a51e CI: pool the per-config runner matrices into parallel make-check jobs
Replace the one-runner-per-configuration matrices across the
make-check workflow family with a generic pooled runner,
.github/scripts/parallel-make-check.py. Each workflow keeps its
configuration list as JSON next to the invocation; one runner (or a
small fixed set of shards, balanced by measured per-config minutes)
builds every config in its own out-of-tree (VPATH) build directory off
a single checkout/autogen, on a pool of one-per-CPU worker threads,
longest first. Concurrent checks are isolated with bubblewrap network
namespaces, compilations are cached with ccache, the first failure
aborts the rest (fail-fast, with --no-fail-fast to run everything),
and per-config timings plus pool efficiency land in the step summary.
Failure logs upload as artifacts. smoke-test.yml is likewise reworked
into a single pooled job that runs its nine configs on one runner.

Converted workflows (runner jobs per full pass):
  os-check.yml             101 -> 8  (92 Ubuntu configs -> 4 shards;
                           the macOS matrix, the user-settings jobs and
                           the standalone
                           macos-apple-native-cert-validation.yml fold
                           into one macOS runner; Windows unchanged)
  pq-all.yml                21 -> 2 shards
  disable-pk-algs.yml       15 -> 1
  wolfCrypt-Wconversion.yml 11 -> 1
  trackmemory.yml            7 -> 1
  cryptocb-only.yml          8 -> 1  (incl. the two new SHA512 entries)
  multi-compiler.yml         6 -> 1
  smallStackSize.yml         6 -> 1
  multi-arch.yml             6 -> 1
  async.yml                  5 -> 1
  psk.yml                    5 -> 1
  no-malloc.yml              3 -> 1
  wolfsm.yml                 3 -> 1
  opensslcoexist.yml         2 -> 1

Measured against current upstream passing runs (job execution time,
queue excluded): ~200 runner jobs / ~374 runner-minutes per full pass
become 23 jobs / ~168 runner-minutes, with more coverage than before.
multi-arch's old matrix combined an "include" list of four
architectures with an "opts" axis; GitHub's include-merge rules made
each arch entry overwrite the previous one, so only the armel
combinations actually ran. The pooled list restores the intended
aarch64/armhf/riscv64 coverage (23 combinations; riscv64 x sp-math is
omitted as invalid - configure rejects sp-math without SP, and
--enable-riscv-asm, unlike --enable-sp-asm, does not bring SP in).

Out-of-tree build fixes this depends on:
- Makefile.am: symlink the read-only test data (certs/, tests/ config
  files, sniffer captures and helpers, examples/crypto_policies,
  input, quit) into the build tree via a BUILT_SOURCES stamp, removed
  again in distclean-local. ChangeToWolfRoot() and the script tests
  resolve everything relative to the working directory, so out-of-tree
  make check and make distcheck now pass.
- scripts/multi-msg-record.py: locate the client binary from the build
  tree working directory rather than the script's source directory.
- configure.ac + wolfssl/include.am: run
  support/gen-debug-trace-error-codes.sh from $srcdir; it reads the
  error-code headers from the source tree and generates into the build
  tree.
- tests/swdev: a WOLFBUILD variable points the sub-make at the build
  tree for the configure-generated headers (wolfssl/options.h,
  wolfssl/version.h); the in-tree-only guards are dropped.

Portions of PR #10649 are incorporated: the cross-platform
ccache-setup composite action, repository_owner gates on check-headers
and check-source-text, the docs-only paths-ignore on os-check, and the
libspdm timeout bumps.
2026-06-12 09:47:13 +00:00
David Garske 08022ffebf CI: cache compiler output in os-check Ubuntu matrix
- Add .github/actions/ccache-setup composite (ccache + PATH intercept).
- Wire into os-check.yml make_check_linux; macOS unchanged for now.
- Measured on master --enable-all: cold 11.6s -> warm 1.1s (~10x), 100% hit.
2026-05-26 10:36:49 -07:00