wolfSSL's configure enables make's jobserver by default
(AX_AM_JOBSERVER([yes]) -> AM_MAKEFLAGS += -j<nproc+1> in aminclude.am),
and automake passes that explicit -j to every recursive sub-make, where
it overrides the invoking make's job limit. The script's -j therefore
only ever scheduled the outermost recursion hop: --jobs was inert.
Measured on a 4-CPU host with 10 build-only configs oversaturating the
worker pool, the jobserver default is also the better policy: capping
sub-makes via --disable-jobserver and -j2 dropped CPU utilization from
96% to 89% and lengthened the wall time, because configs' serial
phases (configure, link) stopped being backfilled by other configs'
compile jobs. So make is now invoked with no -j at all - parallelism
within a config comes from the configure-default jobserver - and the
misleading knob is gone, including the macOS job's --jobs 3.
Third Copilot review round:
- Makefile.am: run the test-data stamp recipe body under set -e. A
failed symlink mid-loop previously did not fail the compound command
(only the last command's status counted), so a partially-populated
build tree could be stamped complete. Now any failed setup command
aborts the recipe and the stamp is not created.
- parallel-make-check.py: fail-fast sent SIGTERM only, so a test that
traps or ignores SIGTERM could keep the job alive until the workflow
timeout. abort_others() now polls the swept processes and SIGKILLs
whatever is still alive after a 10 s grace period, and the
post-registration race-window kill escalates the same way (bounded
wait, then SIGKILL). Verified with a config running
"trap '' TERM; sleep 300": the run completes in ~10 s with the
stubborn config reported as aborted and no surviving processes.
Two fixes from the second Copilot review round:
A process spawned between abort_others()' live_procs snapshot and its
registration escaped the kill sweep, leaving that build/check running
to completion after fail-fast had begun. Re-check stop_event right
after registering the process and SIGTERM its process group if the
abort already started: either the registration happened before the
sweep's snapshot (the sweep kills it) or it happened after stop_event
was set (the re-check sees it), so the window is closed.
Exceptions from callable steps (user_settings staging, private-dir
copies) used to escape the worker thread and crash the whole script
with no summary. They are now recorded as that config's step failure
with the exception written to its make-check.log, e.g. a bad
"user_settings" path reports FAIL (stage <path>) while the other
configs keep running; the fail-fast bookkeeping is shared with the
nonzero-exit path via record_failure().
Address the Copilot review:
- parallel-make-check.py: validate "configure" (list of strings) and
cflags/ldflags (strings) so a malformed entry fails the load instead
of exploding a string into per-character configure arguments; print
a single line for passing configs instead of dumping their full
make-check.log into the CI log (failure dumps unchanged; the logs
remain in build-<name>/ for the failure artifacts).
- Makefile.am: use rm -rf for the certs/input/quit setup and distclean
cleanup. A --private-dir run replaces the certs symlink with a
private directory copy that rm -f cannot remove (verified: make
distclean in a build dir with a privatized certs/ now succeeds and
removes it).
- psk.yml, disable-pk-algs.yml: normalize the single-dash tokens
(-disable-rsa, -disable-ecc, -disable-aescbc, -enable-cryptonly)
carried verbatim from the old matrices to the canonical double-dash
form. No coverage change: configure honors single-dash spellings
(verified -disable-rsa sets NO_RSA with no unrecognized-option
warning), so these were always in effect; both touched configs
re-validated end-to-end.
The --cc default stays "ccache gcc": ccache resolves the compiler
through its own masquerade symlinks (verified: no recursion and normal
cache hits with /usr/lib/ccache prepended to PATH), and the explicit
CC= also covers jobs that use ccache without the PATH masquerade.
Replace the one-runner-per-configuration matrices across the
make-check workflow family with a generic pooled runner,
.github/scripts/parallel-make-check.py. Each workflow keeps its
configuration list as JSON next to the invocation; one runner (or a
small fixed set of shards, balanced by measured per-config minutes)
builds every config in its own out-of-tree (VPATH) build directory off
a single checkout/autogen, on a pool of one-per-CPU worker threads,
longest first. Concurrent checks are isolated with bubblewrap network
namespaces, compilations are cached with ccache, the first failure
aborts the rest (fail-fast, with --no-fail-fast to run everything),
and per-config timings plus pool efficiency land in the step summary.
Failure logs upload as artifacts. smoke-test.yml is likewise reworked
into a single pooled job that runs its nine configs on one runner.
Converted workflows (runner jobs per full pass):
os-check.yml 101 -> 8 (92 Ubuntu configs -> 4 shards;
the macOS matrix, the user-settings jobs and
the standalone
macos-apple-native-cert-validation.yml fold
into one macOS runner; Windows unchanged)
pq-all.yml 21 -> 2 shards
disable-pk-algs.yml 15 -> 1
wolfCrypt-Wconversion.yml 11 -> 1
trackmemory.yml 7 -> 1
cryptocb-only.yml 8 -> 1 (incl. the two new SHA512 entries)
multi-compiler.yml 6 -> 1
smallStackSize.yml 6 -> 1
multi-arch.yml 6 -> 1
async.yml 5 -> 1
psk.yml 5 -> 1
no-malloc.yml 3 -> 1
wolfsm.yml 3 -> 1
opensslcoexist.yml 2 -> 1
Measured against current upstream passing runs (job execution time,
queue excluded): ~200 runner jobs / ~374 runner-minutes per full pass
become 23 jobs / ~168 runner-minutes, with more coverage than before.
multi-arch's old matrix combined an "include" list of four
architectures with an "opts" axis; GitHub's include-merge rules made
each arch entry overwrite the previous one, so only the armel
combinations actually ran. The pooled list restores the intended
aarch64/armhf/riscv64 coverage (23 combinations; riscv64 x sp-math is
omitted as invalid - configure rejects sp-math without SP, and
--enable-riscv-asm, unlike --enable-sp-asm, does not bring SP in).
Out-of-tree build fixes this depends on:
- Makefile.am: symlink the read-only test data (certs/, tests/ config
files, sniffer captures and helpers, examples/crypto_policies,
input, quit) into the build tree via a BUILT_SOURCES stamp, removed
again in distclean-local. ChangeToWolfRoot() and the script tests
resolve everything relative to the working directory, so out-of-tree
make check and make distcheck now pass.
- scripts/multi-msg-record.py: locate the client binary from the build
tree working directory rather than the script's source directory.
- configure.ac + wolfssl/include.am: run
support/gen-debug-trace-error-codes.sh from $srcdir; it reads the
error-code headers from the source tree and generates into the build
tree.
- tests/swdev: a WOLFBUILD variable points the sub-make at the build
tree for the configure-generated headers (wolfssl/options.h,
wolfssl/version.h); the in-tree-only guards are dropped.
Portions of PR #10649 are incorporated: the cross-platform
ccache-setup composite action, repository_owner gates on check-headers
and check-source-text, the docs-only paths-ignore on os-check, and the
libspdm timeout bumps.
CPPFLAGS replaces C_EXTRA_FLAGS with embedded single-quotes, which were
passed as literal characters through the shell variable and caused
configure's C compiler test to fail. Fix the report.json summary parser
to use the actual TLS-Anvil field names (TotalTests, FullyFailedTests,
etc.) and include category scores.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Runs the TLS-Anvil combinatorial test suite nightly against wolfSSL in
all four roles: TLS 1.2/1.3 server and TLS 1.2/1.3 client. Results are
summarized in the job summary and uploaded as artifacts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Juliusz Sosinowicz
Tobias Frauenschläger
Andrew Hutchings
David Garske
sebastian-carpenter
night1rider
Eric Blankenhorn