Replace the one-runner-per-configuration matrices across the
make-check workflow family with a generic pooled runner,
.github/scripts/parallel-make-check.py. Each workflow keeps its
configuration list as JSON next to the invocation; one runner (or a
small fixed set of shards, balanced by measured per-config minutes)
builds every config in its own out-of-tree (VPATH) build directory off
a single checkout/autogen, on a pool of one-per-CPU worker threads,
longest first. Concurrent checks are isolated with bubblewrap network
namespaces, compilations are cached with ccache, the first failure
aborts the rest (fail-fast, with --no-fail-fast to run everything),
and per-config timings plus pool efficiency land in the step summary.
Failure logs upload as artifacts. smoke-test.yml is likewise reworked
into a single pooled job that runs its nine configs on one runner.
Converted workflows (runner jobs per full pass):
os-check.yml 101 -> 8 (92 Ubuntu configs -> 4 shards;
the macOS matrix, the user-settings jobs and
the standalone
macos-apple-native-cert-validation.yml fold
into one macOS runner; Windows unchanged)
pq-all.yml 21 -> 2 shards
disable-pk-algs.yml 15 -> 1
wolfCrypt-Wconversion.yml 11 -> 1
trackmemory.yml 7 -> 1
cryptocb-only.yml 8 -> 1 (incl. the two new SHA512 entries)
multi-compiler.yml 6 -> 1
smallStackSize.yml 6 -> 1
multi-arch.yml 6 -> 1
async.yml 5 -> 1
psk.yml 5 -> 1
no-malloc.yml 3 -> 1
wolfsm.yml 3 -> 1
opensslcoexist.yml 2 -> 1
Measured against current upstream passing runs (job execution time,
queue excluded): ~200 runner jobs / ~374 runner-minutes per full pass
become 23 jobs / ~168 runner-minutes, with more coverage than before.
multi-arch's old matrix combined an "include" list of four
architectures with an "opts" axis; GitHub's include-merge rules made
each arch entry overwrite the previous one, so only the armel
combinations actually ran. The pooled list restores the intended
aarch64/armhf/riscv64 coverage (23 combinations; riscv64 x sp-math is
omitted as invalid - configure rejects sp-math without SP, and
--enable-riscv-asm, unlike --enable-sp-asm, does not bring SP in).
Out-of-tree build fixes this depends on:
- Makefile.am: symlink the read-only test data (certs/, tests/ config
files, sniffer captures and helpers, examples/crypto_policies,
input, quit) into the build tree via a BUILT_SOURCES stamp, removed
again in distclean-local. ChangeToWolfRoot() and the script tests
resolve everything relative to the working directory, so out-of-tree
make check and make distcheck now pass.
- scripts/multi-msg-record.py: locate the client binary from the build
tree working directory rather than the script's source directory.
- configure.ac + wolfssl/include.am: run
support/gen-debug-trace-error-codes.sh from $srcdir; it reads the
error-code headers from the source tree and generates into the build
tree.
- tests/swdev: a WOLFBUILD variable points the sub-make at the build
tree for the configure-generated headers (wolfssl/options.h,
wolfssl/version.h); the in-tree-only guards are dropped.
Portions of PR #10649 are incorporated: the cross-platform
ccache-setup composite action, repository_owner gates on check-headers
and check-source-text, the docs-only paths-ignore on os-check, and the
libspdm timeout bumps.
7.4 KiB
wc_swdev: Software CryptoCb Device for Tests
wc_swdev is a test-only software backend used to exercise builds
that strip a wolfCrypt algorithm in favor of CryptoCb dispatch. It is
compiled separately from the main library, linked into the test
programs only, and exposes exactly two C symbols. It is not a
production component and must not be linked into shipping binaries.
The four switches it supports are:
| Macro | Strips | Test target |
|---|---|---|
WOLF_CRYPTO_CB_ONLY_RSA |
software RSA | RSA via CryptoCb |
WOLF_CRYPTO_CB_ONLY_ECC |
software ECC | ECC via CryptoCb |
WOLF_CRYPTO_CB_ONLY_SHA256 |
software SHA-256 | SHA-256 via CryptoCb |
WOLF_CRYPTO_CB_ONLY_AES |
software AES | AES via CryptoCb |
When a test program calls e.g. wc_AesCbcEncrypt() against a libwolfssl
built with -DWOLF_CRYPTO_CB_ONLY_AES, the software AES path is gone;
the call routes through the CryptoCb dispatch layer. swdev registers
itself as that callback, executes the operation against its own
internal copy of the AES code, and returns the result.
Architecture
+-----------------------------------------------------------+
| TEST PROGRAM (testwolfcrypt, unit.test, examples/...) |
| |
| wolfCrypt_Init() |
| wc_SwDev_Init() -- registers swdev device + a |
| WOLF_CRYPTO_CB_FIND hook |
| ... wc_AesCbcEncrypt(), wc_Sha256Update(), etc. ... |
| wc_SwDev_Cleanup() |
| wolfCrypt_Cleanup() |
+-----------------------------------------------------------+
| ^
| call into LIBWOLFSSL | result
v |
+-----------------------------------------------------------+
| LIBWOLFSSL (compiled with -DWOLF_CRYPTO_CB_ONLY_AES, |
| -DWOLF_CRYPTO_CB_ONLY_SHA256, ...) |
| |
| wc_AesCbcEncrypt() |
| - software AES is #ifdef'd out |
| - dispatch via wc_CryptoCb_AesCbc...() |
+-----------------------------------------------------------+
| ^
| CryptoCb dispatch |
v |
+-----------------------------------------------------------+
| tests/swdev/build/swdev.o |
| (single relocatable .o, only 2 visible symbols) |
| |
| wc_SwDev_Callback(devId, info, ctx) |
| - swdev_ensure_init() lazy wolfCrypt_Init |
| - switch (info->algo_type): |
| PK -> RSA / ECC software impl |
| HASH -> SHA-256 software impl |
| CIPHER -> AES (CBC/CTR/ECB/GCM/CCM) software impl |
| |
| swdev was compiled WITHOUT the WOLF_CRYPTO_CB_ONLY_* |
| gates, so its private copy of wolfcrypt still has the |
| full software implementations. |
+-----------------------------------------------------------+
How the Two-Compile Trick Works
The whole mechanism rests on compiling the wolfcrypt sources twice:
- libwolfssl is built normally with the user's
_ONLY_*flags set, so its software RSA/ECC/SHA-256/AES paths are gone. - swdev recompiles the same source set under
tests/swdev/user_settings.h, which#undefs all four_ONLY_*macros. swdev therefore contains the full software implementations.
To prevent symbol collisions when both are linked into the same test
binary, tests/swdev/Makefile does the following:
- Compiles every swdev TU with
-fvisibility=hidden -fno-common. - Drops
-DBUILDING_WOLFSSLsoWOLFSSL_APIdoes not re-promote symbols to default visibility. - Links all swdev objects with
ld -rinto a single relocatableswdev.partial.o. - Runs
objcopy --keep-global-symbol=wc_SwDev_Callback --keep-global-symbol=wc_SwDev_InternalCleanupto localize every remaining global except the two intended exports.
The Makefile then enforces the invariant directly:
nm --extern-only --defined-only build/swdev.o
must list only wc_SwDev_Callback and wc_SwDev_InternalCleanup.
The build fails loudly otherwise (see tests/swdev/Makefile:122-129).
If you add a third WC_SWDEV_EXPORT API, update the keep-list in the
Makefile too.
ABI Constraint
swdev and libwolfssl share C structs across the CryptoCb boundary
(wc_Sha256, Aes, RsaKey, ecc_key, ...). One compilation
allocates them, the other operates on them. They must therefore be
ABI-identical. The _ONLY_* macros only gate function bodies, not
struct layouts, so flipping them between the two compiles is safe.
Do not introduce other macros that change struct layout into
tests/swdev/user_settings.h.
Building
wc_swdev is enabled with the --enable-swdev configure flag.
./autogen.sh
./configure --enable-swdev --enable-cryptocb \
<other flags> \
CPPFLAGS="-DWOLF_CRYPTO_CB_ONLY_ECC \
-DWOLF_CRYPTO_CB_ONLY_RSA \
-DWOLF_CRYPTO_CB_ONLY_SHA256 \
-DWOLF_CRYPTO_CB_ONLY_AES"
make
make check
Notes:
--enable-swdevrequires--enable-cryptocb, or--enable-usersettingswithWOLF_CRYPTO_CBdefined in the user'suser_settings.h.--enable-swdevdefinesWOLFSSL_SWDEVandWOLF_CRYPTO_CB_FINDautomatically; seeconfigure.ac.- swdev is built from
wolfcrypt/test/include.amand inheritsPARENT_SRCS,PARENT_BUILD_CFLAGS, etc., from the parent build. Out-of-tree (VPATH) builds work:WOLFBUILDpoints the sub-make at the build tree for the configure-generated headers (wolfssl/options.h,wolfssl/version.h).
For the full CI matrix that exercises each _ONLY_* macro, see
.github/workflows/cryptocb-only.yml.
Files
| File | Role |
|---|---|
swdev.h |
Public swdev interface (the two exported symbols) |
swdev.c |
CryptoCb dispatcher: PK / HASH / CIPHER algorithms |
swdev_loader.h |
Test-harness API: wc_SwDev_Init, wc_SwDev_Cleanup |
swdev_loader.c |
Refcounted Init/Cleanup; registers the callback + Find hook |
user_settings.h |
#undefs the WOLF_CRYPTO_CB_ONLY_* gates for swdev's TU |
Makefile |
Two-compile + objcopy + symbol-invariant check |
Production Use
None. swdev exists only to make the WOLF_CRYPTO_CB_ONLY_* builds
testable on a generic Linux runner. Real deployments are expected to
provide their own CryptoCb backed by a hardware engine (TPM, HSM, SoC
crypto block, etc.). swdev is not API-stable, not benchmarked, and not
audited as a production cryptographic provider.