linuxkm/: when logging PTR_ERR(), cast it to int, and use "%d" as the format. Globally, `#define PTR_ERR(x) ((int)PTR_ERR(x))` in linuxkm_wc_port.h to fix clang warnings on kernel headers.
linuxkm/lkcapi_aes_glue.c: add casts in linuxkm_test_aesgcm() to mollify clang.
linuxkm/linuxkm_wc_port.h, linuxkm/module_hooks.c:
* add __clang__ compat code to allow including clang stdatomic.h while masking out kernel-incompatible __CLANG_STDINT_H.
* add clang-specific suppressions for kernel headers (-Wshorten-64-to-32, -Wframe-address).
linuxkm/lkcapi_sha_glue.c:
* in wc__get_random_bytes(), add bounds-checking for len.
* in wc_extract_crng_user(), fix type conflicts.
wolfssl/wolfcrypt/wc_port.h and wolfssl/wolfcrypt/types.h:
* move the old-FIPS compatibility mapping from INLINE to WC_INLINE from types.h to wc_port.h.
* activate stdatomic.h for clang kernel module builds.
linuxkm/Kbuild:
* add clang-specific flags.
* add gcc gate around gcc-specific flags.
* allow override value for MAX_STACK_FRAME_SIZE.
wolfcrypt/src/asn.c: add casts in GetFormattedTime_ex() to mollify clang build of linuxkm.
fix F-3291: Copy-paste error in linuxkm_test_aesgcm error message uses WOLFKM_AESCBC_DRIVER
fix F-1431: AES-GCM RFC4106 SetKey Uses memcpy Instead of XMEMCPY for Nonce Copy
(note, for F-1431, changed all relevant memset() and memcpy() calls in linuxkm/ to XMEMSET() and XMEMCPY() respectively.)
* add WOLFSSL_API attribute to wc_linuxkm_sig_ignore_begin(), wc_linuxkm_sig_ignore_end(), wc_linuxkm_check_for_intr_signals(), and wc_linuxkm_relax_long_loop().
* fix WC_CONTAINERIZE_THIS macro wrappers for wc_linuxkm_sig_ignore_begin() and wc_linuxkm_sig_ignore_end() (stray semicolons).
linuxkm/linuxkm_wc_port.h, linuxkm/lkcapi_sha_glue.c, linuxkm/module_hooks.c: add wc_linuxkm_can_block(), and refactor ad hoc `preempt_count() != 0` checks for sleep safety as calls to wc_linuxkm_can_block().
linuxkm/module_hooks.c: fix wc_linuxkm_malloc_usable_size() implementation for kvmalloc() compatibility.
* wc_rng_bank_default_set()
* wc_rng_bank_default_checkout()
* wc_rng_bank_default_checkin()
* wc_rng_bank_default_clear()
* Added additional argument error checking to existing APIs, with a new
rng_inst_matches_bank() helper function.
* Implemented feature gates WC_RNG_BANK_DEFAULT_SUPPORT and
WC_RNG_BANK_NO_DEFAULT_SUPPORT. When WC_RNG_BANK_DEFAULT_SUPPORT, the new
APIs are available, and a NULL bank passed to APIs implicitly refers to the
default bank.
wolfcrypt/test/test.c: in random_bank_test() add comprehensive smoke test coverage of new APIs and argument checking.
wolfssl/wolfcrypt/wc_port.h and wolfcrypt/src/wc_port.c:
* Add wolfSSL_RefInc2(), wolfSSL_RefDec2(), wolfSSL_RefWithMutexInc2(), and
wolfSSL_RefWithMutexDec2(), returning the atomically determined new count in
the second arg;
* Fix type of second arg in the fallback definition of
wolfSSL_Atomic_Ptr_CompareExchange().
linuxkm/lkcapi_sha_glue.c:
Refactor the _REGISTER_HASH_DRBG / _REGISTER_HASH_DRBG_DEFAULT facility around
the new wc_rng_bank_default facility, eliminating post-init use of
kernel-native crypto_default_rng, crypto_get_default_rng(), and
crypto_put_default_rng(), and eliminating all use on kernel 7.1+ (where these
will become unexported kernel-native statics). With the refactor, the
LINUXKM_DRBG_GET_RANDOM_BYTES facility uses only direct native wolfCrypt
objects and calls to fulfill requests.
wolfssl/wolfcrypt/error-crypt.h, wolfcrypt/src/error.c, wolfcrypt/test/test.c, tests/api.c: add WC_SUCCESS = 0 "wolfCrypt generic success".
fixes and workarounds for clang-tidy complaints:
* clang-diagnostic-unknown-warning-option
* bugprone-sizeof-expression
* clang-diagnostic-error "address argument to atomic operation must be a pointer to a trivially-copyable type"
* bugprone-macro-parentheses
* clang-diagnostic-unused-but-set-variable
* readability-redundant-declaration
wolfcrypt/src/rng_bank.c:
* add wc_local_rng_bank_checkout_for_bankref, wc_BankRef_Release(), wc_rng_bank_new(), and wc_rng_bank_free();
* in wc_rng_bank_checkin(), take a struct wc_rng_bank_inst **rng_inst and NULL it before return;
* in wc_rng_bank_init(), add a devId arg, and handle devId in wc_rng_bank_inst_reinit();
* add WC_RNG_BANK_INST_LOCK_* and use them in wc_rng_bank_checkout() and wc_rng_bank_checkin();
* fix order of operations in wc_rng_bank_checkout() re DISABLE_VECTOR_REGISTERS();
wolfcrypt/src/random.c:
* refactor per-instance salting for wc_rng_bank_inst: remove changes in Hash_df(), Hash_DRBG_Instantiate(), and _InitRng(), and in wc_rng_bank_init() and wc_rng_bank_inst_reinit(), use wc_InitRngNonce_ex() and pass the wc_rng_bank_inst pointer as the nonce;
* simplify the WC_RNG_BANK_SUPPORT variant of wc_RNG_GenerateBlock() -- delegate to wc_local_rng_bank_checkout_for_bankref() and remove supplementary error checking;
* in wc_FreeRng(), call wc_BankRef_Release() when WC_DRBG_BANKREF, and in wc_BankRef_Release(), fix refcount flub (not wolfSSL_RefFree, rather wolfSSL_RefDec);
* streamline the WOLFSSL_LINUXKM wc_GenerateSeed();
wolfcrypt/test/test.c: add random_bank_test();
linuxkm/lkcapi_sha_glue.c: use WC_RNG_BANK_INST_TO_RNG() opportunistically;
configure.ac: add --enable-amdrdseed as a synonym for --enable-amdrand;
linuxkm/linuxkm_wc_port.h: when LINUXKM_LKCAPI_REGISTER_HASH_DRBG_DEFAULT, don't include get_random_bytes() in struct wolfssl_linuxkm_pie_redirect_table;
add various comments for clarity.
* refactor to eliminate recursion in wc_RNG_GenerateBlock();
* refactor enum wc_rng_bank_flags as word32 and macros;
* fix -Wconversions, -Wunused, and stray EINVAL in wc_rng_bank_init();
* make struct wc_rng_bank_inst a top-level definition for C++ compat;
* fix several bugprone-macro-parentheses.
* in get_crypto_default_rng() (linuxkm/lkcapi_sha_glue.c), sanity check that crypto_default_rng isn't null;
* in wc_InitRsaKey_ex(), remove frivolous NULL/zero assignments (XMEMSET clears them implicitly);
* in wc_CheckRsaKey(), check ret from wc_InitRng() and short circuit return if failed.
* add drbg_init_from() and fork_default_rng(), and
* use the latter to define LKCAPI_INITRNG_FOR_SELFTEST() opportunistically (with fallback to plain wc_InitRng());
linuxkm/lkcapi_rsa_glue.c:
* add km_rsa_ctx_init_rng(),
* remove wc_InitRng() from km_rsa_ctx_init(),
* remove the WC_RSA_BLINDING gates around calls to wc_RsaSetRNG(), and
* call km_rsa_ctx_init_rng() before each call that needs an initialized RNG;
linuxkm/lkcapi_dh_glue.c and linuxkm/lkcapi_ecdh_glue.c: in km_ffdhe_init() and km_ecdh_init(), if linuxkm_lkcapi_registering_now, use LKCAPI_INITRNG_FOR_SELFTEST() to initialize ctx->rng;
linuxkm/lkcapi_glue.c: add notes that lkcapi_sha_glue inclusion and registrations must precede PK, and move declaration of linuxkm_lkcapi_registering_now to precede lkcapi glue inclusions.
* as for other glue families, when LINUXKM_LKCAPI_REGISTER_ALL_KCONFIG, don't "#error Config conflict" if explicit LINUXKM_LKCAPI_DONT_REGISTER_foo is defined for the missing algorithm;
* in km_hmac_init(), use wc_HmacCopy() to copy p_ctx->wc_hmac to t_ctx->wc_hmac;
* in get_drbg(), when tfm == crypto_default_rng, only migrate_disable() and local_bh_disable() if preempt_count() == 0, i.e. if not in already in an uninterruptible context;
* add can_spin argument to get_drbg_n() -- wc_linuxkm_drbg_seed() can_spin, wc_mix_pool_bytes() !can_spin, and wc_crng_reseed() can_spin;
* add compile-time assert that WOLFSSL_SMALL_STACK_CACHE is defined if LINUXKM_DRBG_GET_RANDOM_BYTES;
.wolfssl_known_macro_extras: add CONFIG_CRYPTO_DRBG.
* in get_drbg(), call local_bh_disable() for the crypto_default_rng, and in put_drbg(), call local_bh_enable() if needed.
* re-gate migrate_disable() and migrate_enable() so they're called for any SMP kernel >= 5.7, regardless of CONFIG_PREEMPT_COUNT.
* in get_drbg_n(), if the caller can't sleep, return immediately if the requested DRBG is busy, to avoid priority inversions and deadlocks.
linuxkm/lkcapi_sha_glue.c: revert f7c7ac275a (get_drbg() DISABLE_VECTOR_REGISTERS() for crypto_default_rng) -- compiler/inlining bug makes it break on at least one target, so caller needs to retain responsibility;
linuxkm/x86_vector_register_glue.c: in wc_save_vector_registers_x86(), always return WC_ACCEL_INHIBIT_E if already fpu_state & WC_FPU_INHIBITED_FLAG, for safe+correct dynamics on recursive calls.
wc_save_vector_registers_x86() and wc_restore_vector_registers_x86() to allow
recursive WC_SVR_FLAG_INHIBIT while already in a vector save context;
linuxkm/lkcapi_sha_glue.c: in get_drbg() and put_drbg(),
DISABLE_VECTOR_REGISTERS()...REENABLE_VECTOR_REGISTERS() if tfm ==
crypto_default_rng.
add WC_GENERATE_SEED_DEFAULT, which defaults to wc_GenerateSeed if not overridden, and replace wc_GenerateSeed with WC_GENERATE_SEED_DEFAULT in various calls to wc_SetSeed_Cb();
linuxkm/linuxkm_wc_port.h: if FIPS <v6 and RDSEED, define WC_LINUXKM_RDSEED_IN_GLUE_LAYER and define WC_GENERATE_SEED_DEFAULT wc_linuxkm_GenerateSeed_IntelRD;
wolfcrypt/test/test.c: update rng_seed_test() with gating and vectors for FIPS v5 with HAVE_AMD_RDSEED or HAVE_INTEL_RDSEED;
wolfssl/wolfcrypt/types.h: add WC_HAVE_VECTOR_SPEEDUPS helper macro, and enlarge fallthrough definition coverage for DISABLE_VECTOR_REGISTERS.
* move enum wc_svr_flags out of BUILDING_WOLFSSL guard;
* add DISABLE_VECTOR_REGISTERS() and REENABLE_VECTOR_REGISTERS() definitions for !BUILDING_WOLFSSL;
* add #include <linux/spinlock.h> to !WOLFSSL_LINUXKM_USE_MUTEXES implementation to fix compilation (and add usability) to caller code;
linuxkm/lkcapi_sha_glue.c: in wc_linuxkm_drbg_ctx_clear(), fix error-path deallocation of locked object;
wolfcrypt/benchmark/benchmark.c:
* in FIPS v6+ builds, and FIPS linuxkm v5+, check retval from wc_AesEncryptDirect() and wc_AesDecryptDirect();
* add WC_RELAX_LONG_LOOP() in bench_stats_sym_finish() and bench_stats_asym_finish_ex();
wolfcrypt/test/test.c: fix rng_seed_test() with correct test vectors for the relevant combinations of features, and gate the test out if there are user override defines for ENTROPY_SCALE_FACTOR or SEED_BLOCK_SZ.
* refactor the save_vector_registers_x86() algorithm to depend directly on preempt_count(), and use local_bh_enable() and preempt_disable() directly, to mitigate glitchiness around irq_fpu_usable() and crypto_simd_usable();
* eliminate the WC_FPU_ALREADY_FLAG kludge.
* improve the error and warning messages, and add some additional checks and messages for unexpected states; add VRG_PR_ERR_X and VRG_PR_WARN_X for pr_*_once() semantics on regular builds, but unlimited messages when WOLFSSL_LINUXKM_VERBOSE_DEBUG.
linuxkm/linuxkm_wc_port.h and linuxkm/module_hooks.c:
* move the spinlock-based implementation of wc_LockMutex() from linuxkm_wc_port.h to module_hooks.c, due to numerous stuboorn direct external symbol references;
* extensively refactor the kernel header #include strategy, keeping many more superfluous headers out of __PIE__ objects, and fixing unavoidable static header functions with grafted __always_inline attributes;
* add version exceptions for RHEL 9.5.
linuxkm/Kbuild:
* on x86 with CONFIG_MITIGATION_{RETPOLINE,RETHUNK}, use inline rethunks rather than none;
* refactor check for "Error: section(s) missed by containerization." using `readelf --sections --syms`, for 100% coverage, more informative error output, and suppression of false positives on printk-related cruft;
configure.ac and linuxkm/lkcapi_sha_glue.c: use LINUXKM_LKCAPI_[DONT_]REGISTER_{SHA,HMAC}_ALL to represent --enable-linuxkm-lkcapi-register=[-]all-{sha,hmac}, which allows alg families (notably SHA1) to be masked out piecemeal;
linuxkm/lkcapi_rsa_glue.c: in linuxkm_test_pkcs1pad_driver(), mitigate unused args when LINUXKM_AKCIPHER_NO_SIGNVERIFY.
* add wc_linuxkm_drbg_ctx.n_rngs, and in wc_linuxkm_drbg_init_tfm(), set it to max(4, nr_cpu_ids), to avoid stalling on unicore targets;
* add explanatory comments re architecture to get_drbg() and get_drbg_n();
* add opportunistic cond_sched() to get_drbg_n();
* add runtime asserts in get_drbg(), wc_linuxkm_drbg_seed(), and get_default_drbg_ctx(), checking that we have the right tfm with an allocated DRBG array;
* wc_linuxkm_drbg_startup(): return failure if registering the random_bytes handlers fails;
linuxkm/patches/6.1.73/WOLFSSL_LINUXKM_HAVE_GET_RANDOM_CALLBACKS-6v1v73.patch: fix flub.
linuxkm/lkcapi_sha_glue.c:
* in wc_linuxkm_drbg_init_tfm(), sleep if it's safe, and observe a wc_linuxkm_drbg_init_tfm_disable_vector_registers flag;
* in wc_crng_reseed(), preemptively execute the reseed if it's safe to sleep;
* in wc_linuxkm_drbg_startup(), in LINUXKM_DRBG_GET_RANDOM_BYTES section, add reseed test sequence if defined(DEBUG_DRBG_RESEEDS).
wolfssl/wolfcrypt/settings.h: if WOLFSSL_LINUXKM && LINUXKM_LKCAPI_REGISTER && WORD64_AVAILABLE, set default WC_RESEED_INTERVAL to max allowed (2^48);
linuxkm/lkcapi_sha_glue.c: handle NO_LINUXKM_DRBG_GET_RANDOM_BYTES, for build-time override control of LINUXKM_DRBG_GET_RANDOM_BYTES, and handle WOLFSSL_LINUXKM_USE_GET_RANDOM_USER_KRETPROBE, for separate opt-in control of the buggy wc_get_random_bytes_user_kretprobe_enter().
* implement interception of _get_random_bytes() and get_random_bytes_user() (implicitly intercepts /dev/random and /dev/urandom):
* get_crypto_default_rng()
* get_default_drbg_ctx()
* wc__get_random_bytes()
* wc_get_random_bytes_user()
* wc_extract_crng_user()
* wc_mix_pool_bytes()
* wc_crng_reseed()
* wc_get_random_bytes_by_kprobe()
* wc_get_random_bytes_user_kretprobe_enter()
* wc_get_random_bytes_user_kretprobe_exit()
* add LINUXKM_DRBG_GET_RANDOM_BYTES sections to wc_linuxkm_drbg_startup() and wc_linuxkm_drbg_cleanup()
* add linuxkm/patches/*/WOLFSSL_LINUXKM_HAVE_GET_RANDOM_CALLBACKS-*.patch, initially for versions:
* 5.10.17
* 5.10.236
* 5.15
* 5.17
* 6.1.73
* 6.12
* 6.15
* remove "*.patch" from .gitignore.
* add linuxkm/patches/regen-patches.sh.
* in wc_linuxkm_drbg_ctx_clear(), check lock count before freeing.
* in get_drbg() and put_drbg(), use migrate_disable(), not DISABLE_VECTOR_REGISTERS().
* in wc_linuxkm_drbg_generate(), explicitly DISABLE_VECTOR_REGISTERS() for the crypto_default_rng.
* in wc_linuxkm_drbg_generate(), add DRBG reinitialization code to handle RNG_FAILURE_E. This handles the situation where a DRBG was instantiated in a vector-ops-allowed context, caching a vectorized SHA256 ethod, but later used in a no-vector-ops-allowed context.
* in wc_linuxkm_drbg_seed(), add DISABLE_VECTOR_REGISTERS() wrapper around wc_RNG_DRBG_Reseed() for crypto_default_rng.
linuxkm/x86_vector_register_glue.c:
* add crash recovery logic to wc_linuxkm_fpu_state_assoc_unlikely()
* in wc_linuxkm_fpu_state_assoc(), when wc_linuxkm_fpu_states is null, don't call wc_linuxkm_fpu_state_assoc_unlikely() if !assume_fpu_began.
* in can_save_vector_registers_x86(), save_vector_registers_x86(), and restore_vector_registers_x86(), check for hard interrupt context first, to return early failure if current->pid is unusable.
* in save_vector_registers_x86(), tweak logic around WC_FPU_INHIBITED_FLAG, adding local_bh_disable()...local_bh_enable() to provide for safe recursion.
wolfcrypt/src/random.c: optimization: in Hash_df(), for WOLFSSL_LINUXKM, don't put digest[WC_SHA256_DIGEST_SIZE] in the heap, keep it on the stack.
wolfssl/wolfcrypt/types.h: add WOLFSSL_NO_ASM no-op definitions for DISABLE_VECTOR_REGISTERS() and REENABLE_VECTOR_REGISTERS().
configure.ac:
* move --enable-linuxkm and --enable-linuxkm-defaults initial detection early, so that HMAC_COPY_DEFAULT picks it up.
* add ENABLED_ENTROPY_MEMUSE_DEFAULT, and enable it by default when ENABLED_LINUXKM_DEFAULTS.
* update linuxkm-lkcapi-register help message.
linuxkm/linuxkm_wc_port.h:
* add my_kallsyms_lookup_name().
* add preempt_count, _raw_spin_lock_irqsave, _raw_spin_trylock, _raw_spin_unlock_irqrestore, and _cond_resched, to wolfssl_linuxkm_pie_redirect_table, and add spin_unlock_irqrestore() macro to mask native inline.
* move linuxkm mutex wrappers from wolfcrypt/src/wc_port.c to linuxkm_wc_port.h, make them inlines, and add new default spinlock-based implementation, with old method now gated on WOLFSSL_LINUXKM_USE_MUTEXES.
* change malloc() and realloc() wrappers from GFP_KERNEL to GFP_ATOMIC.
linuxkm/lkcapi_glue.c: make misc.h/misc.c inclusion unconditional, and trim now-redundant inclusions out of lkcapi_dh_glue.c and lkcapi_ecdh_glue.c.
linuxkm/x86_vector_register_glue.c: implement support for WC_FPU_INHIBITED_FLAG, and an `int inhibit_p` argument to save_vector_registers_x86();
wolfcrypt/src/random.c: implement linuxkm support for RDSEED and HAVE_ENTROPY_MEMUSE;
wolfssl/wolfcrypt/error-crypt.h and wolfcrypt/src/error.c: add WC_ACCEL_INHIBIT_E "Crypto acceleration is currently inhibited";
linuxkm/module_hooks.c and linuxkm/x86_vector_register_glue.c: remove broken and bit-rotten WOLFSSL_COMMERCIAL_LICENSE and LINUXKM_FPU_STATES_FOLLOW_THREADS code paths.
wolfcrypt/src/random.c: in Hash_DRBG_Generate(), always put digest[] on the stack even in WOLFSSL_SMALL_STACK configuration (it's only 32 bytes);
configure.ac: default smallstackcache on when linuxkm-defaults.
Daniel Pouzzner
JacobBarthelmeh
kaleb-himes
jordan
Daniel Pouzzner