pkcs7: add RSA-PSS support for SignedData

Add full RSA-PSS (RSASSA-PSS) support to PKCS#7 SignedData
encoding and verification.

This change enables SignerInfo.signatureAlgorithm to use
id-RSASSA-PSS with explicit RSASSA-PSS-params (hash, MGF1,
salt length), as required by RFC 4055 and CMS profiles.

Key changes:
- Add RSA-PSS encode and verify paths for PKCS7 SignedData
- Encode full RSASSA-PSS AlgorithmIdentifier parameters
- Decode RSA-PSS parameters from SignerInfo for verification
- Treat RSA-PSS like ECDSA (sign raw digest, not DigestInfo)
- Fix certificate signatureAlgorithm parameter length handling
- Add API test coverage for RSA-PSS SignedData

This resolves failures when using RSA-PSS signer certificates
(e.g. -173 invalid signature algorithm) and maintains backward
compatibility with RSA PKCS#1 v1.5 and ECDSA.

Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
This commit is contained in:
Sameeh Jubran
2026-02-02 11:30:20 +02:00
parent 47033c4b3e
commit deb668ca4b
15 changed files with 1093 additions and 134 deletions
+2
View File
@@ -53,6 +53,8 @@ jobs:
'--enable-opensslall --enable-opensslextra CPPFLAGS=-DWC_RNG_SEED_CB',
'--enable-opensslall --enable-opensslextra
CPPFLAGS=''-DWC_RNG_SEED_CB -DWOLFSSL_NO_GETPID'' ',
# PKCS#7 with RSA-PSS (CMS RSASSA-PSS signers)
'--enable-pkcs7 CPPFLAGS=-DWC_RSA_PSS',
'--enable-opensslextra CPPFLAGS=''-DWOLFSSL_NO_CA_NAMES'' ',
'--enable-opensslextra=x509small',
'CPPFLAGS=''-DWOLFSSL_EXTRA'' ',