Add full RSA-PSS (RSASSA-PSS) support to PKCS#7 SignedData
encoding and verification.
This change enables SignerInfo.signatureAlgorithm to use
id-RSASSA-PSS with explicit RSASSA-PSS-params (hash, MGF1,
salt length), as required by RFC 4055 and CMS profiles.
Key changes:
- Add RSA-PSS encode and verify paths for PKCS7 SignedData
- Encode full RSASSA-PSS AlgorithmIdentifier parameters
- Decode RSA-PSS parameters from SignerInfo for verification
- Treat RSA-PSS like ECDSA (sign raw digest, not DigestInfo)
- Fix certificate signatureAlgorithm parameter length handling
- Add API test coverage for RSA-PSS SignedData
This resolves failures when using RSA-PSS signer certificates
(e.g. -173 invalid signature algorithm) and maintains backward
compatibility with RSA PKCS#1 v1.5 and ECDSA.
Signed-off-by: Sameeh Jubran <sameeh@wolfssl.com>
Sameeh Jubran
deb668ca4b