fixes for clang-tidy/bugprone-macro-parentheses in wolfssl/wolfcrypt/sp_int.{c,h} and wolfcrypt/src/ecc.c;
fix for clang-analyzer-deadcode.DeadStores in olfcrypt/src/asn.c.
Tested with `./configure --disable-asm --enable-32bit --enable-asn=original --enable-cryptonly CFLAGS="-Wconversion -pedantic" && make`. Some progress with ASN template, but not complete.
* Update to ensure full blocks for crypto in En/DecodeAuthEnvelopedData.
* Corrected spacing and comments
* Set plain to NULL after free on non-error path.
* refactor error-checking cascade in wc_PemCertToDer_ex() as in wc_PemPubKeyToDer_ex(),
* refactor staticBuffer gating/dynamics in wc_PemPubKeyToDer_ex() as in wc_PemCertToDer_ex(),
* and use IO_FAILED_E, not BUFFER_E, for I/O errors on the file handles, in both routines;
fix smallstack null pointer dereferences in src/pk.c:wolfSSL_RSA_GenAdd() and src/ssl.c:set_curves_list().
Minimize use of stack.
Make very large stack allocations dynamic memory allocations unless
WOLFSSL_SP_NO_MALLOC.
ProcessBufferTryDecode() split up into a function for each type.
ProcessBufferTryDecodeRsa() decodes the data and gets key size rather
than having or allocating an RsaKey.
Added wc_RsaPrivateKeyValidate() that only validates the encoding is an
RSA key and returns the key size in bytes.
For SP int, only create sp_ints of required size in RSA and ECC
implementation. For WOLFSSL_SMALL_STACK, memory is allocated to have
just enough bytes and size is set to maximum supported. Otherwise,
relies on dynamic stack variables.
For ECC, MAX_ECC_BITS_USE used when dynamic stack variables not
supported. Significantly reduces memory usage when RSA/DH is also built.
Add macros to sp_int.h, tfm.h and integer.h to support declaring,
allocating, initializing and freeing mp_ints.
For integer.h, mp_int is always static as size is no more than 32 bytes.
For tfm.h, WOLFSSL_SMALL_STACK has a full mp_int allocated, otherwise
the full mp_int is put on the stack.
For sp_int.h with new macros, dynamically allocate sp_int to minimal
size when WOLFSSL_SMALL_STACK, or when dynamic stack variables, declare
them to be of minimal size or otherwise declare with a fixed max.
Added mp_bitsused(), for all implementations, to get the number of bits
available based on used. Included for RSA to get the size of the
modulus.
SP int now always uses dynamic stack variables if possible rather than
for builds with WOLFSSL_SP_SMALL.
Moved code out into separate functions so that stack allocations don't
happen when not going down code path.
David Garske
JacobBarthelmeh
Daniel Pouzzner
tim-weller-wolfssl
Eric Blankenhorn
gojimmypi
Chris Conlon
billphipps
Bill Phipps
Sean Parkinson