Fix invalid return value of ASN1_INTEGER_get()

When DIGIT_BIT is less than SIZEOF_LONG * CHAR_BIT, ASN1_INTEGER_get() can
return invalid value. For example, with trailing program, ASN1_INTEGER_get()
unexpectedly returns -268435449 (0xf0000007) on i386.

On the i386 platform (DIGIT_BIT=28), the input value 0x7fffffff is separated
into 0xfffffff and 0x7 and stored in the dp array of mp_int. Previously,
wolfSSL_BN_get_word_1() returned 0xfffffff shifted by 28 bits plus 0x7, so this
patch fixed it to return 0xfffffff plus 0x7 shifted by 28 bits.

int main(void)
{
    ASN1_INTEGER *a;
    long val;
    int ret;

    a = ASN1_INTEGER_new();
    val = 0x7fffffff;
    ret = ASN1_INTEGER_set(a, val);
    if (ret != 1) {
        printf("ret=%d\n", ret);
    }

    if (ASN1_INTEGER_get(a) != val) {
        printf("ASN1_INTEGER_get=%ld\n", ASN1_INTEGER_get(a));
    }

    ASN1_INTEGER_free(a);

    return 0;
}

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
This commit is contained in:
Masashi Honma
2021-11-17 11:04:50 +09:00
parent 995ef60ff1
commit cb3fc0c7ce

View File

@ -52994,10 +52994,8 @@ static WOLFSSL_BN_ULONG wolfSSL_BN_get_word_1(mp_int *mp) {
WOLFSSL_BN_ULONG ret = 0UL;
int digit_i;
for (digit_i = 0; digit_i < mp->used; ++digit_i) {
ret <<= (WOLFSSL_BN_ULONG)DIGIT_BIT;
ret |= (WOLFSSL_BN_ULONG)mp->dp[digit_i];
}
for (digit_i = 0; digit_i < mp->used; ++digit_i)
ret |= ((WOLFSSL_BN_ULONG)mp->dp[digit_i]) << (DIGIT_BIT * digit_i);
return ret;
#endif