Re: [PATCH] Avoid assertion failure in pss_verify_mgf1

nisse@lysator.liu.se (Niels Möller) writes:

...

• /* Check "integer too long" error of I2OSP. */
• if (key_size < nettle_mpz_sizeinbase_256_u(m))
• goto cleanup;

I don't understand the I2OSP acronym. And I think this check would be more explicit as

if (mpz_sizeinbase(m, 2) > bits) goto cleanup;

(one might also move initial size checks before the allocations).

I2OSP is the procedure defined in RFC 3447, which converts a nonnegative integer to an octet string of a specified length. It is based on octets rather than bits.

I think the above check is too rigid, since it is based on bit-length, it wouldn't tolerate some cases such as m is 1016 bits and bits is 1015, where both can be represented in 127 octets.

Regards,