Maamoun TK maamoun.tk@googlemail.com writes:
I'm debugging the issue using gdb. Apparently, ecc->q.bit_size takes 2 values (256 and 512),
That is as expected. I think it makes sense to use gostdsa is only with one of the gost curves, gc256b (both p and q 256 bits) and gc512a (both p and q 512 bits).
the error occurs when bit_size is 512 and eventually it produces a bad_alloc in TLS when one goes further through the calling chain. Let me know if the issue needs further debugging.
Can you give a bit more context of what case you are debugging? For gostdsa proper, the intention of changes on the fix-gostdsa-zero branch is that it should
1. fix the bug with how an input digest that equals q is handled.
2. hit an assertion failure for obscure cases (q->bit_size not a multiple of 8, and an input digest size that is larger. E.g., if one attempts to use gostdsa with the secp521 curve, and and a digest size of 66 octets or larger, e.g., based on shake).
Any other changes in behavior are unintentional and likely bugs...
For gostdsa_vko, I haven't yet made any changes. If you're debugging use of gostdsa_vko in gnutls, I'm interested to know which values of ukm_length can occur when it calls nettle's gostdsa_vko function.
Regards, /Niels