Hi,
I've received a bug report on gostdsa. When the digest is converted to an integer mod q, the spec says that if that happens to become 0, it should be replaced by 1 (and using 0 results in private key leakage).
The current code checks for 0 without first reducing mod q, which is bad. I have a tentative fix on branch https://git.lysator.liu.se/nettle/nettle/-/tree/fix-gostdsa-zero.
Then there's a related problem in handling of the "UKM" input for gostdsa_vko, if that happens to equal q or a multiple thereof. I suspect the UKM is usually rather small, which would help. E.g., if I could do something like
ukm_max_length = (ecc->q.bit_size - 1) / 8; assert (ukm_length <= ukm_max_length);
the number would surely be less than q, and current plain check for 0 would be fine.
Is there a specified maximum size? I've looked at
* RFC 4357, which seems to say the UKM is 64 bits
* RFC 9189, which seems to say that the UKM is 128 bits (16 bytes).
* GnuTLS (only user of this function I'm aware of). It's unobvious to me what size of the nonce/ukm is actually passed down to nettle, but at least I've found a
#define VKO_GOST_UKM_LEN 8
which seems to indicate 64 bits.
Anyone who's more familiar with how this is used in TLS or elsewhere?
Regards, /Niels
Hi Niels,
I'm debugging the issue using gdb. Apparently, ecc->q.bit_size takes 2 values (256 and 512), the error occurs when bit_size is 512 and eventually it produces a bad_alloc in TLS when one goes further through the calling chain. Let me know if the issue needs further debugging.
best regards, Mamone
On Fri, Sep 4, 2026 at 6:34 PM Niels Möller nisse@lysator.liu.se wrote:
Hi,
I've received a bug report on gostdsa. When the digest is converted to an integer mod q, the spec says that if that happens to become 0, it should be replaced by 1 (and using 0 results in private key leakage).
The current code checks for 0 without first reducing mod q, which is bad. I have a tentative fix on branch https://git.lysator.liu.se/nettle/nettle/-/tree/fix-gostdsa-zero.
Then there's a related problem in handling of the "UKM" input for gostdsa_vko, if that happens to equal q or a multiple thereof. I suspect the UKM is usually rather small, which would help. E.g., if I could do something like
ukm_max_length = (ecc->q.bit_size - 1) / 8; assert (ukm_length <= ukm_max_length);
the number would surely be less than q, and current plain check for 0 would be fine.
Is there a specified maximum size? I've looked at
RFC 4357, which seems to say the UKM is 64 bits
RFC 9189, which seems to say that the UKM is 128 bits (16 bytes).
GnuTLS (only user of this function I'm aware of). It's unobvious to me what size of the nonce/ukm is actually passed down to nettle, but at least I've found a
#define VKO_GOST_UKM_LEN 8
which seems to indicate 64 bits.
Anyone who's more familiar with how this is used in TLS or elsewhere?
Regards, /Niels
-- Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance.
nettle-bugs mailing list -- nettle-bugs@lists.lysator.liu.se To unsubscribe send an email to nettle-bugs-leave@lists.lysator.liu.se
Maamoun TK maamoun.tk@googlemail.com writes:
I'm debugging the issue using gdb. Apparently, ecc->q.bit_size takes 2 values (256 and 512),
That is as expected. I think it makes sense to use gostdsa is only with one of the gost curves, gc256b (both p and q 256 bits) and gc512a (both p and q 512 bits).
the error occurs when bit_size is 512 and eventually it produces a bad_alloc in TLS when one goes further through the calling chain. Let me know if the issue needs further debugging.
Can you give a bit more context of what case you are debugging? For gostdsa proper, the intention of changes on the fix-gostdsa-zero branch is that it should
1. fix the bug with how an input digest that equals q is handled.
2. hit an assertion failure for obscure cases (q->bit_size not a multiple of 8, and an input digest size that is larger. E.g., if one attempts to use gostdsa with the secp521 curve, and and a digest size of 66 octets or larger, e.g., based on shake).
Any other changes in behavior are unintentional and likely bugs...
For gostdsa_vko, I haven't yet made any changes. If you're debugging use of gostdsa_vko in gnutls, I'm interested to know which values of ukm_length can occur when it calls nettle's gostdsa_vko function.
Regards, /Niels
On Sun, Sep 6, 2026 at 12:10 PM Niels Möller nisse@lysator.liu.se wrote:
Maamoun TK maamoun.tk@googlemail.com writes:
I'm debugging the issue using gdb. Apparently, ecc->q.bit_size takes 2 values (256 and 512),
That is as expected. I think it makes sense to use gostdsa is only with one of the gost curves, gc256b (both p and q 256 bits) and gc512a (both p and q 512 bits).
the error occurs when bit_size is 512 and eventually it produces a bad_alloc in TLS when one goes further through the calling chain. Let me know if the issue needs further debugging.
Can you give a bit more context of what case you are debugging? For gostdsa proper, the intention of changes on the fix-gostdsa-zero branch is that it should
fix the bug with how an input digest that equals q is handled.
hit an assertion failure for obscure cases (q->bit_size not a multiple of 8, and an input digest size that is larger. E.g., if one attempts to use gostdsa with the secp521 curve, and and a digest size of 66 octets or larger, e.g., based on shake).
Any other changes in behavior are unintentional and likely bugs...
For gostdsa_vko, I haven't yet made any changes. If you're debugging use of gostdsa_vko in gnutls, I'm interested to know which values of ukm_length can occur when it calls nettle's gostdsa_vko function.
For a 256-bit q, a ukm_length up to 34 sustains a valid path, beyond which unstable TLS behavior occurs. At some point, we need to verify these procedures with pre/post-conditions and lemmas to ensure the math behind is identical to the specification.
Regards, /Niels
-- Niels Möller. PGP key CB4962D070D77D7FCB8BA36271D8F1FF368C6677. Internet email is subject to wholesale government surveillance.
nettle-bugs@lists.lysator.liu.se
-
Maamoun TK -
Niels Möller