In a different thread, I wrote:
(Somewhat related: I just realized that the slh-dsa signing functions hit an assertion failure if public and private keys don't match. I think it may be useful with an alternative signing function with a success/fail return value).
This isn't good; having asserts triggered by invalid input is ok only when inputs are invalid in a way that is (1) documented and (2) easy to check for at the call site.
What happens is that creating an SLH-DSA signature will more or less for free recompute the top-level Merkle tree root hash, which is one of the pieces of the public key. So it's easy to compare them. If the two values differ, then either the private or public key doesn't match, or there's a bug in nettle, or there's some miscomputation, possibly caused by some fault injection attack.
At least for some applications, it makes sense to always verify that a computed signature is valid before passing it on. This is pretty important, e.g., if using RSA in a setting where fault injection attacks are relevant, and some of Nettle' too many RSA signing functions have a check for that.
As far as I understand, SLH-DSA is not that vulnerable to fault injection, but it may still be good practice to check that a computed signature is valid, in particular if one gets it for free when signing.
So the options I see:
1. Keep assert, let it be caller's responsibilty to make sure that the public and private keys passed to the signing function are consistent.
2. Just delete the assert.
3. Drop the assert, but instead introduce a success/fail return value. Applications that care can check that; if one believes fault injection and the like isn't a relevant threat to one's application or to SLH-DSA in general, it seems fine to ignore the return value.
4. Have two flavors of signing functions, one with the assert, one with the return value.
Any opinions?
I'm currently leaning towards (3). In principle, changing return type from void to int is an ABI change, but I'm not aware of any platform where it actually is an ABI breakage.
Regards, /Niels