Niels Möller nisse@lysator.liu.se writes:
And on failure, it might make sense to also clear (memset to zero) the signature area before returning? To ensure that no information about the private key is leaked form an incorrect signature operation, and make it a bit safer if the return value is ignored.
Suggested new paragraph for the SLH-DSA section in the manual:
+The signing operation can recompute the top-level Merkle tree root +almost for free. Nettle's SLH-DSA signing functions do that, and compare +the resulting root hash to the provided public key. If the root hash is +not as expected, the signing operation fail, and the returned signature +is cleared to all zeros before return. Failure can happen if the public +or private key has been corrupted and don't match (although not all +kinds of corruption is detected), or if some software or hardware bug +affects the computation. Checking the return value from these signing +functions is not strictly required; it's the application's choice if it +prefers to handle this error, or pass on an invalid all-zero signature.
Does this sound reasonable? I think this should be safe enough to not be considered an ABI or API break. For ABI issues, all archs I'm aware of have an ABI where a function returning void uses the same calling convention as a function returning an int with an undefined value. For the API, a call with inputs that previously hit an assert instead returns an invalid all-zero signature, and behavior for other inputs is unchanged.
Regards, /Niels