I'm thinking about how to represent private keys. The context is the desire to support RSA blinding. I'd like to avoid having to duplicate all the signature functions (of which there's already a large number). This email is an expansion of a 3.0 entry in plan.txt.
One way would be to pass a function pointer (and some optional ctx) for doing the private key operation. This would then be used wherever the current code calls _rsa_compute_root, and it would represent an RSA private key oracle.
If we stick to software implementation, one could pass it as an extra argument, or one could extend the private key struct like
struct rsa_private_key { mpz_t d; ... rsa_compute_root_func *f; };
and one could then "subclass" that as
struct rsa_private_key_blinding { struct rsa_private_key super; struct whatever_random_ctx_needed ctx; };
(not sure this is the best design, but it's one possibility).
Now I have a question: Would it be feasible to support smartcards (and similar hardware) at the same time? Then the secret key wouldn't be available at all to nettle, and a signature function might have an interface like
int rsa_sha1_sign(rsa_compute_root_func *f, void *private, struct sha1_ctx *hash, mpz_t signature);
where the actual private key data is totally dependent on the smartcard and its interface software.
But what interfaces do typical smartcards use? Do they allow you to compute an arbitrary rsa root, or do they only more specific operations like "pkcs#1 v1.5 decrypt", "pkcs#1 v1.5 sha1 sign", #pkcs#1 v2.0 sha256 sign"?
I want to figure out if smartcard support at this level is at all feasible. If not, we have to leave that for applications and libraries on top of nettle, and then the nettle interface should be tailored for software keys only.
I've also had a quick look at the ssh-agent and gpgagent protocols. Both seems a bit too application specific, so I don't think it's feasible to hook in those as general "private key oracles" here.
Regards, /Niels