Modify Testsuite_RNG slightly to avoid rotations

[Since I want to make rotate.h internal in 3.0]

During modification of Testsuite_RNG some hard to debug test failures
occurred. It turned out to be because on occasion, with a sufficiently
bad test RNG, you can end up with p == q during RSA key generation.
Check for this.

Also add a smoke test checking that the test RNG is producing roughly
uniform output.

1 files changed, 7 insertions, 0 deletions

diff --git a/src/lib/pubkey/rsa/rsa.cpp b/src/lib/pubkey/rsa/rsa.cpp
index bce6fae0f..96f405892 100644
--- a/src/lib/pubkey/rsa/rsa.cpp
+++ b/src/lib/pubkey/rsa/rsa.cpp
@@ -291,6 +291,10 @@ RSA_PrivateKey::RSA_PrivateKey(RandomNumberGenerator& rng,
       // TODO could generate primes in thread pool
       p = generate_rsa_prime(rng, rng, p_bits, e);
       q = generate_rsa_prime(rng, rng, q_bits, e);
+
+      if(p == q)
+         throw Internal_Error("RNG failure during RSA key generation");
+
       n = p * q;
       } while(n.bits() != bits);
 
@@ -323,6 +327,9 @@ bool RSA_PrivateKey::check_key(RandomNumberGenerator& rng, bool strong) const
    if(get_p() * get_q() != get_n())
       return false;
 
+   if(get_p() == get_q())
+      return false;
+
    if(get_d1() != ct_modulo(get_d(), get_p() - 1))
       return false;
    if(get_d2() != ct_modulo(get_d(), get_q() - 1))