At some point I 'simplified' the divide code to always run the

division algorithm unless x == y, but this could result in n - t + 1
being negative which would cause an attempt to allocate about 4
gigabytes of memory. Fix this, and also add an assertion check in
the code to ensure that can't happen in any other way.

Never reproduced this with 32 bit digits but it would show up if the build
used 8 or 16 bit words.

1 files changed, 5 insertions, 1 deletions

diff --git a/src/math/bigint/divide.cpp b/src/math/bigint/divide.cpp
index 47df1273a..2eb7538db 100644
--- a/src/math/bigint/divide.cpp
+++ b/src/math/bigint/divide.cpp
@@ -38,6 +38,7 @@ void divide(const BigInt& x, const BigInt& y_arg, BigInt& q, BigInt& r)
 
    BigInt y = y_arg;
    const u32bit y_words = y.sig_words();
+
    r = x;
    q = 0;
 
@@ -51,7 +52,7 @@ void divide(const BigInt& x, const BigInt& y_arg, BigInt& q, BigInt& r)
       q = 1;
       r = 0;
       }
-   else
+   else if(compare > 0)
       {
       u32bit shifts = 0;
       word y_top = y[y.sig_words()-1];
@@ -61,6 +62,9 @@ void divide(const BigInt& x, const BigInt& y_arg, BigInt& q, BigInt& r)
 
       const u32bit n = r.sig_words() - 1, t = y_words - 1;
 
+      if(n < t)
+         throw Internal_Error("BigInt division word sizes");
+
       q.get_reg().resize(n - t + 1);
       if(n <= t)
          {