30-120 seconds sounds surprisingly long for ~368 attempts, do you know which part(s) the slowness comes from?

From doing MR rounds in pure Python: https://github.com/textonly/git-prime/blob/main/git-prime-co....

Should be under 5 seconds in C or C++ using gmp

No, MR in pure python is ~instantaneous for numbers of this magnitude.

From looking at the code, the overhead will be from repeatedly invoking git as a subprocess.

Have not flame graphed or even really considered optimization

Sure hope the first line of that bash script isn’t rm -rf $HOME/*

Please don’t ever suggest to anyone ever to curl a script and pipe it to bash. I’m sure this one is fine (I haven’t looked) but it’s a pretty awful idea. Only way to make it worse is to suggest slapping sudo in front.

Damn i forgot to include that. As well as exfil of all ssh keys and env files. Oh well, you can wait for the update, right?

^^

I was trying to be not-sarcastic. Is there a way I could have phrased that which wouldn’t have garnered a sarcastic, defensive response?

that ship has sailed

I think item IDs here are sequential, including comments, so you could have timed the submission to attempt to get one. More likely to get it when the site's quieter.

> Miller-Rabin primality test (40 rounds, ~10^-24 false positive rate)

It's way better than that. You are using the simplest upper bound for the false positive rate, which is 1/t^4 where t is the number of rounds. More sophisticated analysis can give better bounds.

See the paper "Average Case Error Estimates for the Strong Probable Prime Test" by Ivan Damgård, Peter Landrock and Carl Pomerance, in Mathematics of Computation Vol. 61, No. 203, Special Issue Dedicated to Derrick Henry Lehmer (Jul., 1993), pp. 177-194. Here's a PDF: https://math.dartmouth.edu/~carlp/PDF/paper88.pdf

Here are the bounds given there for t rounds testing a candidate of k bits. I'll give them as Mathematica function definitions because I happen to have them in a Mathematica notebook.

1. This one is valid for k >= 2.

  p1[k_, t_] := k^2 4^(2 - Sqrt[k])

Note this one does not depend on t, and for small k does not give a very useful bound. For 160 the bound is 0.00992742. For large k the story is different. Testing an 8192 bit number this gives a bound of 3.45661 x 10^-46. That's good enough for almost all applications so in most cases if you want an 8192 bit prime one round is good enough.

2. This one is for t = 2, k >= 88 or 3 <= t <= k/9, k >= 21.

  p2[k_, t_] := k^(3/2) 2^t t^(-1/2)  4^(2 - Sqrt[t k])

For k = 160 this is valid for 2 <= t <= 17. For t = 17 it gives 4.1 x 10^-23.

3. This one is for t >= k/9, k >= 21.

  p3[k_, t_] := 
 7/20 2^(-5 t) + 1/7 k^(15/4) 2^(-k/2 - 2 t) + 12 k 2^(-k/4 - 3 t)

For k = 160 this is valid for t >= 18. At 18 it gives 9.75 x 10^-26. At 40 it gives 1.80 x 10^-41.

4. This one is for t >= k/4, k >= 21.

  p4[k_, t_] := 1/7 k^(15/4) 2^(-k/2 - 2 t)

For k = 160 this is valid for k >= 40. At 40 it gives the same bound as p3.

So bottom line is that with your current 40 rounds your false positive rate is under 1.80 x 10^-41, considerably better than 10^-24.

If 10^-24 is an acceptable rate for this application, 18 rounds is sufficient giving a rate under 9.7 x 10^-25.

BTW, the larger the k the lower the rate. I've often seen people looking for 1024+ bit primes doing 64 or more rounds. The simplest 1/4^t bound gives 2.9 x 10^-39. OpenSSL for example does 64 for k up to 2048, and 128 for larger k.

For k = 1024 a mere 6 rounds beats that with a bound of 8.8 x 10^-41.

For k = 2048 it only takes 3 rounds to get 4.4 x 10^-41.

For k = 4096 a mere 2 sounds gives 3.8 x 10^-48.

If we had a population of 1 trillion people, each using 1000 things that needed a 4096 bit prime, and that frequently rekeyed so they needed 1000 new primes per second, and every star in the observable universe also had such a civilization consuming 4096 bit primes at that rate, and they were all using 2 rounds of Miller-Rabin, there would be around 24 false positives a year across the whole universe.

If everyone upped it to 3 rounds there would, across the whole universe, be a false positive approximately every 44 billion years.

For those that want more details on this fun rabbit hole of math, see the theory of Strong Pseudoprimes: https://en.wikipedia.org/wiki/Strong_pseudoprime

Miller-Rabin in considered a simplistic/antiquated primality algorithm in computational number theory. For a much better algorithm see the Baillie–PSW primality test: https://en.wikipedia.org/wiki/Baillie%E2%80%93PSW_primality_... . For an implementation see Math::Prime::Util on CPAN: https://metacpan.org/pod/Math::Prime::Util or one of the many others.

The P in BPSW is for Carl Pomerance mentioned in the academic paper above. According to the wikipedia article "No composite number below 2^64 (approximately 1.845 * 10^19) passes the strong or standard Baillie–PSW test" which means if git-prime used this algorithm and the nonce was below 2^64 (very likely) then it would be provably prime instead of a probabilistic prime and hence have much more "mathematical rigor".

I also agree with other comments here that git-prime would be much cleaner if it put the nonce in git commit header data instead of the message. Something like this was done Long Ago in a "for fun" git patch by Jeff King: https://lore.kernel.org/git/CACBZZX5PqYa0uWiGgs952rk2cy+QRCU...

And he even made a multi-threaded version: https://lore.kernel.org/git/20111024204737.GA25574@sigill.in...

There is a formula to calculate number of rounds in https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-5.pdf Appendix C.1. OpenSSL use it: https://github.com/openssl/openssl/blob/ee8772e3565a84fde9e2...

For 160-bit prime and security level 2^-80, 19 rounds is enough.

Thank you. This is the hn I’m here for. Probabilistic tests for definite primes still being so slimmest definitive is amazing to me. What structure are they approximating?