Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-30 Thread Kevin J. McCarthy

On Sat, May 30, 2020 at 11:44:55PM +0200, Gero Treuner wrote:

According to the documents available in the net, entropy is not
influenced by consuming random numbers. Entropy is used to contantly
(re)reseeding the PRNG, and if insufficient entropy is present, you
might get imperfect random numbers (we don't care much for our purposes).

This could happen right after boot. On Linux this is commonly mitigated
by seeding with saved random from the previous system run. FreeBSD is
reported to wait before emitting random numbers until the entropy level
is reached.

Both, OpenSSL and GnuTLS source /dev/urandom, which doesn't block.


Thanks Gero.  What I'm a bit unclear about is the effect reading from 
/dev/urandom has on /dev/random users.  Some peeking around seems to 
indicate that (at least at one time) they read from the same "pool", but 
when the pool runs low urandom emits "less-random" values, whereas 
/dev/random blocks.  Doesn't this mean there is an effect from using 
/dev/urandom for /dev/random users?


Also, FreeBSD apparently doesn't distinguish between the two.  Does 
GnuTLS and OpenSSL take care to not block in that case?


Is this something we *really* want to do for tempfiles and boundaries?

That said, this is also *really* something I don't care deeply about, 
and I hate to block your and Remco's efforts on this just because it's 
low priority for me.  Would you be willing to collaborate with Remco to 
come up with patches for a random api and message-id generator that you 
both approve of?  If it's something that passes your review, I will 
commit it.


Thank you.

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-30 Thread Steffen Nurpmeso
Ian Collier wrote in
<20200530212040.gk1301...@cs.ox.ac.uk>:
 |On Mon, May 25, 2020 at 04:24:41PM +0200, Oswald Buddenhagen wrote:
 |> why not do something proper and use getentropy() instead?
 |
 |It's been previously suggested on here that a mail client shouldn't
 |consume entropy from the system each time it starts, because other
 |more important processes may want it.

As a non-mathematician i happily disagree and claim you cannot
"consume" entropy of a random pool that is stirred and mixed and
which mixes, where all this mixing is indeed done via
cryptographically advanced algorithms, itself into another pool
that finally serves you (whether directly but especially when also
being served indirectly via such an algorithm, which i think is
what now is done by all; NetBSD has gained a(gain an even more
improvied) tremendous amount of work just a few days/fewest weeks
ago, for example).

For elder code the MUA i maintain also plays fair with the old
per-user entropy ~/.rnd (by default) as managed via OpenSSL, in
that, if every program that uses that stirs the pool and saves the
file again, as we do (as is or was documented that it should be
done like that i think), then every program startup and usage even
"increases entropy", or, how i would say it, increases
unpredictability of the actual entropy data.

I mean, you have regular interrupts and timers and scheduler time
slices and system calls of programs and network traffic and other
I/O events, and all that stirs the pool a bit.  But like i said,
i am not a mathematician, i always wondered how such attacks can
work out at all, yet some did.  But then again we saw bugs in the
past like that only some low bits of time counters were used and/
or that no mixing through the entire pool was done etc.  Iirc.
That is me who only reads such things with only one eye.
But for example i wondered already about twenty years ago why SSL
was not used for some things, and i now wonder even more why
everybody wants to use it for exactly the very same things.  As if
the world had changed.  Hm.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-30 Thread Gero Treuner
Hi Kevin,

> I'm not thinking about performance issues, but about the use and possible
> misuse of entropy.  I'm not even convinced it's appropriate for Message-ID,
> so I certainly wouldn't like to see it put in a generic function used for
> temp files and message part boundaries.

According to the documents available in the net, entropy is not
influenced by consuming random numbers. Entropy is used to contantly
(re)reseeding the PRNG, and if insufficient entropy is present, you
might get imperfect random numbers (we don't care much for our purposes).

This could happen right after boot. On Linux this is commonly mitigated
by seeding with saved random from the previous system run. FreeBSD is
reported to wait before emitting random numbers until the entropy level
is reached.

Both, OpenSSL and GnuTLS source /dev/urandom, which doesn't block.


See

Recommendations for Randomness in the Operating System
  http://www.cs.columbia.edu/~suman/docs/hotos15recommendations.pdf

/dev/random vs /dev/urandom and are they secure?
  https://linuxhint.com/dev_random_vs_dev_urandom/

Myths about /dev/urandom
  https://www.2uo.de/myths-about-urandom/


Earlier Kernels

Analysis of the Linux Random Number Generator
  https://eprint.iacr.org/2006/086.pdf
"Our study is based on version 2.6.10 of the Linux kernel, which was
released on December 24, 2004."
"The/dev/urandominterface, and the kernel interface (get_random_bytes),
return any number of pseudo-random bits, according tothe request. This
difference implies that entropy estimation is important mainly for
the/dev/randominterface."

An implementation of the Yarrow PRNG for FreeBSD
  https://www.usenix.org/legacy/events/bsdcon/full_papers/murray/murray_html/


This (hopefully) is enough to destroy doubts.


Gero


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-30 Thread Ian Collier
On Mon, May 25, 2020 at 04:24:41PM +0200, Oswald Buddenhagen wrote:
> why not do something proper and use getentropy() instead?

It's been previously suggested on here that a mail client shouldn't
consume entropy from the system each time it starts, because other
more important processes may want it.

On Intel processors the TSC register is a good source of random
seeds (and I believe it's what the GNU mkstemp uses).  But coding
processor-dependent stuff such as that might be considered messy.
(For something *really* messy but not too system-dependent, call mkstemp
and use the filename it generated as your seed.)

imc


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-30 Thread Kevin J. McCarthy

On Sat, May 30, 2020 at 07:57:13PM +0200, Gero Treuner wrote:

What are performance expectations and methods for measurement?
Generate random numbers alternating with context switch (to not run
purely CPU top-level caches) such as a kernel rwad/write calls?


Hi Gero,

I'm not thinking about performance issues, but about the use and 
possible misuse of entropy.  I'm not even convinced it's appropriate for 
Message-ID, so I certainly wouldn't like to see it put in a generic 
function used for temp files and message part boundaries.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-30 Thread Gero Treuner
Hi,

On Fri, May 29, 2020 at 03:33:28PM -0700, Kevin J. McCarthy wrote:
> On Fri, May 29, 2020 at 03:58:35PM -0400, Remco Rijnders wrote:

> I think your comments in  make a good case for not
> putting crypto-level randomization into a PRNG function, and I would like to
> think about it before moving in that direction (for message-id's too).

Regarding crypto libraries an important reason to avoid it (if included
in build anyway) is if performance is not competitive. I'd expect that
the difference (impact of side-channel free coding) is not that we worry
about it. As random numbers are required only for specific, not
permanently occuring events, slight disadvantages probably are not
noticeable.

What are performance expectations and methods for measurement?
Generate random numbers alternating with context switch (to not run
purely CPU top-level caches) such as a kernel rwad/write calls?


Gero


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-29 Thread Kevin J. McCarthy

On Fri, May 29, 2020 at 03:58:35PM -0400, Remco Rijnders wrote:
On Mon, May 25, 2020 at 09:28:30AM -0700, Kevin wrote in 
<20200525162830.gd1...@afu.lan>:
Remco, sorry I missed a few of those issues.  In general you'll want 
to follow the pattern of other files:

...


Got it (I hope)!

I have made the necessary changes in message 
<20200529195152.5925-1-re...@webconquest.com>. Furthermore, in that 
patch I have now also changed it to where we use the random numbers 
from OpenSSL/LibreSSL first, if compiled with SSL support, otherwise 
with random numbers from GnuTLS, if compiled with GnuTLS support, and 
only using this new algorithm in the absence of both these options.


I am interested in hearing your thoughts.


I'll take a closer look, but honestly I would prefer that made as a 
separate patch so it can be considered separately.  (In addition to 
Oswald's comments).


I think your comments in  make a good case for 
not putting crypto-level randomization into a PRNG function, and I would 
like to think about it before moving in that direction (for message-id's 
too).


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-29 Thread Remco Rijnders
On Mon, May 25, 2020 at 09:28:30AM -0700, Kevin wrote in 
<20200525162830.gd1...@afu.lan>:
Remco, sorry I missed a few of those issues.  In general you'll want 
to follow the pattern of other files:

...


Got it (I hope)!

I have made the necessary changes in message
<20200529195152.5925-1-re...@webconquest.com>. Furthermore, in that
patch I have now also changed it to where we use the random numbers
from OpenSSL/LibreSSL first, if compiled with SSL support, otherwise
with random numbers from GnuTLS, if compiled with GnuTLS support, and
only using this new algorithm in the absence of both these options.

I am interested in hearing your thoughts.

Kind regards,

Remco


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-25 Thread Kevin J. McCarthy

On Mon, May 25, 2020 at 10:04:40AM -0400, Remco Rijnders wrote:

Having made these changes, I get build errors of the kind:


Remco, sorry I missed a few of those issues.  In general you'll want to 
follow the pattern of other files:


* Your mutt_random.h should have #defines to prevent double inclusion:
#ifndef MUTT_RANDOM_H
#define MUTT_RANDOM_H
...
#endif

* Your mutt_random.c should have at the top:
#if HAVE_CONFIG_H
# include "config.h"
#endif

Take a look at other files to see the pattern.

--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-25 Thread Remco Rijnders
On Mon, May 25, 2020 at 04:24:41PM +0200, Oswald wrote in 
<20200525142441.GC855843@ugly>:

Well, the idea for z4 is really to pick another seed than any other
user running mutt on the same system would have.

every process gets its own address space, so it will be the same in 
each process, unless the system has ASLR enabled (i have no idea what 
percentage of systems enable user space ASLR).


Thanks Oswald,

I was not aware of this and thought this would be unique between
instances. Another day, another lesson learned!

Remco


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-25 Thread Oswald Buddenhagen

On Mon, May 25, 2020 at 09:05:21AM -0400, Remco Rijnders wrote:
On Sun, May 24, 2020 at 06:49:03PM -0700, Kevin wrote in 
<20200525014903.ga1...@afu.lan>:

+  z1 = ((u_int32_t) tv.tv_sec << 20) | tv.tv_usec;
+  z2 = getpid();
+  z3 = getppid();
+  z4 = (intptr_t) &z4;
+}


Comments on the seed choices would be quite welcome.  This first three 
seem okay to me, but the fourth choice is probably a bit weak.  I 
don't know that it will vary much.


Well, the idea for z4 is really to pick another seed than any other
user running mutt on the same system would have.

every process gets its own address space, so it will be the same in each 
process, unless the system has ASLR enabled (i have no idea what 
percentage of systems enable user space ASLR).


why not do something proper and use getentropy() instead?
(the fallback on older/non-linux/-openbsd systems would be trying to 
open /dev/urandom and /dev/random in turn, and only then falling back to 
weak sources like above. the device access attempts can be skipped, as 
this is about low-stakes randomness so far.)




Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-25 Thread Remco Rijnders
On Sun, May 24, 2020 at 06:49:03PM -0700, Kevin wrote in 
<20200525014903.ga1...@afu.lan>:

diff --git a/mutt_random.h b/mutt_random.h



+#include 
+#include 
+#include 
+#include 


Mutt traditionally puts these inside the .c file.  So I'd prefer to 
see them moved there.



+extern void mutt_to_base64 (unsigned char*, const unsigned char*, size_t, 
size_t);


Include "mutt.h" inside mutt_random.c instead.


Having made these changes, I get build errors of the kind:

(mutt_random.c):
...
#include "mutt.h"
#include "mutt_random.h"
...

In file included from mutt_random.c:23:
mutt.h:89:8: error: redefinition of ‘struct timespec’
   89 | struct timespec
  |^~~~
In file included from /usr/include/sys/select.h:39,
 from /usr/include/sys/types.h:179,
 from /usr/include/stdlib.h:394,
 from mutt.h:24,
 from mutt_random.c:23:
/usr/include/bits/types/struct_timespec.h:9:8: note: originally defined here
9 | struct timespec
  |^~~~
In file included from mutt_random.c:23:
mutt.h:774:3: error: unknown type name ‘LOFF_T’
  774 |   LOFF_T offset;/* offset where the actual data begins 
*/
  |   ^~

I haven't quite figured out yet how to address this.

Regards,

Remco


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-25 Thread Remco Rijnders
On Mon, May 25, 2020 at 08:02:54AM +0200, Petr wrote in 
<20200525060254.ga3...@album.bayer.uni.cx>:

Software distributors are not happy seeing various software carrying its own
cryptographic code. Could you please keep using the system-provided functions
if available?


Hi Petr,

Thank you for your comment. While I concur with the sentiment of your
comment that in general you do not want your package to include its
own implementation of code for which good libraries are available
(such as zlib, handling png files, etc.) for reasons of
maintainability and security, I am not sure this applies here. The
proposed PRNG algorithm is very small and actually replaces a mismash
of different PRNG functions that often (dependant on the PRNG function
used and platform it is implemented on) has considerable weaknesses of
their own.

Furthermore, this code is only to be used in cases where we want a
"random enough" looking number, it doesn't have to withstand years of
crypto analysis by a skilled attacker. The code which would be
affected by this approach is currently being used for:

#1) Generation of tmp filenames. The current implementation uses two
31 bit values, with this patch this is replaced by two 32 bit values.

#2a) Generation of a boundary for MIME-attachments. The code would use
the new random function instead of the existing one. This boundary
just has to be "unique enough" so that one doesn't type it by accident
when composing an email.

#2b) One of my other patches proposes to use 96 bits of randomness
directly (=3 calls to the random function) for generating this
boundary instead of generating 16 random numbers (=16 calls to the
random function) and taking the modulo to come up with a Base64
character.

#3) Finally, I also propose to use a random number as part of the
Message ID which is currently not done. This is either a 64 bit or a
96 bit random value. An attacker would not only have to know what
random value will be generated, but also when it will be generated to
generate a colliding Message-ID. This will be much harder to do with
the inclusion of the random part than in its current implementation
where no part of the Message ID is random.

In conclusion, I feel that applying my patches will result in stronger
random numbers being used by mutt for these applications only. It is
my understanding that the chosen implementation does not leak any
information that could be used by an attacker to find out the state of
the PRNG, something which might be possible with the current
implementation(s).

One area of concern is that I haven't been able to find a lot of
credible critiques of the chosen algorithm. While I take comfort from
the fact that the algorithm is not new by any means and is developed
by an expert in the field, and I feel quite confident in the fact that
the produced numbers satisfy the need of appearing random, I would
like to have a little more comfort on the difficulty of brute forcing
it and being able to find the seeds used or the current internal state
of the algorithm.

Kind regards,

Remco


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-25 Thread Remco Rijnders
On Sun, May 24, 2020 at 06:49:03PM -0700, Kevin wrote in 
<20200525014903.ga1...@afu.lan>:

On Sun, May 24, 2020 at 04:59:24PM -0400, Remco Rijnders wrote:

By implementing the LFSR113 function by Pierre L'Ecuyer mutt gets a fast
and high quality PRNG that, given the same seeds, results in the same
output no matter the environment mutt is running on.


I'm not knowledgeable about PRNGs.  The generation code below looks 
the same as other LFSR113 generators I searched around for, but I 
can't vouch for how "good" the generator itself is.  Still, I'm 
inclined to apply the patch (with fixes) so please chime in if you 
think this is a bad idea.


I am not knowledgable in this field either, and as such I am almost
inclined to agree with Petr to not try and reinvent the wheel
myself. I will note though that Pierre L'Ecuyer is a bit of an
authority in this field and that, as far as I understand it, this
algorithm does not leak any state information as its output is based
on the XOR'ed values of four different PRNG algorithms which we also
seed independently of each other.



More comments below.


diff --git a/configure.ac b/configure.ac
index 7906ce35..0f65fc56 100644
--- a/configure.ac
+++ b/configure.ac
@@ -34,7 +34,6 @@ AC_PROG_CPP
AC_PROG_MAKE_SET
AC_PROG_INSTALL
AC_PROG_MKDIR_P
-AC_PROG_RANLIB
AC_CHECK_TOOL(AR, ar, ar)


Ranlib is a library tool, and is still used (in some way) in the intl 
and m4 directories.  Regardless, this kind of change should not be in 
this commit (unless it was a mistaken assumption that it meant "random 
library" or something).


Yes, I do fear I was mistaken here in thinking this was what defines
the HAVE_SRAND48 definition being set or not. I should not have
included this without being more certain about its function.




diff --git a/mutt_random.c b/mutt_random.c



+/* Initialize the four seeds for our PRNG algorithm */
+void mutt_srandom(void)
+{
+  struct timeval tv;
+  gettimeofday(&tv, NULL);
+  /* POSIX.1-2008 states that seed is 'unsigned' without specifying its width.
+   * Use as many of the lower order bits from the current time of day as the 
seed.
+   * If the upper bound is truncated, that is fine.
+   *
+   * tv_sec is integral of type integer or float.  Cast to 'u_int32_t' before
+   * bitshift in case it is a float.
+   */
+  z1 = ((u_int32_t) tv.tv_sec << 20) | tv.tv_usec;
+  z2 = getpid();
+  z3 = getppid();
+  z4 = (intptr_t) &z4;
+}


Comments on the seed choices would be quite welcome.  This first three 
seem okay to me, but the fourth choice is probably a bit weak.  I 
don't know that it will vary much.


Well, the idea for z4 is really to pick another seed than any other
user running mutt on the same system would have. I agree that it is
not the strongest seed, which is also true for z2 and z3 which are low
in entropy by themselves (and might be observable for others on the
same system). Given the nature of the algorithm, an attacker would have
to know all four seeds for this to have an impact though.

Other than picking different seeds, a possible mitigant could be
periodical reseeding. I am not sure how necessary this is given the
preceeding?




diff --git a/mutt_random.h b/mutt_random.h



+#include 
+#include 
+#include 
+#include 


Mutt traditionally puts these inside the .c file.  So I'd prefer to 
see them moved there.


Ok, agreed.


+extern void mutt_to_base64 (unsigned char*, const unsigned char*, size_t, 
size_t);


Include "mutt.h" inside mutt_random.c instead.


Ok!




+void mutt_srandom(void);
+u_int32_t mutt_random32(void);
+void mutt_base64_random96(char output_B64[static 17]);
+
+u_int32_t z1, z2, z3, z4;


z1-z4 should be moved inside mutt_random.c and declared static.  The 
latest gcc won't even compile duplicate global variable definitions.


Ok!

I appreciate your review and will make the recommended changes.

Kind regards,

Remco


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-24 Thread Petr Pisar
On Sun, May 24, 2020 at 04:59:24PM -0400, Remco Rijnders wrote:
> When mutt needs a random number (for example for naming the tmp files or
> generating a boundary for MIME attachments) it currently uses different
> implementation specific functions: random() and either lrand48() or rand()
> are used. Which one gets used in specific and the quality of its randomness
> is dependant on the code calling the function and the OS mutt is running
> on.
> 
> By implementing the LFSR113 function by Pierre L'Ecuyer mutt gets a fast
> and high quality PRNG that, given the same seeds, results in the same
> output no matter the environment mutt is running on.

Software distributors are not happy seeing various software carrying its own
cryptographic code. Could you please keep using the system-provided functions
if available?

-- Petr


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-24 Thread Kevin J. McCarthy

On Sun, May 24, 2020 at 04:59:24PM -0400, Remco Rijnders wrote:

By implementing the LFSR113 function by Pierre L'Ecuyer mutt gets a fast
and high quality PRNG that, given the same seeds, results in the same
output no matter the environment mutt is running on.


I'm not knowledgeable about PRNGs.  The generation code below looks the 
same as other LFSR113 generators I searched around for, but I can't 
vouch for how "good" the generator itself is.  Still, I'm inclined to 
apply the patch (with fixes) so please chime in if you think this is a 
bad idea.


More comments below.


diff --git a/configure.ac b/configure.ac
index 7906ce35..0f65fc56 100644
--- a/configure.ac
+++ b/configure.ac
@@ -34,7 +34,6 @@ AC_PROG_CPP
AC_PROG_MAKE_SET
AC_PROG_INSTALL
AC_PROG_MKDIR_P
-AC_PROG_RANLIB
AC_CHECK_TOOL(AR, ar, ar)


Ranlib is a library tool, and is still used (in some way) in the intl 
and m4 directories.  Regardless, this kind of change should not be in 
this commit (unless it was a mistaken assumption that it meant "random 
library" or something).



diff --git a/mutt_random.c b/mutt_random.c



+/* Initialize the four seeds for our PRNG algorithm */
+void mutt_srandom(void)
+{
+  struct timeval tv;
+  gettimeofday(&tv, NULL);
+  /* POSIX.1-2008 states that seed is 'unsigned' without specifying its width.
+   * Use as many of the lower order bits from the current time of day as the 
seed.
+   * If the upper bound is truncated, that is fine.
+   *
+   * tv_sec is integral of type integer or float.  Cast to 'u_int32_t' before
+   * bitshift in case it is a float.
+   */
+  z1 = ((u_int32_t) tv.tv_sec << 20) | tv.tv_usec;
+  z2 = getpid();
+  z3 = getppid();
+  z4 = (intptr_t) &z4;
+}


Comments on the seed choices would be quite welcome.  This first three 
seem okay to me, but the fourth choice is probably a bit weak.  I don't 
know that it will vary much.



diff --git a/mutt_random.h b/mutt_random.h



+#include 
+#include 
+#include 
+#include 


Mutt traditionally puts these inside the .c file.  So I'd prefer to see 
them moved there.



+extern void mutt_to_base64 (unsigned char*, const unsigned char*, size_t, 
size_t);


Include "mutt.h" inside mutt_random.c instead.


+void mutt_srandom(void);
+u_int32_t mutt_random32(void);
+void mutt_base64_random96(char output_B64[static 17]);
+
+u_int32_t z1, z2, z3, z4;


z1-z4 should be moved inside mutt_random.c and declared static.  The 
latest gcc won't even compile duplicate global variable definitions.


--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature


Re: [PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-24 Thread Remco Rijnders

Makefile.am   |  3 ++-
configure.ac  |  1 -


Please note that I am not comfortable enough with autotools to be
certain if I did this the correct way (configure.ac) and also my
change to the Makefile might be sub optimal, I am not sure.

Cheers,

Remco


[PATCH 1/3] Use LFSR113 PRNG for mutt's internal random needs

2020-05-24 Thread Remco Rijnders
When mutt needs a random number (for example for naming the tmp files or
generating a boundary for MIME attachments) it currently uses different
implementation specific functions: random() and either lrand48() or rand()
are used. Which one gets used in specific and the quality of its randomness
is dependant on the code calling the function and the OS mutt is running
on.

By implementing the LFSR113 function by Pierre L'Ecuyer mutt gets a fast
and high quality PRNG that, given the same seeds, results in the same
output no matter the environment mutt is running on.
---
 Makefile.am   |  3 ++-
 configure.ac  |  1 -
 init.c| 18 +--
 main.c|  1 -
 mutt_random.c | 64 +++
 mutt_random.h | 30 
 muttlib.c |  9 
 protos.h  | 10 
 sendlib.c |  3 ++-
 9 files changed, 104 insertions(+), 35 deletions(-)
 create mode 100644 mutt_random.c
 create mode 100644 mutt_random.h

diff --git a/Makefile.am b/Makefile.am
index 6f166d5a..f7a2816c 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -39,7 +39,8 @@ mutt_SOURCES = \
score.c send.c sendlib.c signal.c sort.c \
status.c system.c thread.c charset.c history.c lib.c \
muttlib.c editmsg.c mbyte.c \
-   url.c ascii.c crypt-mod.c crypt-mod.h safe_asprintf.c
+   url.c ascii.c crypt-mod.c crypt-mod.h safe_asprintf.c \
+   mutt_random.c
 
 nodist_mutt_SOURCES = $(BUILT_SOURCES)
 
diff --git a/configure.ac b/configure.ac
index 7906ce35..0f65fc56 100644
--- a/configure.ac
+++ b/configure.ac
@@ -34,7 +34,6 @@ AC_PROG_CPP
 AC_PROG_MAKE_SET
 AC_PROG_INSTALL
 AC_PROG_MKDIR_P
-AC_PROG_RANLIB
 AC_CHECK_TOOL(AR, ar, ar)
 
 AC_CACHE_CHECK([for GNU make],
diff --git a/init.c b/init.c
index 26f8afcf..7b0756fd 100644
--- a/init.c
+++ b/init.c
@@ -33,6 +33,7 @@
 #include "mutt_crypt.h"
 #include "mutt_idna.h"
 #include "group.h"
+#include "mutt_random.h"
 
 #if defined(USE_SSL)
 #include "mutt_ssl.h"
@@ -3424,23 +3425,6 @@ static int mutt_execute_commands (LIST *p)
   return 0;
 }
 
-static void mutt_srandom (void)
-{
-  struct timeval tv;
-  unsigned seed;
-
-  gettimeofday(&tv, NULL);
-  /* POSIX.1-2008 states that seed is 'unsigned' without specifying its width.
-   * Use as many of the lower order bits from the current time of day as the 
seed.
-   * If the upper bound is truncated, that is fine.
-   *
-   * tv_sec is integral of type integer or float.  Cast to 'long long' before
-   * bitshift in case it is a float.
-   */
-  seed = ((LONGLONG) tv.tv_sec << 20) | tv.tv_usec;
-  srandom(seed);
-}
-
 static char* mutt_find_cfg (const char *home, const char *xdg_cfg_home)
 {
   const char* names[] =
diff --git a/main.c b/main.c
index 293f7172..d5b85525 100644
--- a/main.c
+++ b/main.c
@@ -665,7 +665,6 @@ int main (int argc, char **argv, char **environ)
 
   mutt_error = mutt_nocurses_error;
   mutt_message = mutt_nocurses_error;
-  SRAND (time (NULL));
   umask (077);
 
   memset (Options, 0, sizeof (Options));
diff --git a/mutt_random.c b/mutt_random.c
new file mode 100644
index ..ab0e6a84
--- /dev/null
+++ b/mutt_random.c
@@ -0,0 +1,64 @@
+/*
+ * Copyright (C) 2020 Remco Rijnders 
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  
02110-1301, USA.
+ */
+
+#include "mutt_random.h"
+
+/* Return 32 random bits. This is an implementation of the LFSR113 PNRG 
+   algorithm by Pierre L'Ecuyer. */
+u_int32_t mutt_random32 (void)
+{
+  u_int32_t b;
+  b  = ((z1 << 6) ^ z1) >> 13;
+  z1 = ((z1 & 4294967294U) << 18) ^ b;
+  b  = ((z2 << 2) ^ z2) >> 27;
+  z2 = ((z2 & 4294967288U) << 2) ^ b;
+  b  = ((z3 << 13) ^ z3) >> 21;
+  z3 = ((z3 & 4294967280U) << 7) ^ b;
+  b  = ((z4 << 3) ^ z4) >> 12;
+  z4 = ((z4 & 4294967168U) << 13) ^ b;
+
+  return (z1 ^ z2 ^ z3 ^ z4);
+}
+
+/* Initialize the four seeds for our PRNG algorithm */
+void mutt_srandom(void)
+{
+  struct timeval tv;
+  gettimeofday(&tv, NULL);
+  /* POSIX.1-2008 states that seed is 'unsigned' without specifying its width.
+   * Use as many of the lower order bits from the current time of day as the 
seed.
+   * If the upper bound is truncated, that is fine.
+   *
+   * tv_sec is integral of type integer or float.  Cast to 'u_int32_t' before
+   * bitshift in case it is a float.
+