Re: urandom vs haveged

[Chris Murphy]

On Apr 1, 2012, at 4:41 AM, Glen Turner wrote:
> Keeping a large sample on permanent storage of
> "random numbers" generated by that very machine is providing a very
> large lever to push against any flaw.

So you're suggesting it's better to /dev/zero the disk than /dev/urandom the 
disk?

What about ATA Secure Erase, or Enhance Secure Erase? None of this comes up in 
best practices, although it is certainly the only possible way (not guaranteed, 
but dd is certain to fail) to remove user data from presently unassigned LBA on 
either HDD or SSD.

Best as I can tell, ATA Secure Erase writes zeros. Enhanced Secure Erase writes 
a "pattern" defined by the disk manufacturer. In either case, while the 
encrypted data start/end is likely locatable, unlike if good random data were 
written first, it should at least remove user data in both reserved (or 
removed) blocks and LBA assigned blocks.

Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Glen Turner]

The risk is reading unused blocks using the drive's hardware. Those
unused blocks may contain user data, operating system state, or a covert
channel allowing data or state to be inferred.

The response is to overwrite all of the disk with some value.

The random number generator is a higher risk means to provide that value
than writing a fixed value.

Firstly, it is difficult to test that the operation has succeeded.
Whereas the operation of writing a fixed value is simple to verify.

Secondly, the operation of the random number generator itself is
difficult to test.

In general, non-cryptographers see random numbers as some sort of magic
sauce whereas cryptographers see "random numbers" as a lever to crack
open the machine state. Random numbers are invaluable for forcing
attackers to search an entire state. But where they are not needed they
should not be used, since if you don't provide a lever than an attacker
can't push against it. Keeping a large sample on permanent storage of
"random numbers" generated by that very machine is providing a very
large lever to push against any flaw.

-- 
 Glen Turner 

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Richard W.M. Jones]

On Fri, Mar 30, 2012 at 06:06:02PM -0400, Paul Wouters wrote:
> It's sad that even old cheap VIA CPUs have such a strong random device,
> that's fully supported with Linux, but that Intel and AMD still haven't
> caught up yet. My 3 week old intel cpu still seems to be lacking support
> for anything (like intel-rng.ko). A few years ago, I had a server that
> supported the intel-rng driver, and rngd kept dropping zeroes and
> logging warnings. I've never ever gotten a single warning from a VIA
> CPU.

These are fun ...

https://rwmj.wordpress.com/2010/08/04/usb-hardware-random-number-generator/

# ekeydctl stats 1
BytesRead=71732957056
BytesWritten=4912615
ConnectionNonces=272923
ConnectionPackets=1120827454
ConnectionRekeys=272923
ConnectionResets=1
ConnectionTime=15581455
EntropyRate=18366
FipsFrameRate=1.53
FrameByteLast=71732956992
FramesOk=1120827454
FramingErrors=0
KeyDbsdShannonPerByteL=2.95
KeyDbsdShannonPerByteR=2.01
KeyEnglishBadness=No failure
KeyRawBadness=0
KeyRawShannonPerByteL=2.94
KeyRawShannonPerByteR=2.87
KeyRawShannonPerByteX=2.95
KeyShortBadness=efm_ok
KeyTemperatureC=29.45
KeyTemperatureF=85.01
KeyTemperatureK=302.6
KeyVoltage=3.284
PacketErrors=1
PacketOK=1120827453
ReadRate=36829
TotalEntropy=35772449856
WriteRate=2

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Paul Wouters]

On Fri, 30 Mar 2012, Steve Grubb wrote:


Something else I'd like to mention is that during system installation there is
very little system entropy. There is no saved seed to prime the generators with.
(LiveCD's have the same problem.) I have a feeling that the randomness of the
numbers is not what you would expect.


Exactly. This is why daemons generating keys (opensshd, sendmail, openswan)
generate their keys on "first start" and not on "install".


entropy. But if you don't have a mouse and are doing a text or kickstart
install, you need to find a way to get keystrokes involved. If you can think of 
a
key that has no effect on any questions in the install, hit it a bunch of times.
If you have a kickstart, put something in the script requiring typing a bunch of
keystrokes and throw them away.


Or if it is a net install, you can try and ping (-f) the machine for a
little while and see if the network card or interrupts gives you
entropy. Though that does not seem to be the case for virtual network adaptors.

It's sad that even old cheap VIA CPUs have such a strong random device,
that's fully supported with Linux, but that Intel and AMD still haven't
caught up yet. My 3 week old intel cpu still seems to be lacking support
for anything (like intel-rng.ko). A few years ago, I had a server that
supported the intel-rng driver, and rngd kept dropping zeroes and
logging warnings. I've never ever gotten a single warning from a VIA
CPU.

Paul

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[David Sommerseth]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 30/03/12 21:01, Przemek Klosowski wrote:
> On 03/30/2012 01:02 PM, David Sommerseth wrote:
>> 
>> That sounds reasonable.  Unless I mix /dev/random and
>> /dev/urandom, the latter blocks until it has filled up the
>> entropy pool again.
>> 
> Eh, the FORMER (/dev/random) blocks until it has filled up the
> entropy pool again. I seem to remember that urandom is a PRNG
> seeded with data from /dev/random; it never blocks.

Right!  I too often mix those two.  Thanks for the correction.


kind regards,

David Sommerseth
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk92GqEACgkQIIWEatLf4Hc1ewCgxoOLNEvXjKvIw9zr2/UjsrYL
UecAn3Ssyfvmo1JTfBA3nVv9cDjv67M5
=qVJL
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Przemek Klosowski]

On 03/30/2012 01:02 PM, David Sommerseth wrote:


That sounds reasonable.  Unless I mix /dev/random and /dev/urandom,
the latter blocks until it has filled up the entropy pool again.

Eh, the FORMER (/dev/random) blocks until it has filled up the entropy 
pool again. I seem to remember that urandom is a PRNG seeded with data 
from /dev/random; it never blocks.


--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[David Sommerseth]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 26/03/12 21:56, Chris Murphy wrote:
> 
> Performance:
> 
> dd if=/dev/zero   ~56MB/s CPU < 10% dd if=/dev/urandom
> ~12MB/s
> CPU 99% haveged   ~54MB/s CPU < 25%
> 
> 
> The dd relative values are consistent with kernels in Fedora 16. 
> However these tests were done with 3.3.0-1. The questions are:
> 
> Is the urandom performance expected?

That sounds reasonable.  Unless I mix /dev/random and /dev/urandom,
the latter blocks until it has filled up the entropy pool again.

> What is the quality of pseudo-random data produced by urandom vs 
> haveged?

PolarSSL used havege in v1.0 and below.  It used RDTSC as a source for
seeding the randomisation.  However, it turned out that in some
virtualised environment, the RDTSC values was quite easy to predict.
I'm not sure if I mix it with another issue, but I believe I remember
some reporting it to constantly be seeded with 0.

Havege is in otherwords something to play carefully with.  If used
together with other randomisation sources or on bare metal, it's okay.

Kind of interesting, as LWN had an article pointing at this blog post
today <http://rusty.ozlabs.org/?p=265> ... and yesterday the havege
implementation in OpenVPN when using PolarSSL was discussed in the
developers meeting.  As a side note, OpenVPN decided to deprecate
PolarSSL versions below v1.1, thus enforcing DRBG as a replacement for
havege, due to the mentioned reasons.  (Using OpenSSL, nothing changes
btw.)

> If the qualities are similar, or haveged's is better, is there 
> anything that can be done to improve urandom's performance? It
> really takes quite a bit longer to prepare a disk/volume for
> encryption.

The reason /dev/urandom is experienced as slow is because it tries to
ensure a certain level of randomness.  That's also a device provided
by the kernel.

While havege and other rngd's are probably faster as they can use more
sources for entropy generation and prepare bigger entropy pools than
what's default in the kernel space.

But as mentioned, be careful with havege.  It might not be as random
as you'd expect.


kind regards,

David Sommerseth
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk915wcACgkQIIWEatLf4HerZQCglA4QOSgRoIR8FwSMBBfR52Su
PNgAoKzFupvU2MwuFHty+3cDiUJJPGnv
=JN6G
-END PGP SIGNATURE-
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Steve Grubb]

On Monday, March 26, 2012 03:56:43 PM Chris Murphy wrote:
> Performance:
> 
> dd if=/dev/zero   ~56MB/s CPU < 10%
> dd if=/dev/urandom~12MB/s CPU 99%
> haveged   ~54MB/s CPU < 25%
> 
> 
> The dd relative values are consistent with kernels in Fedora 16. However
> these tests were done with 3.3.0-1. The questions are:
> 
> Is the urandom performance expected?

I get this:

# dd if=/dev/zero of=/dev/null 
4775272+0 records in
4775271+0 records out
2444938752 bytes (2.4 GB) copied, 4.12342 s, 593 MB/s

# dd if=/dev/urandom of=/dev/null 
118512+0 records in
118511+0 records out
60677632 bytes (61 MB) copied, 8.0117 s, 7.6 MB/s

On ^^  quadcore using 2.6.35.14 kernel.

I would say this is somewhat expected because /dev/zero does nothing while 
/dev/urandom stirs in system entropy and hashes it before letting it out. 


> What is the quality of pseudo-random data produced by urandom vs haveged?

The quality of urandom is very good. Its studied every couple years for common 
criteria purposes. Haveged on the other is never used in common criteria and 
its 
properties are largely unknown. From its home page:

HAVEGE (HArdware Volatile Entropy Gathering and Expansion) is a user-level 
software unpredictable random number generator for general-purpose computers 
that exploits these modifications of the internal volatile hardware states as a 
source of uncertainty

Unpredictable means someone needs to do a lot of study to determine if there 
are 
predictable cycles to it. Does it have scheduler artifacts in its numbers? What 
if the hardware its using is not available during system installation? Does it 
work on all platforms? Does it do any conditioning of its entropy sources? Does 
it quality check its numbers before sending them out?

 
> If the qualities are similar, or haveged's is better, is there anything
> that can be done to improve urandom's performance?

Possibly. I don't know if anyone has looked at making it faster or studied 
where 
the bottleneck is. It does produce very high quality numbers and its well 
studied, though. That has always been the prime focus.

Something else I'd like to mention is that during system installation there is 
very little system entropy. There is no saved seed to prime the generators 
with. 
(LiveCD's have the same problem.) I have a feeling that the randomness of the 
numbers is not what you would expect. If you have a mouse attached and are 
doing 
a graphical install, then waving the mouse around will make sure you have 
entropy. But if you don't have a mouse and are doing a text or kickstart 
install, you need to find a way to get keystrokes involved. If you can think of 
a 
key that has no effect on any questions in the install, hit it a bunch of 
times. 
If you have a kickstart, put something in the script requiring typing a bunch 
of 
keystrokes and throw them away.

In a way, if encrypted disks are being created at install time, Anaconda might 
want to measure entropy before creating the keys and optionally allow you to 
add 
keystrokes or wave the mouse around or startup rngd to gather entropy from a 
tpm 
chip or rdrand instructions.

-Steve
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Pádraig Brady]

On 03/26/2012 11:55 PM, Chris Murphy wrote:
> 
> 
> On Mar 26, 2012, at 4:31 PM, Pádraig Brady wrote:
>>
>> Well if you're just writing huge amounts of "random" data
>> to clear existing space, then you don't need it to be cryptographically 
>> secure.
>> Why are you doing this exactly? Would /dev/zero suffice?
> 
> In every supposed best practice case of dm-crypt LUKS setup, urandom is used 
> by example. Including by Red Hat and Fedora Projects. The Fedora link says: 
> "You're looking at a process that takes many hours, but it is imperative to 
> do this in order to have good protection against break-in attempts. Just let 
> it run overnight."
> 
> http://www.redhat.com/summit/2011/presentations/summit/taste_of_training/wednesday/Strickland_On_Disk_Encryption_with_RHEL.pdf
> 
> http://fedoraproject.org/wiki/Implementing_LUKS_Disk_Encryption
> 
> http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sect-Security_Guide-LUKS_Disk_Encryption-Manually_Encrypting_Directories-Step_by_Step_Instructions.html
> 
> So then the question is, if urandom is what's recommended, are faster 
> substitutes just as good? If they are just as good, then why aren't they the 
> first recommendation? And if this step is superfluous, then I'd suggest 
> documentation be changed to eliminate the suggestion altogether.

Well I can only think of two reasons for writing "random" data.
1. Ensure all existing data is overwritten (zeros would do just as well on 
modern drives)
2. Homogenize written (with luks data) and nonwritten parts of the drive,
so that you can't determine the extent of the real encrypted data.

I think `shred -v -n1 /your/device` is fine for this:
http://burtleburtle.net/bob/rand/isaacafa.html

cheers,
Pádraig.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Roberto Ragusa]

On 03/27/2012 05:23 AM, Gregory Maxwell wrote:
> On Mon, Mar 26, 2012 at 6:55 PM, Chris Murphy  wrote:
>> So then the question is, if urandom is what's recommended, are faster 
>> substitutes just as good? If they are just as good, then why aren't they the 
>> first recommendation? And if this step is superfluous, then I'd suggest 
>> documentation be changed to eliminate the suggestion altogether.
> 
> Personally, I setup dmcrypt (w/o luks) first using /dev/urandom as the
> key and one of the secure block modes (e.g. aes-lrw or aes-essiv).
> Then I fill the dmcrypt device with /dev/zero.  This goes fairly fast,
> filling the device with securely encrypted zeros.
> 
> Then I drop the volume and set up luks normally.

Nice trick. Does this saturate the disk speed?

Last time I had to do this I compiled my own random generator,
using some code from a research article.
That was fast C code, when compiled for x86_64 with good gcc
options the speed (>/dev/null) was 1.75GB/s (!!!).


-- 
   Roberto Ragusamail at robertoragusa.it
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Gregory Maxwell]

On Mon, Mar 26, 2012 at 6:55 PM, Chris Murphy  wrote:
> So then the question is, if urandom is what's recommended, are faster 
> substitutes just as good? If they are just as good, then why aren't they the 
> first recommendation? And if this step is superfluous, then I'd suggest 
> documentation be changed to eliminate the suggestion altogether.

Personally, I setup dmcrypt (w/o luks) first using /dev/urandom as the
key and one of the secure block modes (e.g. aes-lrw or aes-essiv).
Then I fill the dmcrypt device with /dev/zero.  This goes fairly fast,
filling the device with securely encrypted zeros.

Then I drop the volume and set up luks normally.

From a security perspective an attack which allowed the attacker to
distinguish the randomly encrypted /dev/zero from your other data
would be a fairly bad vulnerability generally against the encrypted
volume... much worse than the information leak wrt used blocks.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Chris Murphy]

On Mar 26, 2012, at 4:31 PM, Pádraig Brady wrote:
> 
> Well if you're just writing huge amounts of "random" data
> to clear existing space, then you don't need it to be cryptographically 
> secure.
> Why are you doing this exactly? Would /dev/zero suffice?

In every supposed best practice case of dm-crypt LUKS setup, urandom is used by 
example. Including by Red Hat and Fedora Projects. The Fedora link says: 
"You're looking at a process that takes many hours, but it is imperative to do 
this in order to have good protection against break-in attempts. Just let it 
run overnight."

http://www.redhat.com/summit/2011/presentations/summit/taste_of_training/wednesday/Strickland_On_Disk_Encryption_with_RHEL.pdf

http://fedoraproject.org/wiki/Implementing_LUKS_Disk_Encryption

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Security_Guide/sect-Security_Guide-LUKS_Disk_Encryption-Manually_Encrypting_Directories-Step_by_Step_Instructions.html

So then the question is, if urandom is what's recommended, are faster 
substitutes just as good? If they are just as good, then why aren't they the 
first recommendation? And if this step is superfluous, then I'd suggest 
documentation be changed to eliminate the suggestion altogether.

Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: urandom vs haveged

[Pádraig Brady]

On 03/26/2012 08:56 PM, Chris Murphy wrote:
> 
> Performance:
> 
> dd if=/dev/zero   ~56MB/s CPU < 10%
> dd if=/dev/urandom~12MB/s CPU 99%
> haveged   ~54MB/s CPU < 25%
> 
> 
> The dd relative values are consistent with kernels in Fedora 16. However 
> these tests were done with 3.3.0-1. The questions are:
> 
> Is the urandom performance expected?
> 
> What is the quality of pseudo-random data produced by urandom vs haveged?
> 
> If the qualities are similar, or haveged's is better, is there anything that 
> can be done to improve urandom's performance? It really takes quite a bit 
> longer to prepare a disk/volume for encryption.

Well if you're just writing huge amounts of "random" data
to clear existing space, then you don't need it to be cryptographically secure.
Why are you doing this exactly? Would /dev/zero suffice?
In any case it seems you could use shred rather than dd to clear data?
It has been changed to use a much faster internal generator:

http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=commit;h=v7.2-21-gaf5723c

cheers,
Pádraig.
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

urandom vs haveged

[Chris Murphy]

Performance:

dd if=/dev/zero ~56MB/s CPU < 10%
dd if=/dev/urandom  ~12MB/s CPU 99%
haveged ~54MB/s CPU < 25%


The dd relative values are consistent with kernels in Fedora 16. However these 
tests were done with 3.3.0-1. The questions are:

Is the urandom performance expected?

What is the quality of pseudo-random data produced by urandom vs haveged?

If the qualities are similar, or haveged's is better, is there anything that 
can be done to improve urandom's performance? It really takes quite a bit 
longer to prepare a disk/volume for encryption.


Chris Murphy
-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel