[
https://issues.apache.org/jira/browse/HADOOP-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14052008#comment-14052008
]
Colin Patrick McCabe commented on HADOOP-10734:
-----------------------------------------------
bq. \[discussion of randomness methods\]
I don't think {{/dev/random}} is a practical choice, due to the blocking issue.
From what I've heard, {{/dev/urandom}} can often be a good choice. I'm
tempted to try a super-simple piece of code that just periodically fills a big
buffer from {{/dev/urandom}} and see if that performs well. I have a hunch
that it would (and it would also use RDRAND on supported platforms.) But I
think it's fine if you want to provide an option to go through openssl as
well... we already have a dependency on that library.
bq. And we can also add enable flag in configuration and user can disable it.
I agree. I think we should have a configuration option like
{{encryption.random.number.generator}} which specifies a comma-separated list
of class names to try to use. That way a user could specify the openssl one
plus a fallback to the standard java one if they so chose. Or alternately, the
user could enable just the java one (and configure it to use /dev/urandom) to
get something which used RDRAND plus some additional randomness. If you want
to do this in a follow-on JIRA, that's OK too.
I think it's confusing that we have both
{{org.apache.hadoop.crypto.random.SecureRandom}} and
{{java.security.SecureRandom}}. Maybe a better name for this new class would
be {{OpenSslSecureRandom}} or something like that, to emphasize that it is
using OpenSSL to get random bits.
{code}
+/**
+ * Utilize RdRand to return random numbers from hardware random number
+ * generator. It's TRNG(True Random Number generators) having high
performance.
+ * https://wiki.openssl.org/index.php/Random_Numbers#Hardware
+ * http://en.wikipedia.org/wiki/RdRand
+ */
+static ENGINE * rdrand_init(JNIEnv *env)
{code}
I think the comment is a bit misleading here. Openssl compiles on a lot of
platforms that don't have RDRAND. So all we really know here is that we're
using openssl, not that we're using RDRAND. I think it's appropriate to have a
comment saying, "if you are using an Intel chipset with RDRAND, the
high-performance random number generator will be used", or something like that.
But it's platform specific and we may be compiling on another platform.
{code}
+ \@Test(timeout=120000)
+ public void testRandomInt() throws Exception {
+ SecureRandom random = new SecureRandom();
+
+ int rand1 = random.nextInt();
+ int rand2 = random.nextInt();
+ Assert.assertFalse(rand1 == rand2);
+ }
{code}
It's definitely difficult to test something which is returning true random
numbers. It requires a lot of mathematics. So I see why you did it this way.
Just one comment... maybe I'm being overly paranoid here, but can we loop until
rand2 is not equal to rand1?
bq. I suppose you mean direct ByteBuffer. Per my understanding, merit of direct
ByteBuffer is to avoid bytes copy. But SecureRandom#nextBytes will accept an
pre-allocated byte[] array, if we use direct ByteBuffer for JNI, then there is
additional copy in java layer, so the performance is the same, and we need to
manage the direct ByteBuffer.
OK.
bq. Can you explain a bit more, I’m not sure I get your meaning. Per my
understanding, pthread_t is defined in /usr/include/bits/pthreadtypes.h as
The stuff in {{/usr/include/bits}} is not public; it is an implementation
detail that could change at any time.
from {{man pthread_self}}:
bq. POSIX.1 allows an implementation wide freedom in choosing the type used to
represent a thread ID; for example, representation using either an arithmetic
type or a structure is permitted. Therefore, variables of type pthread_t
can't portably be compared using the C equality operator (==); use
pthread_equal(3) instead. Thread identifiers should be considered opaque: any
attempt to use a thread ID other than in pthreads calls is nonportable and can
lead to unspecified results.
> Implementation of true secure random with high performance using hardware
> random number generator.
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10734
> URL: https://issues.apache.org/jira/browse/HADOOP-10734
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
> Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
> Reporter: Yi Liu
> Assignee: Yi Liu
> Fix For: fs-encryption (HADOOP-10150 and HDFS-6134)
>
> Attachments: HADOOP-10734.patch
>
>
> This JIRA is to implement Secure random using JNI to OpenSSL, and
> implementation should be thread-safe.
> Utilize RdRand to return random numbers from hardware random number
> generator. It's TRNG(True Random Number generators) having much higher
> performance than {{java.security.SecureRandom}}.
> https://wiki.openssl.org/index.php/Random_Numbers
> http://en.wikipedia.org/wiki/RdRand
> https://software.intel.com/en-us/articles/performance-impact-of-intel-secure-key-on-openssl
--
This message was sent by Atlassian JIRA
(v6.2#6252)
