[
https://issues.apache.org/jira/browse/HADOOP-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051500#comment-14051500
]
Yi Liu commented on HADOOP-10734:
---------------------------------
[~cmccabe], thanks for the review :-).
{quote}
I actually have the same problem with the scheme here: JNI calls are
expensive... do we know how many random bits the API user is getting at a time?
If that number is small, we might want to implement batching.
{quote}
In most cases, we use it to generate key(16bytes, 32bytes, 128bytes, 256bytes),
IV(16 bytes), long (8 bytes).
Furthermore, to make the random bytes good enough, we can’t avoid JNI, even
{{java.security.SecureRandom}} also uses JNI.
{quote}
I also think we should consider using ByteBuffer rather than byte[] array, if
performance is the primary goal.
{quote}
I suppose you mean direct ByteBuffer. Per my understanding, merit of direct
ByteBuffer is to avoid bytes copy. But {{SecureRandom#nextBytes}} will accept
an pre-allocated byte[] array, if we use direct ByteBuffer for JNI, then there
is additional copy in java layer, so the performance is the same, and we need
to manage the direct ByteBuffer.
{quote}
{code}
+ final protected int next(int numBits) {
{code}
Should be private
{quote}
OK, I will update it.
{quote}
{code}
+ public long nextLong() {
+ return ((long)(next(32)) << 32) + next(32);
+ }
{code}
Why use addition rather than bitwise OR here?
{quote}
Bitwise OR is also OK. Actually {{nextLong}}, {{nextFloat}} and {{nextDouble}}
are copied from implementations in {{java.security.SecureRandom}}
{quote}
This is not correct. The type of {{pthread_t}} is not known. If you want a
numeric thread ID, you could try gettid on Linux.
{quote}
Can you explain a bit more, I’m not sure I get your meaning. Per my
understanding, {{pthread_t}} is defined in {{/usr/include/bits/pthreadtypes.h}}
as
{code}
typedef unsigned long int pthread_t;
{code}
And this patch is compiled and run successfully on my Linux server.
> Implementation of true secure random with high performance using hardware
> random number generator.
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10734
> URL: https://issues.apache.org/jira/browse/HADOOP-10734
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: security
> Affects Versions: fs-encryption (HADOOP-10150 and HDFS-6134)
> Reporter: Yi Liu
> Assignee: Yi Liu
> Fix For: fs-encryption (HADOOP-10150 and HDFS-6134)
>
> Attachments: HADOOP-10734.patch
>
>
> This JIRA is to implement Secure random using JNI to OpenSSL, and
> implementation should be thread-safe.
> Utilize RdRand to return random numbers from hardware random number
> generator. It's TRNG(True Random Number generators) having much higher
> performance than {{java.security.SecureRandom}}.
> https://wiki.openssl.org/index.php/Random_Numbers
> http://en.wikipedia.org/wiki/RdRand
> https://software.intel.com/en-us/articles/performance-impact-of-intel-secure-key-on-openssl
--
This message was sent by Atlassian JIRA
(v6.2#6252)
