Thanks Bourg for these questions. >> Sorry if it sounds naive, but why not accessing the OpenSSL functions >> through a JCE provider instead of building an abstraction layer on top of >> another abstraction layer (JCE). The Apache JuiCE project was an attempt to >> implement this idea a few years ago [1]. With an OpenSSL based JCE provider >> the CryptoInput/OutputStream could probably be replaced by >> javax.crypto.CipherInput/OutputStream. As for the secure random part the >> standard java.security.SecureRandomSpi mechanism could be used to provide an >> OpenSSL based implementation. >> What do you think? This is not naïve and very valid questions. We actually struggled a long time on these points from Bourg. We in Intel was driving the community to get the full use of hardware acceleration in encryption for a long time. We tried that way before. Diceros (https://github.com/intel-hadoop/diceros) project is exactly like what you mentioned. It provides OpenSSL based JCE provider as well as Secure Random SPI.
Technically and architecturally, this approach is nicely fit what the JCE architecture targets to do. But a JCE Provider needs to be officially certificated and signed, which can only be built and distributed by the specific organization such as Intel. This will prevent its adoption in the open source community who put much their trust on Apache open source. The developer cannot build their own version unless they have Oracle certificated private key. Another issue of this approach is the need for user to configure their JRE environment with the new JCE provider through java.security file (Although this can be overcome technically by dynamic register). While Chimera approach is similar to JCE approach technically but instead as a JCD provider, it acts as a "reference it and use" library. This second value proposition for Chimera is for its stream layer and its abstraction of Input and Output interface. Bourg mentioned javax CipherInput and CipherOutputStream. The layer will support Stream, Channel input out of box and can be easily extended with new Input and Output sources. We also see the importance of CTR mode which provides advanced characteristics such as random seek features on crypto streams. This is very important for big data encryption such HDFS which needs to read at the random positions. In this layer, other valuable things can be added when needed. The utility library is needed in these area in Apache Commons whose development, enhancement and release will be much agile than JDK improve its classes. >> Another dumb question, isn't AES-NI enabled by default in Java 8 nowadays >> [2]? Do you still get a significant speed up with Chimera in this case? If >> so I think contributing the improvements to OpenJDK would be a good idea >> too, this will benefit everyone in the next Java releases. This is a very good question. JDK 6 is not optimized. JDK 7 and JDK 8 is partial optimized for some modes. We can see 17x throughput improvement for AES/CTR mode. 5x - 6x speedup for GCM mode. >> If so I think contributing the improvements to OpenJDK would be a good idea >> too, this will benefit everyone in the next Java releases. Exactly, we have been working with Oracle team to bring these optimizations in latest JDK development. In Jan 2016, we committed the optimization on CTR to OpenJDK. (https://bugs.openjdk.java.net/browse/JDK-8143925) Hope this information helps. Thanks Bourg for these insightful questions. Thanks, Haifeng -----Original Message----- From: Emmanuel Bourg [mailto:emmanuel.bo...@gmail.com] On Behalf Of Emmanuel Bourg Sent: Tuesday, February 23, 2016 6:53 PM To: Commons Developers List <dev@commons.apache.org> Cc: common-...@hadoop.apache.org Subject: Re: [crypto][chimera] Next steps Hi all, I got a quick look at the Chimera code. If I understand well it consists in: - a native interface to the OpenSSL AES & secure random functions - an abstraction layer to use the JCE or OpenSSL AES implementation - an abstraction layer to use the JCE or OpenSSL secure random - encrypting/decrypting input/output streams Sorry if it sounds naive, but why not accessing the OpenSSL functions through a JCE provider instead of building an abstraction layer on top of another abstraction layer (JCE). The Apache JuiCE project was an attempt to implement this idea a few years ago [1]. With an OpenSSL based JCE provider the CryptoInput/OutputStream could probably be replaced by javax.crypto.CipherInput/OutputStream. As for the secure random part the standard java.security.SecureRandomSpi mechanism could be used to provide an OpenSSL based implementation. What do you think? Another dumb question, isn't AES-NI enabled by default in Java 8 nowadays [2]? Do you still get a significant speed up with Chimera in this case? If so I think contributing the improvements to OpenJDK would be a good idea too, this will benefit everyone in the next Java releases. Emmanuel Bourg [1] http://incubator.apache.org/projects/juice.html [2] http://openjdk.java.net/jeps/164 --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
