Zero-copy ByteBuffer-based compressor / decompressor API --------------------------------------------------------
Key: HADOOP-8148 URL: https://issues.apache.org/jira/browse/HADOOP-8148 Project: Hadoop Common Issue Type: New Feature Components: io Reporter: Tim Broberg Per Todd Lipcon's comment in HDFS-2834, " Whenever a native decompression codec is being used, ... we generally have the following copies: 1) Socket -> DirectByteBuffer (in SocketChannel implementation) 2) DirectByteBuffer -> byte[] (in SocketInputStream) 3) byte[] -> Native buffer (set up for decompression) 4*) decompression to a different native buffer (not really a copy - decompression necessarily rewrites) 5) native buffer -> byte[] with the proposed improvement we can hopefully eliminate #2,#3 for all applications, and #2,#3,and #5 for libhdfs. " The interfaces in the attached patch attempt to address: A - Compression and decompression based on ByteBuffers (HDFS-2834) B - Zero-copy compression and decompression (HDFS-3051) C - Provide the caller a way to know how the max space required to hold compressed output. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira