Parth Chandra created PARQUET-77:
------------------------------------
Summary: Improvements in ByteBuffer read path
Key: PARQUET-77
URL: https://issues.apache.org/jira/browse/PARQUET-77
Project: Parquet
Issue Type: Improvement
Components: parquet-mr
Reporter: Parth Chandra
Attachments: parquet-ByteBuffer-extended.diff
For Apache Drill, we are looking to pass in a buffer that we have already
allocated (in this case from Direct memory), wrapped in a ByteBuffer.
The current effort to allow a ByteBuffer read path is great except that the
interface allocates the memory and there is no way for an application to pass
in memory that has been allocated or (even better) to provide an allocator.
Additionally, it would be great to be able to use the same approach while
decompressing.
As a starting point here is a patch on top of the ByteBuffer read effort that
adds a function in CompatibilityUtils and also adds a ByteBuffer path for the
Snappy Decompressor. The latter requires Hadoop 2.3 though, so some discussion
on this would be called for.
Please let me have any feedback and I can make changes/additions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
