[
https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571877#comment-14571877
]
Kai Zheng commented on HDFS-8319:
---------------------------------
Thanks Jing for the update.
bq. We use a direct ByteBuffer as buffer for the stripe, while for parity data
we use non-direct ByteBuffer. But the current decoder code only checks the
first input to determine whether using direct bytebuffer API. Thus I updated
the decoder code and made it use direct bytebuffer API if there is at least one
direct buffer in inputs/outputs.
In current codec framework it assumes all the inputs and outputs are either
on-heap buffers or direct buffers, no mixing of the both being allowed. This
assumption would simplify the implementation for coders. Basically, Java coders
use on-heap buffers that's converted into bytes array, native coders use direct
buffers, for the performance sake. Though for a coder it will also support
other buffer type, but may suffer performance loss. In the coder interface,
{{preferDirectBuffer}} can be used by the caller to see what buffer type would
be preferred as better. Using the same buffer type (no mixing) will also make
the buffer allocation and management easier overall. I'm wondering could we
avoid the mixing?
bq. the decoder code uses ByteBuffer#array() for non-direct bytebuffer cases.
This can cause issue if inputs/outputs are derived from slices of the same
underlying buffer.
I thought it's easy to support this by some double check and small fixing. Did
you see any problem doing that way? I remembered buffer's position and limit
are used, even the underlying arrays are the same.
> Erasure Coding: support decoding for stateful read
> --------------------------------------------------
>
> Key: HDFS-8319
> URL: https://issues.apache.org/jira/browse/HDFS-8319
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Attachments: HDFS-8319.001.patch, HDFS-8319.002.patch,
> HDFS-8319.003.patch
>
>
> HDFS-7678 adds the decoding functionality for pread. This jira plans to add
> decoding to stateful read.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
