[
https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643733#comment-14643733
]
Tsz Wo Nicholas Sze commented on HDFS-8287:
-------------------------------------------
I think we should have double buffering. In the beginning, user client writes
data to the first buffer. When the first cell strip is full, user client
continues writing to the second buffer. At the same time, the fastest parity
streamer picks up the first buffer and computes the parity. Once all parity
have been computed, the other parity streamers enqueue the parity packets.
Once all parity packet are enqueued, the first buffer can be released so that
when the second buffer is full, the user client can continue write to the first
buffer again.
> DFSStripedOutputStream.writeChunk should not wait for writing parity
> ---------------------------------------------------------------------
>
> Key: HDFS-8287
> URL: https://issues.apache.org/jira/browse/HDFS-8287
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: Tsz Wo Nicholas Sze
> Assignee: Kai Sasaki
>
> When a stripping cell is full, writeChunk computes and generates parity
> packets. It sequentially calls waitAndQueuePacket so that user client cannot
> continue to write data until it finishes.
> We should allow user client to continue writing instead but not blocking it
> when writing parity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
