[
https://issues.apache.org/jira/browse/HBASE-15709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15596906#comment-15596906
]
stack commented on HBASE-15709:
-------------------------------
Block size seems fine as limit. Block would run on till opportunity to roll so
could be bigger than 256M, right?
DFSClient#DFSOS has its own machinery. There is nought we can do to influence
its operation. Changing Q size or packet size doesn't matter. We are careful
aggregating syncs and appends on our side but then the DFSClient#DFSOS
machinery does its own thing regardless (we need to be in control of the packet
sending). Given this, AsyncWAL for this reason and others is the path forward.
Let us know how can help. Will look at Rams perf issue this w/e.
> Handle large edits for asynchronous WAL
> ---------------------------------------
>
> Key: HBASE-15709
> URL: https://issues.apache.org/jira/browse/HBASE-15709
> Project: HBase
> Issue Type: Sub-task
> Components: io, wal
> Reporter: Duo Zhang
> Priority: Critical
>
> First, FanOutOneBlockAsyncDFSOutput can not work if the buffered data is
> larger than PacketReceiver.MAX_PACKET_SIZE(16MB).
> Second, since we only allow one block here, we need to make sure we do not
> exceed the block size after writing a large chunk of data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
