[
https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603867#comment-15603867
]
Duo Zhang commented on HBASE-16890:
-----------------------------------
The DFSOutputStream works like this
1. Copy data into the buf in FSOutputSummer
2. If the size exceeds limit(checksumChunkSize * BUFFER_NUM_CHUNKS), then
compute the checksum and call writeChunk method in DFSOutputStream
3. Serialize the chunk data into currentPacket field in DFSOutputStream.
4. When flushing, compute the checksum of the remaining data in buf if any,
call writeChunk, and then write the currentPacket out.
And for FanOutOneBlockAsyncDFSOutput, the processing is straight forward.
Buffer all data in buf, and do all other things such as compute checksum,
construct packet when flushing.
I do not think this will cause much difference unless we write very large data
at the same time that we maybe blocked in the JNI call for a long time which
has some bad impact on gc.
> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
> Key: HBASE-16890
> URL: https://issues.apache.org/jira/browse/HBASE-16890
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: contention.png, contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower
> than the Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
