[
https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605994#comment-15605994
]
ramkrishna.s.vasudevan commented on HBASE-16890:
------------------------------------------------
Some updates here. Read the code and discussed internally on how things work.
Need to do some verification in cluster tomorrow. Will report back. MAy be what
i think is that
In FSHLog case once the data was equal to (512 * 9) bytes we were flushing the
buffer and that flush was resulting in a chunk packet ( which is part of the
bigger chunk 64K). So only when the sync is called we may end up in non-aligned
chunks.
All this happens on the dfs client side.
On the DN side when it sees the unaligned block it tries to overwrite the CRC
of the current block with the previous block CRC. Now in the FSHLog case we may
get only the one chunk packet as unaligned per sync call.
But in case of AsyncHLog every packet we generate will mostly be unaligned and
so the DN needs to do the extra work of overwriting the CRC every time. This is
what I see from code but let me profile the DN also to verify. I may be wrong.
Will come back here. From HBase side profiling not much is revealed as per my
tests.
> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
> Key: HBASE-16890
> URL: https://issues.apache.org/jira/browse/HBASE-16890
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: contention.png, contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower
> than the Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
