[
https://issues.apache.org/jira/browse/HBASE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15603977#comment-15603977
]
ramkrishna.s.vasudevan commented on HBASE-16890:
------------------------------------------------
bq.Which means that when the amount of data to be checksummed is more in async
wal case we do spend lot of CPU there.
Yes [~Apache9]. See my previous comment. It is sync that is heavy now.
Previously the amount of data to be synced was less in size and once the
required size was reached we handed it over to the dfs client to queue those
synced data to form the packet header. Am still to read this part of code but i
just got inputs from HDFS guys here.
Now the amount of data to be synced is much bigger and I feel the CPU spends
more time there.
The append synchronization cost I think is something we cannot avoid because
that is where we sync the append/sync operations and the consumer uses that
lock to actual do the consumption logic.
[~saint.ack[~saint....@gmail.com]
I have few patches to directly copy to Asyncoutput (it is hacky for now).
Next is to use onheap buffers instead of DBB because the checkSum calculation
API does not deal with DBB directly.
But before that let me finish the hadoop code and the logic here to actually
calculate the checksum.
> Analyze the performance of AsyncWAL and fix the same
> ----------------------------------------------------
>
> Key: HBASE-16890
> URL: https://issues.apache.org/jira/browse/HBASE-16890
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Affects Versions: 2.0.0
> Reporter: ramkrishna.s.vasudevan
> Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0
>
> Attachments: contention.png, contention_defaultWAL.png
>
>
> Tests reveal that AsyncWAL under load in single node cluster performs slower
> than the Default WAL. This task is to analyze and see if we could fix it.
> See some discussions in the tail of JIRA HBASE-15536.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
