[
https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13286638#comment-13286638
]
Kannan Muthukkaruppan commented on HBASE-6134:
----------------------------------------------
The SequenceFile writer does its own buffering. And so, my guess is that, at
close time, the flushing of these buffers for every region's recovered.edits
(SequenceFile) file is what is causing the close to be slow because it is
serialized on these flushes and then DN reporting back to NN that it has
received the blocks. Parallelizing the close in threads makes sense.
Good catch Chunhui! May I request you to update the description for the JIRA
with a summary of what the fix does, so it is useful for future readers of this
JIRA.
> Improvement for split-worker to speed up distributed-split-log
> --------------------------------------------------------------
>
> Key: HBASE-6134
> URL: https://issues.apache.org/jira/browse/HBASE-6134
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: chunhui shen
> Assignee: chunhui shen
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6134.patch, HBASE-6134v2.patch, HBASE-6134v3.patch
>
>
> First，we do the test between local-master-split and distributed split log
> Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths
> splitting work)
> local-master-split:60s+
> distributed-split-log:165s+
> In fact, in our production environment, distributed-split-log also took 60s
> with 30 regionservers for 34 hlog files (regionserver may be in high load)
> We found split-worker split one log file took about 20s.
> I think we should do the improvement for this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
