[
https://issues.apache.org/jira/browse/HBASE-21651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16729509#comment-16729509
]
Jingyun Tian commented on HBASE-21651:
--------------------------------------
HBASE-19358 Please check out this issue. It can address this problem.
> when splitting hlog，occurred exception
> ---------------------------------------
>
> Key: HBASE-21651
> URL: https://issues.apache.org/jira/browse/HBASE-21651
> Project: HBase
> Issue Type: Bug
> Affects Versions: 1.3.1, 2.1.0
> Reporter: Bo Cui
> Priority: Critical
>
> if hlog contains too many regions,when hlog splits, WALSplitter will open a
> discovered.edits(FSDataOutputStream) for each region,
> {code:title=WALSplitter.java|borderStyle=solid}
> protected Map<byte[], SinkWriter> writers = Collections
> .synchronizedMap(new TreeMap<byte[],
> SinkWriter>(Bytes.BYTES_COMPARATOR));
> {code}
> but datanode has a limit (dfs.datanode.max.transfer.threads), which will
> cause splithlog to be very slow,even failed.
> solution:add an max openEdits value（a configuration）, when openEdits exceeds
> this value, the oldest FSDataOutputStream will be closed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
