[ https://issues.apache.org/jira/browse/HBASE-19344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265720#comment-16265720 ]
Duo Zhang commented on HBASE-19344: ----------------------------------- See the comments of syncFailed method in AsyncFSWAL {code} private void syncFailed(Throwable error) { LOG.warn("sync failed", error); // Here we depends on the implementation of FanOutOneBlockAsyncDFSOutput and netty. // When error occur, FanOutOneBlockAsyncDFSOutput will fail all pending flush requests. It // is execute inside EventLoop. And in DefaultPromise in netty, it will notifyListener // directly if it is already in the EventLoop thread. And in the listener method, it will // call us. So here we know that all failed flush request will call us continuously, and // before the last one finish, no other task can be executed in EventLoop. So here we are // safe to use writerBroken as a guard. // Do not forget to revisit this if we change the implementation of // FanOutOneBlockAsyncDFSOutput! consumeLock.lock(); try { if (writerBroken) { return; } writerBroken = true; if (waitingRoll) { readyForRolling = true; readyForRollingCond.signalAll(); } } finally { consumeLock.unlock(); } for (Iterator<FSWALEntry> iter = unackedAppends.descendingIterator(); iter.hasNext();) { toWriteAppends.addFirst(iter.next()); } highestUnsyncedTxid = highestSyncedTxid.get(); // request a roll. requestLogRoll(); } {code} It is not straight-forward to use different event loop for AsyncDFSOutput. But anyway, I think this is a way to increase performance as we can shard the computations from one thread to several threads. > improve asyncWAL by using Independent thread for netty #IO in > FanOutOneBlockAsyncDFSOutput > ------------------------------------------------------------------------------------------- > > Key: HBASE-19344 > URL: https://issues.apache.org/jira/browse/HBASE-19344 > Project: HBase > Issue Type: Improvement > Components: wal > Affects Versions: 2.0.0-beta-1 > Reporter: Chance Li > Assignee: Chance Li > Fix For: 2.0.0 > > Attachments: HBASE-19344-branch2.patch, wal-1-test-result.png, > wal-8-test-result.png > > > The logic now is that the netty #IO thread and asyncWal's thread are the same > one. > Improvement proposal: > 1, Split into two. > 2, All multiWal share the netty #IO thread pool. -- This message was sent by Atlassian JIRA (v6.4.14#64029)