Thanks Stack. Few comments.
> Have you put much work into it? If you went to HBASE-10156, write > throughput is better again and you'd be on same system as is in current > hbase-1.x so would be easier to help you. > Good to know, 1.x move is planned soonish so will wait for it. <snip> > > Since AsyncWriter got aborted due to an unhanded exception the > AsyncSyncer > > etc. will not be notified, thus deadlocking the server. I am not familiar > > with HLog's life cycle, how does it handles errors in its worker threads? > > > My guess is that they are not supposed to have exited... the exception is > unexpected. They are not supposed to die in normal operation. > Thats true - in case of a checked exception (IOE in this case) - HLog's async mechanism marks an append as failed - goes ahead with sync, moves on & never dies. And I think what you are saying is it might be okay to do same for this specific (AIOBException raised by HDFS) error too. Fixing HDFS in not an option right now - so I might have to put a band aid in HLog. Let me know if I misunderstood. <snip> In the later versions of the WAL subsystem, there is > careful handling to make sure we don't do things like fail an append but > successfully sync. > By later version you mean 1.x and beyond? Thanks again! > St.Ack > > > > > > > > // 4. update 'lastWrittenTxid' and notify AsyncSyncer to do 'sync' > > asyncSyncer.setWrittenTxid(this.lastWrittenTxid); > > >
