comnetwork commented on PR #4721: URL: https://github.com/apache/hbase/pull/4721#issuecomment-1228364454
After I read the FSHLog code in current master,2.x, 1.x and 0.98, I find they always has a single-thread consumer to call blocking `ProtobufLog.append`(eg. in master `FSHLog.RingBufferEventHandler` corresponding to `AsyncFSWAL.consumeExecutor` calls `ProtobufLog.append` in its single-thread onEvent method), the problem you describe always exists in `FSHLog`. One possible solution I can think of is to wrap FSOutputStream with a BufProtobufLogWriter in a similar way to AsyncProtobufLogWriter. In consumeExecutor just written to the internal BufProtobufLogWriter ByteBuffer, BufProtobufLogWriter when needed. The sync, The FSoutputStream.write will be synched by SyncRunner. Fsoutputstream.write may also block a single thread. So between the FSWriter and consumeExecutor queue is likely to be full, once the queue is full, can let call BufProtobufLogWriter. Sync request fails, then execute syncFailed logic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
