[ https://issues.apache.org/jira/browse/HBASE-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14059092#comment-14059092 ]
Nicolas Liochon commented on HBASE-11492: ----------------------------------------- Nagle and tcp delayed ack makes HBase slow with some usage pattern, with an impact up to 200ms, so it's definitively needed. I'm not sure that hdfs has the same issue, or at least not for all patterns. For example hbase could have an issue if the client sends 2 puts in parallel, but for hdfs there will be a single put so nagle or not nagle won't change much (i think). But yeah, I found the issue by looking at a scenario with everything in cache. > The servers do not honor the tcpNoDelay option > ---------------------------------------------- > > Key: HBASE-11492 > URL: https://issues.apache.org/jira/browse/HBASE-11492 > Project: HBase > Issue Type: Bug > Components: regionserver > Affects Versions: 0.98.0, 0.99.0 > Reporter: Nicolas Liochon > Assignee: Nicolas Liochon > Priority: Critical > Fix For: 0.99.0, 0.98.5 > > Attachments: 11492.v1.patch, 11492.v1.withp1.patch, 11492.v2.patch > > > There is an option to set tcpNoDelay, defaulted to true, but the socket > channel is actually not changed. As a consequence, the server works with > nagle enabled. This leads to very degraded behavior when a single connection > is shared between threads. We enter into conflicts with nagle and tcp delayed > ack. > Here is an example of performance with the PE tool plus HBASE-11491: > {noformat} > oneCon #client sleep exeTime (seconds) > avg latency, sleep excluded (microseconds) > true 1 0 31 > 310 > false 1 0 31 > 310 > true 2 0 50 > 500 > false 2 0 31 > 310 > true 2 5 488 (including 200s sleeping) > 2880 > false 2 5 246 (including 200s sleeping) > 460 > {noformat} > The latency is multiple by 5 (2880 vs 460) when the connection is shared. > This is the delayed ack kicking in. This can be fixed by really using tcp no > delay. > Any application sharing the tcp connection between threads has the issue. -- This message was sent by Atlassian JIRA (v6.2#6252)