[jira] [Commented] (HBASE-15805) Canary test timeout if there is failed or shutdown zookeeper node in a quorum
[ https://issues.apache.org/jira/browse/HBASE-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15409335#comment-15409335 ] Vishal Khandelwal commented on HBASE-15805: --- [~apurtell] : Looks like this integration never. Can you please have a look. > Canary test timeout if there is failed or shutdown zookeeper node in a quorum > - > > Key: HBASE-15805 > URL: https://issues.apache.org/jira/browse/HBASE-15805 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.20 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal > Fix For: 0.98.20 > > Attachments: HBASE-15805.0.98.00.patch, HBASE-15805.0.98.01.patch, > HBASE-15805_0.98.patch > > > Canary tools creates HBaseadmin object which which starts a thread to keep > pinging zookeeper on a regular interval. This cause delay when one of the > node in zookeeper is down and test timeouts. > Canary tool is creating HBaseadmin whose isEnabled and tableExists calls > starts a catalog tacker --> retries for zookeeper connection. This zookeeper > (6s) retry happens for multiple times for scanning 390+ region it is causing > lot of sleeps. By removing the failed zookeeper from the test it actually > took not more than 9 sec where it was taking almost 8 mins. Since "sniff" > calls each table one by one thus causes the delay in the test. > {code} > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(203): Ignoring exception during shutdown input > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:401) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:200) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:782) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:399) > ... 4 more > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(210): Ignoring exception during shutdown output > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:409) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:207) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:799) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:407) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15805) Canary test timeout if there is failed or shutdown zookeeper node in a quorum
[ https://issues.apache.org/jira/browse/HBASE-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291650#comment-15291650 ] Hudson commented on HBASE-15805: ABORTED: Integrated in HBase-0.98-matrix #346 (See [https://builds.apache.org/job/HBase-0.98-matrix/346/]) HBASE-15805 Canary test timeout if there is failed or shutdown zookeeper (apurtell: rev b8cb2e43db10be501cf9a5263b7cb6ab184bc0c2) * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary test timeout if there is failed or shutdown zookeeper node in a quorum > - > > Key: HBASE-15805 > URL: https://issues.apache.org/jira/browse/HBASE-15805 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.20 >Reporter: Vishal Khandelwal > Fix For: 0.98.20 > > Attachments: HBASE-15805.0.98.00.patch, HBASE-15805.0.98.01.patch, > HBASE-15805_0.98.patch > > > Canary tools creates HBaseadmin object which which starts a thread to keep > pinging zookeeper on a regular interval. This cause delay when one of the > node in zookeeper is down and test timeouts. > Canary tool is creating HBaseadmin whose isEnabled and tableExists calls > starts a catalog tacker --> retries for zookeeper connection. This zookeeper > (6s) retry happens for multiple times for scanning 390+ region it is causing > lot of sleeps. By removing the failed zookeeper from the test it actually > took not more than 9 sec where it was taking almost 8 mins. Since "sniff" > calls each table one by one thus causes the delay in the test. > {code} > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(203): Ignoring exception during shutdown input > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:401) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:200) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:782) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:399) > ... 4 more > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(210): Ignoring exception during shutdown output > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:409) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:207) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:799) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:407) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15805) Canary test timeout if there is failed or shutdown zookeeper node in a quorum
[ https://issues.apache.org/jira/browse/HBASE-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15291562#comment-15291562 ] Hudson commented on HBASE-15805: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1218 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1218/]) HBASE-15805 Canary test timeout if there is failed or shutdown zookeeper (apurtell: rev b8cb2e43db10be501cf9a5263b7cb6ab184bc0c2) * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java > Canary test timeout if there is failed or shutdown zookeeper node in a quorum > - > > Key: HBASE-15805 > URL: https://issues.apache.org/jira/browse/HBASE-15805 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.20 >Reporter: Vishal Khandelwal > Fix For: 0.98.20 > > Attachments: HBASE-15805.0.98.00.patch, HBASE-15805.0.98.01.patch, > HBASE-15805_0.98.patch > > > Canary tools creates HBaseadmin object which which starts a thread to keep > pinging zookeeper on a regular interval. This cause delay when one of the > node in zookeeper is down and test timeouts. > Canary tool is creating HBaseadmin whose isEnabled and tableExists calls > starts a catalog tacker --> retries for zookeeper connection. This zookeeper > (6s) retry happens for multiple times for scanning 390+ region it is causing > lot of sleeps. By removing the failed zookeeper from the test it actually > took not more than 9 sec where it was taking almost 8 mins. Since "sniff" > calls each table one by one thus causes the delay in the test. > {code} > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(203): Ignoring exception during shutdown input > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:401) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:200) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:782) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:399) > ... 4 more > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(210): Ignoring exception during shutdown output > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:409) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:207) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:799) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:407) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-15805) Canary test timeout if there is failed or shutdown zookeeper node in a quorum
[ https://issues.apache.org/jira/browse/HBASE-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15290439#comment-15290439 ] Vishal Khandelwal commented on HBASE-15805: --- Thanks [~andrew.purt...@gmail.com] > Canary test timeout if there is failed or shutdown zookeeper node in a quorum > - > > Key: HBASE-15805 > URL: https://issues.apache.org/jira/browse/HBASE-15805 > Project: HBase > Issue Type: Bug > Components: canary >Affects Versions: 0.98.20 >Reporter: Vishal Khandelwal > Fix For: 0.98.20 > > Attachments: HBASE-15805.0.98.00.patch, HBASE-15805.0.98.01.patch, > HBASE-15805_0.98.patch > > > Canary tools creates HBaseadmin object which which starts a thread to keep > pinging zookeeper on a regular interval. This cause delay when one of the > node in zookeeper is down and test timeouts. > Canary tool is creating HBaseadmin whose isEnabled and tableExists calls > starts a catalog tacker --> retries for zookeeper connection. This zookeeper > (6s) retry happens for multiple times for scanning 390+ region it is causing > lot of sleeps. By removing the failed zookeeper from the test it actually > took not more than 9 sec where it was taking almost 8 mins. Since "sniff" > calls each table one by one thus causes the delay in the test. > {code} > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(203): Ignoring exception during shutdown input > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:401) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:200) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:782) > at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:399) > ... 4 more > 2016-04-29 09:52:13 [Thread-6-SendThread()] DEBUG > zookeeper.ClientCnxnSocketNIO(210): Ignoring exception during shutdown output > java.net.SocketException: Socket is not connected > at sun.nio.ch.Net.translateToSocketException(Net.java:123) > at sun.nio.ch.Net.translateException(Net.java:157) > at sun.nio.ch.Net.translateException(Net.java:163) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:409) > at > org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:207) > at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1246) > at > org.apache.zookeeper.ClientCnxn$SendThread.cleanAndNotifyState(ClientCnxn.java:1236) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1169) > Caused by: java.nio.channels.NotYetConnectedException > at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:799) > at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:407) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)