[jira] [Comment Edited] (ACCUMULO-4561) Crash when using ping on a non-existing server
[ https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200425#comment-16200425 ] Mark Owens edited comment on ACCUMULO-4561 at 10/11/17 5:51 PM: These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {noformat} TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty 9.3.21.v20170918 44263/tcp open httpJetty 6.1.26 50070/tcp open httpJetty 6.1.26 50090/tcp open httpJetty 6.1.26 >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 7693"... >>> Killed This port returned a different response after a timeout: 50075/tcp open httpJetty 6.1.26 >>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 >>> remote=localhost/127.0.0.1:50075] {noformat} I have no feel for how often the 'ping -ts ' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect. I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously? was (Author: jmark99): These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open http
[jira] [Comment Edited] (ACCUMULO-4561) Crash when using ping on a non-existing server
[ https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200425#comment-16200425 ] Mark Owens edited comment on ACCUMULO-4561 at 10/11/17 3:10 PM: These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {{TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty 9.3.21.v20170918 44263/tcp open httpJetty 6.1.26 50070/tcp open httpJetty 6.1.26 50090/tcp open httpJetty 6.1.26 >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 7693"... >>> Killed This port returned a different response after a timeout: 50075/tcp open httpJetty 6.1.26 >>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 >>> remote=localhost/127.0.0.1:50075]}} I have no feel for how often the 'ping -ts ' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect. I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously? was (Author: jmark99): These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {{TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty 9.3.21.v20170918
[jira] [Comment Edited] (ACCUMULO-4561) Crash when using ping on a non-existing server
[ https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200425#comment-16200425 ] Mark Owens edited comment on ACCUMULO-4561 at 10/11/17 3:10 PM: These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty 9.3.21.v20170918 44263/tcp open httpJetty 6.1.26 50070/tcp open httpJetty 6.1.26 50090/tcp open httpJetty 6.1.26 >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 7693"... >>> Killed This port returned a different response after a timeout: 50075/tcp open httpJetty 6.1.26 >>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 >>> remote=localhost/127.0.0.1:50075] I have no feel for how often the 'ping -ts ' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect. I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously? was (Author: jmark99): These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {{TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty 9.3.21.v20170918
[jira] [Comment Edited] (ACCUMULO-4561) Crash when using ping on a non-existing server
[ https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200425#comment-16200425 ] Mark Owens edited comment on ACCUMULO-4561 at 10/11/17 3:09 PM: These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {{TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty 9.3.21.v20170918 44263/tcp open httpJetty 6.1.26 50070/tcp open httpJetty 6.1.26 50090/tcp open httpJetty 6.1.26 >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 7693"... >>> Killed This port returned a different response after a timeout: 50075/tcp open httpJetty 6.1.26 >>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 >>> remote=localhost/127.0.0.1:50075]}} I have no feel for how often the 'ping -ts ' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect. I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously? was (Author: jmark99): These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {{ TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? /tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 12 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open httpJetty 6.1.26 8088/tcp open httpJetty 6.1.26 9995/tcp open httpJetty