[ https://issues.apache.org/jira/browse/ACCUMULO-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16200425#comment-16200425 ]
Mark Owens edited comment on ACCUMULO-4561 at 10/11/17 3:10 PM: ---------------------------------------------------------------- These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? 9999/tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 120000 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open http Jetty 6.1.26 8088/tcp open http Jetty 6.1.26 9995/tcp open http Jetty 9.3.21.v20170918 44263/tcp open http Jetty 6.1.26 50070/tcp open http Jetty 6.1.26 50090/tcp open http Jetty 6.1.26 >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 7693"... >>> Killed This port returned a different response after a timeout: 50075/tcp open http Jetty 6.1.26 >>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 120000 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 >>> remote=localhost/127.0.0.1:50075] I have no feel for how often the 'ping -ts <port>' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect. I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously? was (Author: jmark99): These crashes appear to be occurring when sending a ping request to ports that have Jetty listening. I ran an nmap scan on my local machine looking for open ports and then ran the accumulo shell ping command against the open ports (closed ports return connection refused). Note that all these tests were run on the 2.0.0-SNAPSHOT. My results are listed below: {{TServer port on local instance: 9997/tcp open palace-6? >>> localhost:9997:OK Following ports all returned same response: 2181/tcp open eforward? 4560/tcp open unknown 5355/tcp open llmnr? 8030/tcp open hadoop-ipc Hadoop IPC 8031/tcp open hadoop-ipc Hadoop IPC 8032/tcp open hadoop-ipc Hadoop IPC 8033/tcp open hadoop-ipc Hadoop IPC 8040/tcp open hadoop-ipc Hadoop IPC 9000/tcp open hadoop-ipc Hadoop IPC 34737/tcp open unknown 39473/tcp open hadoop-ipc Hadoop IPC 50010/tcp open unknown 50020/tcp open hadoop-ipc Hadoop IPC >>> localhost:8031 ERROR org.apache.thrift.transport.TTransportException 9998/tcp open distinct32? 9999/tcp open abyss? 10001/tcp open scp-config? >>> localhost:9998 ERROR org.apache.thrift.TApplicationException: Invalid >>> method name: 'getTabletServerStatus' 13562/tcp open unknown >>> localhost:13562 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 120000 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:36716 >>> remote=localhost/127.0.0.1:13562] Jetty ports: 8042/tcp open http Jetty 6.1.26 8088/tcp open http Jetty 6.1.26 9995/tcp open http Jetty 9.3.21.v20170918 44263/tcp open http Jetty 6.1.26 50070/tcp open http Jetty 6.1.26 50090/tcp open http Jetty 6.1.26 >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 7693"... >>> Killed This port returned a different response after a timeout: 50075/tcp open http Jetty 6.1.26 >>> localhost:50075 ERROR org.apache.thrift.transport.TTransportException: >>> java.net.SocketTimeoutException: 120000 millis timeout while waiting for >>> channel to be ready for read. ch : >>> java.nio.channels.SocketChannel[connected local=/127.0.0.1:37190 >>> remote=localhost/127.0.0.1:50075]}} I have no feel for how often the 'ping -ts <port>' command is run and how often it would be provided an invalid port? I would assume a user would only supply a port if they suspected it to be a tserver. Given that case this situation would not happen very often, I suspect. I also noticed that if I stop the tablet servers after I'm in the shell and then run the ping command, the shell never returns the prompt to the user. Ctrl-C'ing at that points exits the shell as well. I would think that should be fixed since the purpose of the ping is to retrieve the status of a tablet server. Has that behavior been documented and/or verified previously? > Crash when using ping on a non-existing server > ---------------------------------------------- > > Key: ACCUMULO-4561 > URL: https://issues.apache.org/jira/browse/ACCUMULO-4561 > Project: Accumulo > Issue Type: Bug > Components: shell > Affects Versions: 2.0.0 > Reporter: Luis Tavarez > > While working on ACCUMULO-4558, I tried running > {code}ping -ts localhost:9995{code} (localhost:9995 does not have a a tserver > on my setup.) > And it caused the shell to exit (crashed) and show the following message. > {code}# > # java.lang.OutOfMemoryError: Java heap space > # -XX:OnOutOfMemoryError="kill -9 %p" > # Executing /bin/sh -c "kill -9 25561"... > /home/lmtavar/git/uno/bin/uno: line 44: 25561 Killed > "$ACCUMULO_HOME"/bin/accumulo shell -u "$ACCUMULO_USER" -p > "$ACCUMULO_PASSWORD" "${@:2}" > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)