Quanlong Huang created IMPALA-10842: ---------------------------------------
Summary: Show full stacktraces of jvm threads in web UI Key: IMPALA-10842 URL: https://issues.apache.org/jira/browse/IMPALA-10842 Project: IMPALA Issue Type: Improvement Components: Frontend Reporter: Quanlong Huang Attachments: image-2021-08-05-11-01-20-978.png IMPALA-4885 adds a web page showing all the JVM threads, which saves us some time to ssh to the machine and perform jstack manually. However, the summary column doesn't show the full stacktrace if it has more than 8 frames. For instance, when debugging a long running DDL in catalogd, I search for "execDdl" in the search box and get nothing since "execDdl" is truncated: !image-2021-08-05-11-01-20-978.png|width=905,height=208! I finally did a jstack and find the thread: {code:java} "Thread-11" #42 prio=5 os_prio=0 tid=0x000000000f86c000 nid=0x4c99 runnable [0x00007f9a8f8e1000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) - locked <0x00000005d4dbfa88> (a java.io.BufferedInputStream) at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:425) at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:321) at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:225) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table_objects_by_name_req(ThriftHiveMetastore.java:2492) at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table_objects_by_name_req(ThriftHiveMetastore.java:2479) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:2333) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTableObjectsByName(HiveMetaStoreClient.java:2314) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212) at com.sun.proxy.$Proxy11.getTableObjectsByName(Unknown Source) at org.apache.impala.service.CatalogOpExecutor.dropTablesFromKudu(CatalogOpExecutor.java:2349) at org.apache.impala.service.CatalogOpExecutor.dropDatabase(CatalogOpExecutor.java:2255) - locked <0x00000005d4a101e8> (a java.lang.Object) at org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:462) at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:240) {code} The cause is that we simply use TheadInfo#toString() which truncates the stacktrace: https://github.com/apache/impala/blob/47d868fe586905b2f4ea347a97ce582a45aac405/fe/src/main/java/org/apache/impala/common/JniUtil.java#L281 We can improve this to get the whole stacktrace. -- This message was sent by Atlassian Jira (v8.3.4#803005)