[
https://issues.apache.org/jira/browse/IMPALA-5392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17702896#comment-17702896
]
ASF subversion and git services commented on IMPALA-5392:
---------------------------------------------------------
Commit 6570c05fd449582c5a0094dd0f3ae0b3f23faf83 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=6570c05fd ]
IMPALA-5392: Display full Java stacktraces in /jvm-threadz
jvm-threadz provides information about all the JVM
threads running in an Impalad/Catalogd. Currently, this
relies on ThreadInfo.toString() for a text representation
of the thread information and stack. This output only contains
the first few frames, which can be an issue for deeper stacks
(e.g. the Catalog talking to HMS via Thrift). The output does
contain useful non-stacktrace information like what lock the
thread is waiting on.
This keeps the first line of the ThreadInfo.toString() output
that contains non-stacktrace information. Then it walks through
the StackTraceElements from ThreadInfo.getStackTrace() and
adds the full stack string. This is intended to mimic
the jstack output (with indentation and "at").
To avoid licensing issues, I have not looked at any of the
OpenJDK code (which is GPL 2.0) or any of the previous attempts
at solving this issue (which consulted OpenJDK code).
To make the stacks display correctly in the WebUI page,
this puts the thread information in a <span> with
"white-space: pre-wrap", which respects the newlines in
the original output.
Testing:
- gdb attached to the HMS and then issued a select in
impala-shell to force a metadata load. Then, looked at
the catalogd /jvm-threadz page to see the longer stacks.
Change-Id: I6730336600a8130e6452c682bcf249cac528ceee
Reviewed-on: http://gerrit.cloudera.org:8080/19632
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Reviewed-by: Quanlong Huang <huangquanl...@gmail.com>
> Stack depth for threads printed in the Catalog UI under JVM Threads is not
> deep enough
> --------------------------------------------------------------------------------------
>
> Key: IMPALA-5392
> URL: https://issues.apache.org/jira/browse/IMPALA-5392
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 2.9.0
> Reporter: Mostafa Mokhtar
> Assignee: Joe McDonnell
> Priority: Major
> Labels: newbie, supportability
> Attachments: CatalogUI_FixedStackTrace.png, StackFrame_len_1.png,
> StackFrame_len_2.png, impala5392_output_stack.png
>
>
> The depth of the call stack is not sufficient to understand the status of the
> system.
> |Summary||CPU time (s)||User time (s)||Blocked time (ms)||Blocked
> times||Native|
> |"Thread-11" Id=39 RUNNABLE (in native) at
> java.net.SocketInputStream.socketRead0(Native Method) at
> java.net.SocketInputStream.read(SocketInputStream.java:152) at
> java.net.SocketInputStream.read(SocketInputStream.java:122) at
> java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at
> java.io.BufferedInputStream.read1(BufferedInputStream.java:275) at
> java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked
> java.io.BufferedInputStream@5172f7b7 at
> org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
> at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84) ...
> Number of locked synchronizers = 1 -
> java.util.concurrent.locks.ReentrantLock$NonfairSync@52e53927|72.09|58.61|-1|1|true|
> |"Thread-34" Id=72 RUNNABLE (in native)|48.0821|39.3|-1|0|true|
> |"Thread-7" Id=35 WAITING on
> java.util.concurrent.locks.ReentrantLock$NonfairSync@52e53927 owned by
> "Thread-11" Id=39 at sun.misc.Unsafe.park(Native Method) - waiting on
> java.util.concurrent.locks.ReentrantLock$NonfairSync@52e53927 at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:867)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1197)
> at
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:214)
> at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:290) at
> org.apache.impala.catalog.CatalogServiceCatalog.getCatalogObjects(CatalogServiceCatalog.java:331)
> ...|38.1586|17.27|-1|0|false|
> |"Thread-20" Id=53 RUNNABLE (in native)|34.9055|28.29|-1|1|true|
> |"pool-3-thread-4" Id=63 WAITING on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@612caa8c
> at sun.misc.Unsafe.park(Native Method) - waiting on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@612caa8c
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)|6.85982|5.19|-1|7314|false|
> |"pool-3-thread-8" Id=88 WAITING on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@612caa8c
> at sun.misc.Unsafe.park(Native Method) - waiting on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@612caa8c
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)|5.09183|3.35|-1|8022|false|
> |"pool-3-thread-10" Id=107 WAITING on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@612caa8c
> at sun.misc.Unsafe.park(Native Method) - waiting on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@612caa8c
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> at
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)|4.29704|2.77|-1|7353|false|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
