[
https://issues.apache.org/jira/browse/FLINK-30184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638192#comment-17638192
]
Rui Fan commented on FLINK-30184:
---------------------------------
Hi [~gaoyunhaii] , thanks for your reply. I'm sorry, I may not have expressed
my thoughts clearly.
As I understand, Flame Graph can only display the current stack.
What I mean is: if it's 5am, the Flink job lag is huge. The Flink user did not
troubleshoot the problem in time, and started to troubleshoot after working
hours. However, Metric can only find out which Task is slow, and cannot analyze
where the Task is stuck at 5 o'clock in the morning? Why are Tasks slow? The
Thread stack at 5am can help users analyze where the task is stuck.
In general, the historical thread stack can tell the user what the flink job is
doing every minute.
> Save TM/JM thread stack periodically
> ------------------------------------
>
> Key: FLINK-30184
> URL: https://issues.apache.org/jira/browse/FLINK-30184
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Web Frontend
> Reporter: Rui Fan
> Priority: Major
> Fix For: 1.17.0
>
>
> After FLINK-14816 FLINK-25398 and FLINK-25372 , flink user can view the
> thread stack of TM/JM in Flink WebUI.
> It can help flink users to find out why the Flink job is stuck, or why the
> processing is slow. It is very useful for trouble shooting.
> However, sometimes Flink tasks get stuck or process slowly, but when the user
> troubleshoots the problem, the job has resumed. It is difficult to find out
> what happened to the Flink job at the time and why is it slow?
>
> So, could we periodically save the thread stack of TM or JM in the TM log
> directory?
> Define some configurations:
> cluster.thread-dump.interval=1min
> cluster.thread-dump.cleanup-time=48 hours
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
