[
https://issues.apache.org/jira/browse/SPARK-34523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kent Yao updated SPARK-34523:
-----------------------------
Description:
h2. Instruction
This will cause deadlock and hangs concurrent tasks forever on the same
executor. for example,
In the Spark UI stage tab, you may find some of the tasks hang for hours and
all others complete without delay.
!screenshot-2.png!
Also, you may find that these hanging tasks belong to the same executors.
Usually, in this case, you will also get nothing helpful from the executor log.
If you print the executor jstack or you check the ThreadDump via SparkUI
executor tab and you find some task thread blocked like below, you are very
likely to hit the JDK-8194653 issue.
!screenshot-1.png!
h2. Solutions
1. Update JDK version
was:
This will cause deadlock and hangs concurrent tasks forever on the same
executor. for example,
In the Spark UI stage tab, you may find some of the tasks hang for hours and
all others complete without delay.
!screenshot-2.png!
Also, you may find that these hanging tasks belong to the same executors.
Usually, in this case, you will also get nothing helpful from the executor log.
If you print the executor jstack or you check the ThreadDump via SparkUI
executor tab and you find some task thread blocked like below, you are very
likely to hit the JDK
!screenshot-1.png!
> JDK-8194653: Deadlock involving FileSystems.getDefault and
> System.loadLibrary call
> -----------------------------------------------------------------------------------
>
> Key: SPARK-34523
> URL: https://issues.apache.org/jira/browse/SPARK-34523
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 2.4.7, 3.0.2, 3.1.1
> Reporter: Kent Yao
> Priority: Major
> Attachments: 4303.log, screenshot-1.png, screenshot-2.png
>
>
> h2. Instruction
> This will cause deadlock and hangs concurrent tasks forever on the same
> executor. for example,
>
> In the Spark UI stage tab, you may find some of the tasks hang for hours and
> all others complete without delay.
>
> !screenshot-2.png!
> Also, you may find that these hanging tasks belong to the same executors.
> Usually, in this case, you will also get nothing helpful from the executor
> log.
> If you print the executor jstack or you check the ThreadDump via SparkUI
> executor tab and you find some task thread blocked like below, you are very
> likely to hit the JDK-8194653 issue.
> !screenshot-1.png!
> h2. Solutions
> 1. Update JDK version
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
