[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240238#comment-14240238 ] Andrew Or commented on SPARK-4759: -- Aha, that is a great idea. Thanks [~joshrosen] > Dea

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240079#comment-14240079 ] Josh Rosen commented on SPARK-4759: --- [~andrewor14], If you wanted to run the reproducti

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238289#comment-14238289 ] Andrew Or commented on SPARK-4759: -- I have a smaller reproduction for branch-1.1. It seem

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238221#comment-14238221 ] Davis Shepherd commented on SPARK-4759: --- The fix appears to resolve the issue in mas

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237597#comment-14237597 ] Andrew Or commented on SPARK-4759: -- Hey I have opened the following PR to fix the symptom

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237587#comment-14237587 ] Apache Spark commented on SPARK-4759: - User 'andrewor14' has created a pull request fo

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237550#comment-14237550 ] Andrew Or commented on SPARK-4759: -- Ok yeah you're right, I can't reproduce it from the c

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237393#comment-14237393 ] Davis Shepherd commented on SPARK-4759: --- I still cannot reproduce the issue in the s

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237383#comment-14237383 ] Davis Shepherd commented on SPARK-4759: --- Ok your version does reproduce the issue ag

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237377#comment-14237377 ] Andrew Or commented on SPARK-4759: -- Hm I'll try branch 1.1 again later tonight. There mig

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237376#comment-14237376 ] Davis Shepherd commented on SPARK-4759: --- local[2] on the latest commit of branch 1.1

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237374#comment-14237374 ] Andrew Or commented on SPARK-4759: -- [~dgshep] That's strange. I am able to reproduce this

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237372#comment-14237372 ] Andrew Or commented on SPARK-4759: -- Found the issue. The task scheduler schedules tasks b

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237371#comment-14237371 ] Davis Shepherd commented on SPARK-4759: --- This doesn't seem to reproduce the issue fo

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237321#comment-14237321 ] Andrew Or commented on SPARK-4759: -- Hey I came up with a much smaller reproduction for th

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237264#comment-14237264 ] Davis Shepherd commented on SPARK-4759: --- It is possible to reproduce the issue with

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14236517#comment-14236517 ] Andrew Or commented on SPARK-4759: -- Quick update, I was only able to reproduce this in lo