Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/12153#issuecomment-212424785 All the timeout in the Jenkins builds were due to a deadlock in `DAGScheduler`, and can be steadily reproduced locally by running the following test case > BucketedReadSuite.only shuffle one side when 2 bucketed tables have different bucket keys. This test case creates two bucketed tables both with 8 buckets and then joins them. Reducing 8 to 5 eliminates the deadlock. But I haven't figured out the real reason behind the deadlock. The deadlock also disappears if I remove FileScanRDD.preferredLocations(). Maybe that too many tasks are scheduled to the same place and exhausted some thread-pool?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org