[jira] [Commented] (SPARK-37239) Avoid unnecessary `setReplication` in Yarn mode

2021-11-08 Thread Victor Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440852#comment-17440852 ] Victor Wong commented on SPARK-37239: - HDFS `setReplication` is a time-consuming rpc call, it should

[jira] [Commented] (SPARK-37239) Avoid unnecessary `setReplication` in Yarn mode

2021-11-08 Thread Victor Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17440851#comment-17440851 ] Victor Wong commented on SPARK-37239: - [~LuciferYang] It seems the reporter could come up with a PR

[jira] [Commented] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

2020-09-03 Thread Victor Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190470#comment-17190470 ] Victor Wong commented on SPARK-24981: - My mistake:D, thanks for reply. > ShutdownHook timeout

[jira] [Commented] (SPARK-24981) ShutdownHook timeout causes job to fail when succeeded when SparkContext stop() not called by user program

2020-09-03 Thread Victor Wong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17190153#comment-17190153 ] Victor Wong commented on SPARK-24981: - `ERROR Utils: Uncaught exception in thread pool-4-thread-1`