[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Priority: Minor (was: Critical) > Spark doesn't create SUCCESS file in Spark 3.3.0+ when

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Priority: Major (was: Minor) > Spark doesn't create SUCCESS file in Spark 3.3.0+ when

[jira] [Comment Edited] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759065#comment-17759065 ] Dipayan Dev edited comment on SPARK-44884 at 8/25/23 2:22 PM: -- Right, the

[jira] [Comment Edited] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759065#comment-17759065 ] Dipayan Dev edited comment on SPARK-44884 at 8/25/23 2:21 PM: -- Right, the

[jira] [Comment Edited] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759065#comment-17759065 ] Dipayan Dev edited comment on SPARK-44884 at 8/25/23 2:20 PM: -- Right, the

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17759065#comment-17759065 ] Dipayan Dev commented on SPARK-44884: - Right, the behaviour is same in Spark 2 and 3. However, in

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17758879#comment-17758879 ] Dipayan Dev commented on SPARK-44884: - I am able to narrow down the problem. This issue is happening

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Summary: Spark doesn't create SUCCESS file in Spark 3.3.0+ when partitionOverwriteMode is dynamic

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0

[jira] [Comment Edited] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757052#comment-17757052 ] Dipayan Dev edited comment on SPARK-44884 at 8/25/23 7:34 AM: -- There is no

[jira] [Comment Edited] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757052#comment-17757052 ] Dipayan Dev edited comment on SPARK-44884 at 8/25/23 7:33 AM: -- There is no

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Attachment: (was: image-2023-08-20-18-08-38-531.png) > Spark doesn't create SUCCESS file in

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Attachment: image-2023-08-25-13-01-42-137.png > Spark doesn't create SUCCESS file in Spark 3.3.0+

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file in Spark 3.3.0+

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Summary: Spark doesn't create SUCCESS file in Spark 3.3.0+ (was: Spark doesn't create SUCCESS

[jira] [Comment Edited] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757550#comment-17757550 ] Dipayan Dev edited comment on SPARK-44884 at 8/25/23 7:06 AM: --

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757550#comment-17757550 ] Dipayan Dev commented on SPARK-44884: - [~ste...@apache.org] , I am running on DataProc but I am able

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757052#comment-17757052 ] Dipayan Dev commented on SPARK-44884: - There is no reason to disable this feature in Spark 3.3.0. We

[jira] [Commented] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757050#comment-17757050 ] Dipayan Dev commented on SPARK-44884: - [~ste...@apache.org] , have set that also, but still no

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Attachment: image-2023-08-20-18-46-53-342.png > Spark doesn't create SUCCESS file when external

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when external path is passed

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Summary: Spark doesn't create SUCCESS file when external path is passed (was: Spark doesn't

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Attachment: image-2023-08-20-18-08-38-531.png > Spark doesn't create SUCCESS file when

[jira] [Updated] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

[ https://issues.apache.org/jira/browse/SPARK-44884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44884: Description: The issue is not happening in Spark 2.x (I am using 2.4.0), but only in 3.3.0 Code

[jira] [Created] (SPARK-44884) Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic

Dipayan Dev created SPARK-44884: --- Summary: Spark doesn't create SUCCESS file when partitionOverwriteMode is Dynamic Key: SPARK-44884 URL: https://issues.apache.org/jira/browse/SPARK-44884 Project:

[jira] [Updated] (SPARK-44883) Spark insertInto with location GCS bucket root causes NPE

[ https://issues.apache.org/jira/browse/SPARK-44883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44883: Description: In our Organisation, we are using GCS bucket root location to point to our Hive

[jira] [Created] (SPARK-44883) Spark insertInto with location GCS bucket root causes NPE

Dipayan Dev created SPARK-44883: --- Summary: Spark insertInto with location GCS bucket root causes NPE Key: SPARK-44883 URL: https://issues.apache.org/jira/browse/SPARK-44883 Project: Spark

[jira] [Commented] (SPARK-41401) spark3 stagedir can't be change

[ https://issues.apache.org/jira/browse/SPARK-41401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746978#comment-17746978 ] Dipayan Dev commented on SPARK-41401: - This doesn't work in Google Cloud Storage even in Spark 2.x. 

[jira] [Updated] (SPARK-44543) Cleanup .spark-staging directories when yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-44543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44543: Description: Spark creates the staging directories like .hive-staging, .spark-staging etc which

[jira] [Updated] (SPARK-44543) Cleanup .spark-staging directories when yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-44543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44543: Description: Spark creates the staging directories like .hive-staging, .spark-staging etc which

[jira] [Commented] (SPARK-44543) Cleanup .spark-staging directories when yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-44543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746974#comment-17746974 ] Dipayan Dev commented on SPARK-44543: - Happy to contribute to this development of this feature.  >

[jira] [Updated] (SPARK-44543) Cleanup .spark-staging directories when yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-44543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44543: Description: Spark creates the staging directories like .hive-staging, .spark-staging etc which

[jira] [Updated] (SPARK-44543) Cleanup .spark-staging directories when yarn application fails

[ https://issues.apache.org/jira/browse/SPARK-44543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dipayan Dev updated SPARK-44543: Attachment: image-2023-07-25-17-52-55-006.png > Cleanup .spark-staging directories when yarn

[jira] [Created] (SPARK-44543) Cleanup .spark-staging directories when yarn application fails

Dipayan Dev created SPARK-44543: --- Summary: Cleanup .spark-staging directories when yarn application fails Key: SPARK-44543 URL: https://issues.apache.org/jira/browse/SPARK-44543 Project: Spark