[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2022-01-10 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17471971#comment-17471971 ] Steve Loughran commented on HADOOP-17201: - If you are confident that all versions of the S3a

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2021-06-28 Thread Daniel Zhi (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17370953#comment-17370953 ] Daniel Zhi commented on HADOOP-17201: - I noticed the task could be "stuck" in 

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2021-01-22 Thread Brandon (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17270478#comment-17270478 ] Brandon commented on HADOOP-17201: -- Some more context. For a while I had been using the staging

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2021-01-08 Thread Brandon (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17261496#comment-17261496 ] Brandon commented on HADOOP-17201: -- Adding another observation of this issue. My setup is Spark 2.4.4

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-29 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17223273#comment-17223273 ] James Yu commented on HADOOP-17201: --- [~ste...@apache.org] I turned on both org.apache.hadoop.fs.s3a

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219058#comment-17219058 ] Steve Loughran commented on HADOOP-17201: - ...OK. the HADOOP-17244 patch will eliminate the

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219036#comment-17219036 ] Steve Loughran commented on HADOOP-17201: - update. Yes, we're doing a retry in

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17219029#comment-17219029 ] Steve Loughran commented on HADOOP-17201: - bq. Regarding the bulk delete call failure, I do

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-21 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17218140#comment-17218140 ] James Yu commented on HADOOP-17201: --- [~ste...@apache.org] Sorry it may not be easy for us to collect

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17217637#comment-17217637 ] Steve Loughran commented on HADOOP-17201: - James, if you see this, can you set all of the

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-10-15 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17215091#comment-17215091 ] James Yu commented on HADOOP-17201: --- We have this "never ending last task" issue a lot too, except

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-08-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183162#comment-17183162 ] Steve Loughran commented on HADOOP-17201: - here's the AWS permissions we expect right now, KMS

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-08-21 Thread Dyno (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17182187#comment-17182187 ] Dyno commented on HADOOP-17201: --- what i can confirm now is - the bucket is versioned. - HADOOP-17063 is

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-08-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178253#comment-17178253 ] Steve Loughran commented on HADOOP-17201: - wait, HADOOP-17063 was yours too, wasn't it? I

[jira] [Commented] (HADOOP-17201) Spark job with s3acommitter stuck at the last stage

2020-08-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-17201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17178252#comment-17178252 ] Steve Loughran commented on HADOOP-17201: - possibilities * could be HADOOP-16823. Large