[jira] [Updated] (MAPREDUCE-7475) 2 tests are non-idempotent (passes in the first run but fails in repeated runs in the same JVM)

2024-05-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7475: -- Fix Version/s: 3.5.0 > 2 tests are non-idempotent (passes in the first run but

[jira] [Updated] (MAPREDUCE-7475) 2 tests are non-idempotent (passes in the first run but fails in repeated runs in the same JVM)

2024-05-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7475: -- Affects Version/s: 3.4.0 > 2 tests are non-idempotent (passes in the first run but

[jira] [Assigned] (MAPREDUCE-7475) 2 tests are non-idempotent (passes in the first run but fails in repeated runs in the same JVM)

2024-05-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned MAPREDUCE-7475: - Assignee: Kaiyao Ke > 2 tests are non-idempotent (passes in the first run

[jira] [Updated] (MAPREDUCE-7475) 2 tests are non-idempotent (passes in the first run but fails in repeated runs in the same JVM)

2024-05-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7475: -- Priority: Minor (was: Major) > 2 tests are non-idempotent (passes in the first

[jira] [Updated] (MAPREDUCE-7475) 2 tests are non-idempotent (passes in the first run but fails in repeated runs in the same JVM)

2024-05-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7475: -- Component/s: test > 2 tests are non-idempotent (passes in the first run but fails

[jira] [Resolved] (MAPREDUCE-7474) [ABFS] Improve commit resilience and performance in Manifest Committer

2024-05-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7474. --- Fix Version/s: 3.3.9 3.5.0 3.4.1

[jira] [Updated] (MAPREDUCE-7476) Follow up of MAPREDUCE-7475 - fix more non-idempotent tests

2024-05-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7476: -- Summary: Follow up of MAPREDUCE-7475 - fix more non-idempotent tests (was: Follow

[jira] [Updated] (MAPREDUCE-7474) [ABFS] Improve commit resilience and performance in Manifest Committer

2024-04-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7474: -- Description: * Manifest committer is not resilient to rename failures on task

[jira] [Created] (MAPREDUCE-7474) [ABFS] Improve commit resilience and performance in Manifest Committer

2024-04-03 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7474: - Summary: [ABFS] Improve commit resilience and performance in Manifest Committer Key: MAPREDUCE-7474 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7474

[jira] [Assigned] (MAPREDUCE-7474) [ABFS] Improve commit resilience and performance in Manifest Committer

2024-04-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned MAPREDUCE-7474: - Assignee: Steve Loughran > [ABFS] Improve commit resilience and performance

[jira] [Resolved] (MAPREDUCE-7470) multi-thread mapreduce committer

2024-03-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7470. --- Resolution: Duplicate > multi-thread mapreduce committer >

[jira] [Updated] (MAPREDUCE-7470) multi-thread mapreduce v1 FileOutputcommitter

2024-03-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7470: -- Summary: multi-thread mapreduce v1 FileOutputcommitter (was: multi-thread

[jira] [Updated] (MAPREDUCE-7465) performance problem in FileOutputCommitter for big list processed by single thread

2024-03-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7465: -- Summary: performance problem in FileOutputCommitter for big list processed by

[jira] [Commented] (MAPREDUCE-7471) Hadoop mapred minicluster command line fails with class not found

2024-02-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17813156#comment-17813156 ] Steve Loughran commented on MAPREDUCE-7471: --- * for at getBaseDirectory() I think we should

[jira] [Commented] (MAPREDUCE-7470) multi-thread mapreduce committer

2024-01-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811557#comment-17811557 ] Steve Loughran commented on MAPREDUCE-7470: --- This is a duplicate of MAPREDUCE-7465, and I

[jira] [Commented] (MAPREDUCE-7465) performance problem in FileOutputCommiter for big list processed by single thread

2024-01-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802259#comment-17802259 ] Steve Loughran commented on MAPREDUCE-7465: --- [~arnaud.nauwynck]:

[jira] [Commented] (MAPREDUCE-7465) performance problem in FileOutputCommiter for big list processed by single thread

2024-01-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17802213#comment-17802213 ] Steve Loughran commented on MAPREDUCE-7465: --- bq. As of now, I have seen that the class

[jira] [Commented] (MAPREDUCE-7465) performance problem in FileOutputCommiter for big list processed by single thread

2024-01-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801918#comment-17801918 ] Steve Loughran commented on MAPREDUCE-7465: --- bq. I understand that developpers are

[jira] [Commented] (MAPREDUCE-7465) performance problem in FileOutputCommiter for big list processed by single thread

2024-01-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17801584#comment-17801584 ] Steve Loughran commented on MAPREDUCE-7465: --- I understand your pain here, but am reluctant

[jira] [Moved] (MAPREDUCE-7459) Sort strings to fix flaky test

2023-10-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran moved HADOOP-18947 to MAPREDUCE-7459: Component/s: test

[jira] [Updated] (MAPREDUCE-7449) Add add-opens flag to container launch commands on JDK17 nodes

2023-10-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7449: -- Fix Version/s: 3.3.9 Resolution: Fixed Status: Resolved (was:

[jira] [Commented] (MAPREDUCE-7449) Add add-opens flag to container launch commands on JDK17 nodes

2023-08-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17760010#comment-17760010 ] Steve Loughran commented on MAPREDUCE-7449: --- HADOOP-18862 should be fixed by this; will

[jira] [Resolved] (MAPREDUCE-7451) review TrackerDistributedCacheManager.checkPermissionOfOther

2023-08-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7451. --- Resolution: Won't Fix The class TrackerDistributedCacheManager only exists in

[jira] [Updated] (MAPREDUCE-7451) review TrackerDistributedCacheManager.checkPermissionOfOther

2023-08-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7451: -- Affects Version/s: 1.2.1 0.22.0 > review

[jira] [Created] (MAPREDUCE-7452) ManifestCommitter to support / as a destination

2023-08-21 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7452: - Summary: ManifestCommitter to support / as a destination Key: MAPREDUCE-7452 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7452 Project: Hadoop

[jira] [Updated] (MAPREDUCE-7451) review TrackerDistributedCacheManager.checkPermissionOfOther

2023-08-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7451: -- Summary: review TrackerDistributedCacheManager.checkPermissionOfOther (was:

[jira] [Updated] (MAPREDUCE-7451) review TrackerDistributedCacheManager.checkPermissionOfOther

2023-08-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7451: -- Description: TrackerDistributedCacheManager.checkPermissionOfOther() doesn't seem

[jira] [Commented] (MAPREDUCE-7448) Inconsistent Behavior for FileOutputCommitter V1 to commit successfully many times

2023-07-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748946#comment-17748946 ] Steve Loughran commented on MAPREDUCE-7448: --- process: can you tag with the version you are

[jira] [Resolved] (MAPREDUCE-7432) Make Manifest Committer the default for abfs and gcs

2023-06-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7432. --- Fix Version/s: 3.4.0 3.3.9 Release Note: By default, the

[jira] [Resolved] (MAPREDUCE-7435) ManifestCommitter OOM on azure job

2023-06-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7435. --- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed >

[jira] [Updated] (MAPREDUCE-7437) MR Fetcher class to use an AtomicInteger to generate IDs.

2023-04-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7437: -- Summary: MR Fetcher class to use an AtomicInteger to generate IDs. (was: spotbugs

[jira] [Resolved] (MAPREDUCE-7437) spotbugs complaining about .Fetcher's update of a nonatomic static counter

2023-04-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7437. --- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed >

[jira] [Commented] (MAPREDUCE-7437) spotbugs complaining about .Fetcher's update of a nonatomic static counter

2023-04-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17715024#comment-17715024 ] Steve Loughran commented on MAPREDUCE-7437: --- {code} CodeWarning ST Write to

[jira] [Created] (MAPREDUCE-7437) spotbugs complaining about .Fetcher's update of a nonatomic static counter

2023-04-21 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7437: - Summary: spotbugs complaining about .Fetcher's update of a nonatomic static counter Key: MAPREDUCE-7437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7437

[jira] [Commented] (MAPREDUCE-7435) ManifestCommitter OOM on azure job

2023-04-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707960#comment-17707960 ] Steve Loughran commented on MAPREDUCE-7435: --- comments of HADOOP-18650 also apply,

[jira] [Created] (MAPREDUCE-7435) ManifestCommitter OOM on azure job

2023-03-27 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7435: - Summary: ManifestCommitter OOM on azure job Key: MAPREDUCE-7435 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7435 Project: Hadoop Map/Reduce

[jira] [Assigned] (MAPREDUCE-7432) Make Manifest Committer the default for abfs and gcs

2023-02-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned MAPREDUCE-7432: - Assignee: Steve Loughran > Make Manifest Committer the default for abfs and

[jira] [Created] (MAPREDUCE-7432) Make Manifest Committer the default for abfs and gcs

2023-02-09 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7432: - Summary: Make Manifest Committer the default for abfs and gcs Key: MAPREDUCE-7432 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7432 Project: Hadoop

[jira] [Updated] (MAPREDUCE-7375) JobSubmissionFiles don't set right permission after mkdirs

2023-01-30 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7375: -- Fix Version/s: 3.3.5 (was: 3.3.9) > JobSubmissionFiles

[jira] [Commented] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648997#comment-17648997 ] Steve Loughran commented on MAPREDUCE-7428: --- i'm doing a full revert. don't want to have

[jira] [Commented] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648995#comment-17648995 ] Steve Loughran commented on MAPREDUCE-7428: --- the brute force action is to roll back all the

[jira] [Commented] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648768#comment-17648768 ] Steve Loughran commented on MAPREDUCE-7428: --- revert? > Fix failures related to Junit 4 to

[jira] [Commented] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17648115#comment-17648115 ] Steve Loughran commented on MAPREDUCE-7428: --- [~groot] to repeate my question: which patch

[jira] [Resolved] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7428. --- Fix Version/s: 3.4.0 Resolution: Fixed > Fix failures related to Junit 4

[jira] [Updated] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7428: -- Priority: Critical (was: Major) > Fix failures related to Junit 4 to Junit 5

[jira] [Commented] (MAPREDUCE-7422) Upgrade Junit 4 to 5 in hadoop-mapreduce-examples

2022-12-14 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17647080#comment-17647080 ] Steve Loughran commented on MAPREDUCE-7422: --- I'm assuming this is the cause of

[jira] [Commented] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2022-12-13 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17646540#comment-17646540 ] Steve Loughran commented on MAPREDUCE-7428: --- 1. what is the jira which caused this? add a

[jira] [Resolved] (MAPREDUCE-7401) Optimize liststatus for better performance by using recursive listing

2022-11-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7401. --- Resolution: Won't Fix > Optimize liststatus for better performance by using

[jira] [Commented] (MAPREDUCE-7401) Optimize liststatus for better performance by using recursive listing

2022-11-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640794#comment-17640794 ] Steve Loughran commented on MAPREDUCE-7401: --- the current pr isn't going in for the reasons

[jira] [Updated] (MAPREDUCE-7401) Optimize liststatus for better performance by using recursive listing

2022-11-29 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7401: -- Target Version/s: (was: 3.3.5) > Optimize liststatus for better performance by

[jira] [Resolved] (MAPREDUCE-7386) Maven parallel builds (skipping tests) fail

2022-11-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7386. --- Fix Version/s: 3.4.0 Resolution: Fixed in trunk, backport once we are

[jira] [Commented] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624459#comment-17624459 ] Steve Loughran commented on MAPREDUCE-7411: --- ok; changetd to the newer a/c > Use secure

[jira] [Assigned] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned MAPREDUCE-7411: - Assignee: PJ Fanning (was: PJ Fanning) > Use secure XML parser utils in

[jira] [Commented] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17624348#comment-17624348 ] Steve Loughran commented on MAPREDUCE-7411: --- [~fanningpj] assigned to you after adding you

[jira] [Assigned] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned MAPREDUCE-7411: - Assignee: PJ Fanning > Use secure XML parser utils in MapReduce >

[jira] [Updated] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7411: -- Description: Uptake of HADOOP-18469 If anyone is landing on this page following

[jira] [Resolved] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7411. --- Fix Version/s: 3.4.0 3.3.5 Resolution: Fixed merged

[jira] [Updated] (MAPREDUCE-7411) Use secure XML parser utils in MapReduce

2022-10-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7411: -- Summary: Use secure XML parser utils in MapReduce (was: Use secure XML parser

[jira] [Resolved] (MAPREDUCE-7403) Support spark dynamic partitioning in the Manifest Committer

2022-08-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7403. --- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed hadoop

[jira] [Created] (MAPREDUCE-7403) Support spark dynamic partitioning in the Manifest Committer

2022-08-09 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7403: - Summary: Support spark dynamic partitioning in the Manifest Committer Key: MAPREDUCE-7403 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7403

[jira] [Commented] (MAPREDUCE-7398) Improve TestDFSIO to support different filesystem

2022-07-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17569954#comment-17569954 ] Steve Loughran commented on MAPREDUCE-7398: --- moved to the right project; reassigned. now,

[jira] [Assigned] (MAPREDUCE-7398) Improve TestDFSIO to support different filesystem

2022-07-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned MAPREDUCE-7398: - Assignee: Vijayakumar Govindasamy > Improve TestDFSIO to support different

[jira] [Moved] (MAPREDUCE-7398) Improve TestDFSIO to support different filesystem

2022-07-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran moved HDFS-16674 to MAPREDUCE-7398: -- Component/s: (was: benchmarks)

[jira] [Commented] (MAPREDUCE-7397) Received status code 501 from server: HTTPS Required

2022-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17568152#comment-17568152 ] Steve Loughran commented on MAPREDUCE-7397: --- looks like related to this.

[jira] [Resolved] (MAPREDUCE-7397) Received status code 501 from server: HTTPS Required

2022-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7397. --- Resolution: Cannot Reproduce > Received status code 501 from server: HTTPS

[jira] [Commented] (MAPREDUCE-7397) Received status code 501 from server: HTTPS Required

2022-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17567909#comment-17567909 ] Steve Loughran commented on MAPREDUCE-7397: --- that's your maven repository playing up,

[jira] [Updated] (MAPREDUCE-7397) Received status code 501 from server: HTTPS Required

2022-07-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7397: -- Component/s: build > Received status code 501 from server: HTTPS Required >

[jira] [Updated] (MAPREDUCE-7371) DistributedCache alternative APIs should not use DistributedCache APIs internally

2022-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7371: -- Fix Version/s: 3.3.9 > DistributedCache alternative APIs should not use

[jira] [Resolved] (MAPREDUCE-7391) TestLocalDistributedCacheManager failing after HADOOP-16202

2022-06-22 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7391. --- Fix Version/s: 3.4.0 3.3.9 Resolution: Fixed >

[jira] [Moved] (MAPREDUCE-7391) TestLocalDistributedCacheManager failing after HADOOP-16202

2022-06-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran moved YARN-11189 to MAPREDUCE-7391: -- Component/s: test (was:

[jira] [Updated] (MAPREDUCE-7391) TestLocalDistributedCacheManager failing after HADOOP-16202

2022-06-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7391: -- Affects Version/s: 3.3.4 > TestLocalDistributedCacheManager failing after

[jira] [Commented] (MAPREDUCE-7331) Make temporary directory used by FileOutputCommitter configurable

2022-05-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17537508#comment-17537508 ] Steve Loughran commented on MAPREDUCE-7331: --- update on this. * the manifest committer is

[jira] [Resolved] (MAPREDUCE-7378) An error occurred while concurrently writing to a path

2022-05-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7378. --- Resolution: Won't Fix > An error occurred while concurrently writing to a path >

[jira] [Commented] (MAPREDUCE-7378) An error occurred while concurrently writing to a path

2022-05-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17537504#comment-17537504 ] Steve Loughran commented on MAPREDUCE-7378: --- this is a duplicate of MAPREDUCE-7331. see

[jira] [Resolved] (MAPREDUCE-7373) Building MapReduce NativeTask fails on Fedora 34+

2022-04-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7373. --- Fix Version/s: 3.3.3 (was: 3.4.0)

[jira] [Reopened] (MAPREDUCE-7373) Building MapReduce NativeTask fails on Fedora 34+

2022-04-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened MAPREDUCE-7373: --- > Building MapReduce NativeTask fails on Fedora 34+ >

[jira] [Updated] (MAPREDUCE-7373) Building MapReduce NativeTask fails on Fedora 34+

2022-04-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7373: -- Target Version/s: 3.3.3 > Building MapReduce NativeTask fails on Fedora 34+ >

[jira] [Resolved] (MAPREDUCE-7341) Add a task-manifest output committer for Azure and GCS

2022-03-17 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7341. --- Fix Version/s: 3.3.3 Resolution: Fixed > Add a task-manifest output

[jira] [Commented] (MAPREDUCE-7369) MapReduce tasks timing out when spends more time on MultipleOutputs#close

2022-01-20 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17479333#comment-17479333 ] Steve Loughran commented on MAPREDUCE-7369: --- # can this be a github PR? # why does this

[jira] [Updated] (MAPREDUCE-7331) Make temporary directory used by FileOutputCommitter configurable

2022-01-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7331: -- Issue Type: New Feature (was: Bug) > Make temporary directory used by

[jira] [Commented] (MAPREDUCE-5907) Improve getSplits() performance for fs implementations that can utilize performance gains from recursive listing

2022-01-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17469434#comment-17469434 ] Steve Loughran commented on MAPREDUCE-5907: --- abfs now does incremental listing, but not

[jira] [Commented] (MAPREDUCE-7369) MapReduce tasks timing out when spends more time on MultipleOutputs#close

2021-11-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17448662#comment-17448662 ] Steve Loughran commented on MAPREDUCE-7369: --- I can see this matters for cloud deployments

[jira] [Created] (MAPREDUCE-7367) Parallelize file moves in FileOutputCommitter v1 job commit

2021-10-28 Thread Steve Loughran (Jira)
Steve Loughran created MAPREDUCE-7367: - Summary: Parallelize file moves in FileOutputCommitter v1 job commit Key: MAPREDUCE-7367 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7367 Project:

[jira] [Updated] (MAPREDUCE-7366) FileOutputCommitter Enable Concurrent Writes

2021-10-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7366: -- Summary: FileOutputCommitter Enable Concurrent Writes (was: FileOutputCommitter

[jira] [Resolved] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7366. --- Resolution: Duplicate topic already covered in MAPREDUCE-7331 >

[jira] [Reopened] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened MAPREDUCE-7366: --- > FileOutputCommitter Enable Concurent Writes >

[jira] [Commented] (MAPREDUCE-7341) Add a task-manifest output committer for Azure and GCS

2021-10-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434443#comment-17434443 ] Steve Loughran commented on MAPREDUCE-7341: --- PR contains etag collection and use in

[jira] [Updated] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7366: -- Component/s: mrv2 > FileOutputCommitter Enable Concurent Writes >

[jira] [Resolved] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7366. --- Resolution: Won't Fix closing as a wontfix as * v2 is broken * this doesn't

[jira] [Updated] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7366: -- Affects Version/s: 3.3.1 > FileOutputCommitter Enable Concurent Writes >

[jira] [Updated] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7366: -- Labels: committers pull-request-available (was: committers easyfix easytask

[jira] [Updated] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7366: -- Labels: pull-request-available (was: committers pull-request-available) >

[jira] [Comment Edited] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433853#comment-17433853 ] Steve Loughran edited comment on MAPREDUCE-7366 at 10/25/21, 4:16 PM:

[jira] [Moved] (MAPREDUCE-7366) FileOutputCommitter Enable Concurent Writes

2021-10-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran moved HADOOP-17977 to MAPREDUCE-7366: Key: MAPREDUCE-7366 (was: HADOOP-17977)

[jira] [Updated] (MAPREDUCE-7287) Distcp will delete existing file , If we use "-delete and -update" options and distcp file.

2021-08-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated MAPREDUCE-7287: -- Summary: Distcp will delete existing file , If we use "-delete and -update"

[jira] [Resolved] (MAPREDUCE-7300) PathOutputCommitter to add method failedTaskAttemptCommitRecoverable()

2021-07-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7300. --- Resolution: Won't Fix mapreduce-7341 is intended to replace v2 algorithm on

[jira] [Resolved] (MAPREDUCE-7282) MR v2 commit algorithm should be deprecated and not the default

2021-07-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved MAPREDUCE-7282. --- Resolution: Won't Fix > MR v2 commit algorithm should be deprecated and not the

[jira] [Commented] (MAPREDUCE-7351) CleanupJob during handle of SIGTERM signal

2021-07-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372847#comment-17372847 ] Steve Loughran commented on MAPREDUCE-7351: --- 1. Submit it as a github PR 2. we use SLF4J

[jira] [Commented] (MAPREDUCE-7351) CleanupJob during handle of SIGTERM signal

2021-06-12 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17362352#comment-17362352 ] Steve Loughran commented on MAPREDUCE-7351: --- do you mean calling

[jira] [Commented] (MAPREDUCE-7287) Distcp will delete exists file , If we use "-delete and -update" options and distcp file.

2021-05-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/MAPREDUCE-7287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17353549#comment-17353549 ] Steve Loughran commented on MAPREDUCE-7287: --- it's in 3.3.2; I don't see any reason not to

  1   2   3   4   5   6   >