[jira] [Commented] (MAPREDUCE-6197) Cache MapOutputLocations in ShuffleHandler

2016-06-13 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328449#comment-15328449 ] Junping Du commented on MAPREDUCE-6197: --- The similar issue get fixed for Hive in

[jira] [Updated] (MAPREDUCE-6197) Cache MapOutputLocations in ShuffleHandler

2016-06-13 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6197: -- Target Version/s: 2.9.0 Status: Patch Available (was: Open) > Cache MapOutput

[jira] [Updated] (MAPREDUCE-6197) Cache MapOutputLocations in ShuffleHandler

2016-06-13 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6197: -- Attachment: MAPREDUCE-6197.patch > Cache MapOutputLocations in ShuffleHandler >

[jira] [Updated] (MAPREDUCE-6197) Cache MapOutputLocations in ShuffleHandler

2016-06-09 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6197: -- Assignee: Junping Du > Cache MapOutputLocations in ShuffleHandler >

[jira] [Updated] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-06-07 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6608: -- Target Version/s: MAPREDUCE-6608 > Work Preserving AM Restart for MapReduce > --

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-06-07 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318652#comment-15318652 ] Junping Du commented on MAPREDUCE-6608: --- I have create branch MAPREDUCE-6608 for

[jira] [Updated] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-17 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6657: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status:

[jira] [Commented] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-16 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285726#comment-15285726 ] Junping Du commented on MAPREDUCE-6657: --- The test failure is not related. 007 p

[jira] [Commented] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-13 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283178#comment-15283178 ] Junping Du commented on MAPREDUCE-6657: --- bq. I think the HDFS change is trivial

[jira] [Updated] (MAPREDUCE-6146) Reduce tar ball size for MR over distributed cache

2016-05-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6146: -- Labels: (was: BB2015-05-TBR) > Reduce tar ball size for MR over distributed cache > --

[jira] [Updated] (MAPREDUCE-6146) Reduce tar ball size for MR over distributed cache

2016-05-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6146: -- Target Version/s: (was: 2.8.0) > Reduce tar ball size for MR over distributed cache >

[jira] [Commented] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281517#comment-15281517 ] Junping Du commented on MAPREDUCE-6657: --- bq. Do you think we should create a sub

[jira] [Commented] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280564#comment-15280564 ] Junping Du commented on MAPREDUCE-6657: --- Thanks for updating the patch, [~haiboc

[jira] [Commented] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase

2016-05-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15280426#comment-15280426 ] Junping Du commented on MAPREDUCE-6657: --- Thanks [~haibochen] for the patch. The

[jira] [Commented] (MAPREDUCE-6672) TestTeraSort fails on Windows

2016-04-29 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15264066#comment-15264066 ] Junping Du commented on MAPREDUCE-6672: --- Thanks Tibor updating the patch and Chr

[jira] [Commented] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-21 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251894#comment-15251894 ] Junping Du commented on MAPREDUCE-6680: --- Thanks [~jianhe] for review and commit!

[jira] [Comment Edited] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-04-19 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248151#comment-15248151 ] Junping Du edited comment on MAPREDUCE-6608 at 4/19/16 4:46 PM:

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-04-19 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15248151#comment-15248151 ] Junping Du commented on MAPREDUCE-6608: --- [~vinodkv], thanks for review and comme

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-19 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Labels: Azure S3 (was: ) > JHS UserLogDir scan algorithm sometime could skip directory with

[jira] [Commented] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246735#comment-15246735 ] Junping Du commented on MAPREDUCE-6680: --- bq. (T1 != T3) or (T1 == T3 but T1.toS

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Attachment: MAPREDUCE-6680-v3.patch > JHS UserLogDir scan algorithm sometime could skip dire

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Attachment: (was: MAPREDUCE-6680-v3.patch) > JHS UserLogDir scan algorithm sometime coul

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Attachment: MAPREDUCE-6680-v3.patch > JHS UserLogDir scan algorithm sometime could skip dire

[jira] [Commented] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15246699#comment-15246699 ] Junping Du commented on MAPREDUCE-6680: --- bq. we can record latest scanning time

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Attachment: MAPREDUCE-6680-v2.patch v2 patch fix checkstyle issue. > JHS UserLogDir scan al

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Status: Patch Available (was: Open) > JHS UserLogDir scan algorithm sometime could skip dir

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Attachment: MAPREDUCE-6680.patch Put a patch with proper fix. The unit test could be very ha

[jira] [Commented] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15245731#comment-15245731 ] Junping Du commented on MAPREDUCE-6680: --- Given analysis in description, the solu

[jira] [Updated] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6680: -- Description: In our cluster based on a Cloud FileSystem, we notice JHS sometimes could skip

[jira] [Created] (MAPREDUCE-6680) JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.)

2016-04-18 Thread Junping Du (JIRA)
Junping Du created MAPREDUCE-6680: - Summary: JHS UserLogDir scan algorithm sometime could skip directory with update in CloudFS (Azure FileSystem, S3, etc.) Key: MAPREDUCE-6680 URL: https://issues.apache.org/jira/

[jira] [Updated] (MAPREDUCE-6670) TestJobListCache#testEviction sometimes fails on Windows with timeout

2016-04-13 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6670: -- Fix Version/s: 2.7.3 2.8.0 Release Note: Backport the fix to 2.7 and

[jira] [Commented] (MAPREDUCE-6670) TestJobListCache#testEviction sometimes fails on Windows with timeout

2016-04-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237511#comment-15237511 ] Junping Du commented on MAPREDUCE-6670: --- Sure. I will backport it to 2.7 and 2.8

[jira] [Commented] (MAPREDUCE-6672) TestTeraSort fails on Windows

2016-04-12 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15237333#comment-15237333 ] Junping Du commented on MAPREDUCE-6672: --- Thanks for the patch, [~kis...@freemail

[jira] [Commented] (MAPREDUCE-6660) Add MR Counters for bytes-read-by-network-distance FileSystem metrics

2016-04-08 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15233200#comment-15233200 ] Junping Du commented on MAPREDUCE-6660: --- Thanks [~mingma] for delivering the pat

[jira] [Updated] (MAPREDUCE-6670) TestJobListCache#testEviction sometimes fails on Windows with timeout

2016-04-06 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6670: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status:

[jira] [Commented] (MAPREDUCE-6670) TestJobListCache#testEviction sometimes fails on Windows with timeout

2016-04-06 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228346#comment-15228346 ] Junping Du commented on MAPREDUCE-6670: --- +1. Will commit it shortly. > TestJobL

[jira] [Commented] (MAPREDUCE-6670) TestJobListCache#testEviction sometimes fails on Windows with timeout

2016-04-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226588#comment-15226588 ] Junping Du commented on MAPREDUCE-6670: --- Thanks [~GergelyNovak] for updating the

[jira] [Commented] (MAPREDUCE-6670) TestJobListCache#testEviction sometimes fails on Windows with timeout

2016-04-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15226555#comment-15226555 ] Junping Du commented on MAPREDUCE-6670: --- Thanks [~GergelyNovak] for the patch. A

[jira] [Commented] (MAPREDUCE-5705) mapreduce.task.io.sort.mb hardcoded cap at 2047

2016-04-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15224234#comment-15224234 ] Junping Du commented on MAPREDUCE-5705: --- MAPREDUCE-2308 is a very old JIRA for M

[jira] [Reopened] (MAPREDUCE-5705) mapreduce.task.io.sort.mb hardcoded cap at 2047

2016-04-04 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reopened MAPREDUCE-5705: --- > mapreduce.task.io.sort.mb hardcoded cap at 2047 > --

[jira] [Updated] (MAPREDUCE-6655) Fix a typo (STRICT_IE6) in Encrypted Shuffle

2016-03-28 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6655: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status:

[jira] [Commented] (MAPREDUCE-6655) Fix a typo (STRICT_IE6) in Encrypted Shuffle

2016-03-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15213616#comment-15213616 ] Junping Du commented on MAPREDUCE-6655: --- Patch LGTM. +1. Will commit it shortly.

[jira] [Commented] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event

2016-03-23 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208571#comment-15208571 ] Junping Du commented on MAPREDUCE-6424: --- Sure. I just assign this JIRA to you. T

[jira] [Updated] (MAPREDUCE-6424) Store MR counters as timeline metrics instead of event

2016-03-23 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6424: -- Assignee: Naganarasimha G R (was: Junping Du) > Store MR counters as timeline metrics inste

[jira] [Commented] (MAPREDUCE-6338) MR AppMaster does not honor ephemeral port range

2016-03-14 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193346#comment-15193346 ] Junping Du commented on MAPREDUCE-6338: --- It sounds like many ports for RPC clien

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-02-24 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163297#comment-15163297 ] Junping Du commented on MAPREDUCE-6608: --- bq. Please let me know your recommendat

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-02-23 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159319#comment-15159319 ] Junping Du commented on MAPREDUCE-6608: --- Thanks [~srikanth.sampath] for updating

[jira] [Commented] (MAPREDUCE-6635) Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException

2016-02-23 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15159179#comment-15159179 ] Junping Du commented on MAPREDUCE-6635: --- Thanks [~vvasudev] for review and commi

[jira] [Updated] (MAPREDUCE-6635) Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException

2016-02-22 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6635: -- Priority: Critical (was: Major) > Unsafe long to int conversion in UncompressedSplitLineRea

[jira] [Updated] (MAPREDUCE-6635) Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException

2016-02-17 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6635: -- Status: Patch Available (was: Open) > Unsafe long to int conversion in UncompressedSplitLin

[jira] [Updated] (MAPREDUCE-6635) Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException

2016-02-17 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6635: -- Target Version/s: 2.8.0, 2.7.3, 2.6.5 > Unsafe long to int conversion in UncompressedSplitLi

[jira] [Updated] (MAPREDUCE-6635) Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException

2016-02-17 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6635: -- Attachment: MAPREDUCE-6635.patch Thanks [~sershe] for reporting this issue. Upload a patch t

[jira] [Assigned] (MAPREDUCE-6635) Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException

2016-02-17 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reassigned MAPREDUCE-6635: - Assignee: Junping Du > Unsafe long to int conversion in UncompressedSplitLineReader a

[jira] [Updated] (MAPREDUCE-6621) Memory Leak in JobClient#submitJobInternal()

2016-02-02 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6621: -- Target Version/s: 2.6.4 > Memory Leak in JobClient#submitJobInternal() > ---

[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6363: -- Target Version/s: 2.8.0, 2.6.4 (was: 2.8.0) > [NNBench] Lease mismatch error when running w

[jira] [Updated] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6554: -- Target Version/s: 2.6.4 > MRAppMaster servicestart failing with NPE in > MRAppMaster#parse

[jira] [Updated] (MAPREDUCE-5982) Task attempts that fail from the ASSIGNED state can disappear

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-5982: -- Target Version/s: 2.6.4 > Task attempts that fail from the ASSIGNED state can disappear > --

[jira] [Updated] (MAPREDUCE-6618) YarnClientProtocolProvider leaking the YarnClient thread.

2016-02-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6618: -- Target Version/s: 2.6.4 > YarnClientProtocolProvider leaking the YarnClient thread. > -

[jira] [Updated] (MAPREDUCE-6595) Fix findbugs warnings in OutputCommitter and FileOutputCommitter

2016-01-28 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6595: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status:

[jira] [Commented] (MAPREDUCE-6595) Fix findbugs warnings in OutputCommitter and FileOutputCommitter

2016-01-28 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15121199#comment-15121199 ] Junping Du commented on MAPREDUCE-6595: --- Thanks [~ajisakaa] for reporting the is

[jira] [Commented] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe

2016-01-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15120154#comment-15120154 ] Junping Du commented on MAPREDUCE-6542: --- Sorry for late response, [~piaoyu zhang

[jira] [Updated] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe

2016-01-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6542: -- Target Version/s: 2.7.3, 2.6.5 (was: 2.7.3, 2.6.4) > HistoryViewer use SimpleDateFormat,But

[jira] [Updated] (MAPREDUCE-6492) AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask

2016-01-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6492: -- Target Version/s: 2.6.4 > AsyncDispatcher exit with NPE on > TaskAttemptImpl#sendJHStartEve

[jira] [Updated] (MAPREDUCE-6558) multibyte delimiters with compressed input files generate duplicate records

2016-01-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6558: -- Target Version/s: 2.8.0, 2.7.3, 2.6.5 (was: 2.8.0, 2.7.3, 2.6.4) > multibyte delimiters wit

[jira] [Commented] (MAPREDUCE-6558) multibyte delimiters with compressed input files generate duplicate records

2016-01-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15119968#comment-15119968 ] Junping Du commented on MAPREDUCE-6558: --- Move it to 2.6.5 as 2.6.4 is almost the

[jira] [Commented] (MAPREDUCE-6454) MapReduce doesn't set the HADOOP_CLASSPATH for jar lib in distributed cache.

2016-01-26 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15117460#comment-15117460 ] Junping Du commented on MAPREDUCE-6454: --- MAPREDUCE-6619 is filed to track/fix th

[jira] [Updated] (MAPREDUCE-6619) HADOOP_CLASSPATH is overwritten in MR container

2016-01-26 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6619: -- Target Version/s: 2.8.0, 2.7.3, 2.6.4 Status: Patch Available (was: Open) > H

[jira] [Updated] (MAPREDUCE-6619) HADOOP_CLASSPATH is overwritten in MR container

2016-01-26 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6619: -- Attachment: MAPREDUCE-6619-branch-2.patch It sounds like MAPREDUCE-6454 only went to branch-

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-01-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15106139#comment-15106139 ] Junping Du commented on MAPREDUCE-6608: --- bq. I am just thinking that will it be

[jira] [Updated] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API

2016-01-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-5485: -- Fix Version/s: 2.8.0 > Allow repeating job commit by extending OutputCommitter API > ---

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-01-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15105267#comment-15105267 ] Junping Du commented on MAPREDUCE-6608: --- Thanks [~srikanth.sampath] and [~raju.b

[jira] [Commented] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-01-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15105251#comment-15105251 ] Junping Du commented on MAPREDUCE-6608: --- Move it to MAPREDUCE project given most

[jira] [Moved] (MAPREDUCE-6608) Work Preserving AM Restart for MapReduce

2016-01-18 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du moved YARN-3972 to MAPREDUCE-6608: - Key: MAPREDUCE-6608 (was: YARN-3972) Project: Hadoop Map/Redu

[jira] [Commented] (MAPREDUCE-5982) Task attempts that fail from the ASSIGNED state can disappear

2016-01-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092535#comment-15092535 ] Junping Du commented on MAPREDUCE-5982: --- Thanks [~jlowe]! > Task attempts that

[jira] [Commented] (MAPREDUCE-5718) MR job will fail after commit fail

2016-01-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092352#comment-15092352 ] Junping Du commented on MAPREDUCE-5718: --- Given MAPREDUCE-5485 is already commit

[jira] [Updated] (MAPREDUCE-6068) Illegal progress value warnings in map tasks

2016-01-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6068: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status:

[jira] [Updated] (MAPREDUCE-6068) Illegal progress value warnings in map tasks

2016-01-11 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6068: -- Labels: (was: BB2015-05-RFC) > Illegal progress value warnings in map tasks >

[jira] [Commented] (MAPREDUCE-6068) Illegal progress value warnings in map tasks

2016-01-10 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15091297#comment-15091297 ] Junping Du commented on MAPREDUCE-6068: --- Forget to mention, there is still anoth

[jira] [Commented] (MAPREDUCE-6068) Illegal progress value warnings in map tasks

2016-01-08 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15089172#comment-15089172 ] Junping Du commented on MAPREDUCE-6068: --- 002 patch LGTM. +1. Will commit it shor

[jira] [Updated] (MAPREDUCE-6577) MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set

2016-01-06 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6577: -- Target Version/s: 2.6.4 > MR AM unable to load native library without MR_AM_ADMIN_USER_ENV s

[jira] [Commented] (MAPREDUCE-5982) Task attempts that fail from the ASSIGNED state can disappear

2016-01-03 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080561#comment-15080561 ] Junping Du commented on MAPREDUCE-5982: --- Hi [~jlowe], do you think we should bac

[jira] [Commented] (MAPREDUCE-6436) JobHistory cache issue

2015-12-14 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15056154#comment-15056154 ] Junping Du commented on MAPREDUCE-6436: --- I think the impact of this issue could

[jira] [Updated] (MAPREDUCE-5485) Allow repeating job commit by extending OutputCommitter API

2015-12-10 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-5485: -- Release Note: Previously, the MR job will get failed if AM get restarted for some reason (l

[jira] [Commented] (MAPREDUCE-6549) multibyte delimiters with LineRecordReader cause duplicate records

2015-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034114#comment-15034114 ] Junping Du commented on MAPREDUCE-6549: --- Sounds good to have it in 2.6.3. Thanks

[jira] [Updated] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe

2015-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6542: -- Target Version/s: 2.7.3, 2.6.4 (was: 2.6.3, 2.7.3) > HistoryViewer use SimpleDateFormat,But

[jira] [Commented] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe

2015-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033748#comment-15033748 ] Junping Du commented on MAPREDUCE-6542: --- Moving target releases to 2.6.4. > His

[jira] [Updated] (MAPREDUCE-6558) multibyte delimiters with compressed input files generate duplicate records

2015-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6558: -- Target Version/s: 2.8.0, 2.7.3, 2.6.4 (was: 2.8.0, 2.6.3, 2.7.3) > multibyte delimiters wit

[jira] [Commented] (MAPREDUCE-6549) multibyte delimiters with LineRecordReader cause duplicate records

2015-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033565#comment-15033565 ] Junping Du commented on MAPREDUCE-6549: --- Hi [~wilfreds], [~rkanter] and [~jlowe]

[jira] [Commented] (MAPREDUCE-6558) multibyte delimiters with compressed input files generate duplicate records

2015-12-01 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033566#comment-15033566 ] Junping Du commented on MAPREDUCE-6558: --- Thanks [~wilfreds], moving this to 2.6.

[jira] [Commented] (MAPREDUCE-6542) HistoryViewer use SimpleDateFormat,But SimpleDateFormat is not threadsafe

2015-11-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15030116#comment-15030116 ] Junping Du commented on MAPREDUCE-6542: --- Hi, can we move this out of 2.6.3? Than

[jira] [Commented] (MAPREDUCE-6558) multibyte delimiters with compressed input files generate duplicate records

2015-11-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15030114#comment-15030114 ] Junping Du commented on MAPREDUCE-6558: --- Hi, can we move this out of 2.6.3? Than

[jira] [Updated] (MAPREDUCE-6355) 2.5 client cannot communicate with 2.5 job on 2.6 cluster

2015-11-27 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6355: -- Target Version/s: 2.7.2 (was: 2.7.2, 2.6.3) > 2.5 client cannot communicate with 2.5 job on

[jira] [Commented] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027262#comment-15027262 ] Junping Du commented on MAPREDUCE-6557: --- Sure. Thanks for reminding here, [~aw]!

[jira] [Updated] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6557: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0

[jira] [Commented] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027059#comment-15027059 ] Junping Du commented on MAPREDUCE-6557: --- Sounds like we are updating patch at th

[jira] [Updated] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6557: -- Attachment: MAPREDUCE-6557.004.patch Hi [~ajisakaa], I mark cleanup method as static in 004

[jira] [Commented] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027030#comment-15027030 ] Junping Du commented on MAPREDUCE-6557: --- Oh. Forget to mention, test will run fa

[jira] [Commented] (MAPREDUCE-6555) TestMRAppMaster fails on trunk

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027006#comment-15027006 ] Junping Du commented on MAPREDUCE-6555: --- Thanks [~varun_saxena] for reporting it

[jira] [Commented] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026844#comment-15026844 ] Junping Du commented on MAPREDUCE-6557: --- The failed tests are unrelated and are

[jira] [Commented] (MAPREDUCE-6557) Some tests in mapreduce-client-app are writing outside of target

2015-11-25 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-6557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026770#comment-15026770 ] Junping Du commented on MAPREDUCE-6557: --- 03 patch LGTM. +1 pending on Jenkins re

<    1   2   3   4   5   6   7   >