[jira] [Commented] (MAPREDUCE-6519) hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979941#comment-14979941 ] Nemanja Matkovic commented on MAPREDUCE-6519: - TL; DR; I think we can commit this patch as it doesn't introduce regressions Two QA bot runs are running identical code, only two test cases with same failure are: * hadoop.mapreduce.v2.TestMRJobsWithHistoryService * hadoop.mapred.TestMiniMRClientCluster Both of these have very wild history full of reds and blues in pass/fail chart. In order to be good citizen (and sleep better at night) I went back and re-ran all these failed tests locally on java 7 and java 8 (named java7 and java8 below). I also ran them all on both java 7 and java 8 without any of IPv6 changes (head being eb6379ca25e1bb6d3978bd3a021723c38c95bec9). These are named base_java7 and base_java8 below. These are local results I have and investigation of history of these test cases that I saw: * hadoop.mapreduce.filecache.TestClientDistributedCacheManager ** java7, java8, base_java7, base_java8: FAIL. This is consistently failing because it fails here: "Assert.assertFalse(jobConf.getBoolean(MRJobConfig.CACHE_FILE_VISIBILITIES,true));" and I haven't changed this configuration. * hadoop.mapreduce.v2.TestMRJobsWithProfiler ** java7, base_java7: Pass ** java8, base_java8: Fail ** Consistent behavior, no regression ** There is no history data for this test case * hadoop.mapred.TestMiniMRClientCluster ** java7, base_java7: Fail ** java8, base_java8: Pass ** Consistent behavior, not regression ** History data shows lots of flakiness with same failure * hadoop.mapreduce.v2.TestMRJobs ** java7, java8, base_java7, base_java8: Pass ** Failure in QABot run looks strange, but similar to the way other flaky tests are failing * hadoop.mapreduce.v2.TestNonExistentJob ** java7, java8, test_java7, test_java8: PASS ** There is one previous run QA bot with same failure, so very occasional flakyness * hadoop.mapreduce.v2.TestUberAM * java7, java8, base_java7: Pass * base_java8: Timeout * No regression * hadoop.mapreduce.v2.TestMRJobsWithHistoryService ** java7: Multiple runs got both pass and fail locally ** java8: Pass ** base_java7: Fail ** base_java8: Pass ** Test historically flaky > hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 > literals > -- > > Key: MAPREDUCE-6519 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6519 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6519-HADOOP-11890.1.patch, > MAPREDUCE-6519-HADOOP-11890.2.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > mapreduce portion of patch in HADOOP-12122 that couldn't run due to number of > components touched. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6519) hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated MAPREDUCE-6519: Attachment: MAPREDUCE-6519-HADOOP-11890.2.patch Identical code as in HADOOP-11890.1.patch, only difference is that YARN changes are committed. All tests passing locally as with HADOOP-11890.1.patch, re-ran tests that failed with patch 1 and they still passed, let's see if things repro again or not. > hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 > literals > -- > > Key: MAPREDUCE-6519 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6519 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6519-HADOOP-11890.1.patch, > MAPREDUCE-6519-HADOOP-11890.2.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > mapreduce portion of patch in HADOOP-12122 that couldn't run due to number of > components touched. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6523) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic resolved MAPREDUCE-6523. - Resolution: Duplicate > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: MAPREDUCE-6523 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6523 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6523-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Yarn changes needed for IPv6 support for work > It seems I cannot assign JIRA under YARN (see YARN-4283) so creating under > mapreduce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6523) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated MAPREDUCE-6523: Status: Open (was: Patch Available) > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: MAPREDUCE-6523 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6523 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6523-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Yarn changes needed for IPv6 support for work > It seems I cannot assign JIRA under YARN (see YARN-4283) so creating under > mapreduce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6523) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14972703#comment-14972703 ] Nemanja Matkovic commented on MAPREDUCE-6523: - Thanks [~jlowe] for adding me to YARN will resolve this jira and reactivate YARN one. For QA bot test failures - there is something strange going on here, full "./dev-support/test-patch.sh --run-tests --branch-default=HADOOP-11890 ~/hdossPatches/MAPREDUCE-6523-HADOOP-11890.1.patch" passes on my box, all of these tests are passing. In QA bot run there is first a couple of test cases that failed with NoClassDefFoundError for some classes that are definitively there (like org/apache/hadoop/yarn/ipc/YarnRPC), so will just let it re-run under YARN in hope there is something misconfigured in mapreduce project. > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: MAPREDUCE-6523 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6523 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6523-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Yarn changes needed for IPv6 support for work > It seems I cannot assign JIRA under YARN (see YARN-4283) so creating under > mapreduce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6523) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated MAPREDUCE-6523: Attachment: MAPREDUCE-6523-HADOOP-11890.1.patch map-reduce portion of the patch from HADOOP-12122 (rebased head and retested locally) > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: MAPREDUCE-6523 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6523 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6523-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Yarn changes needed for IPv6 support for work > It seems I cannot assign JIRA under YARN (see YARN-4283) so creating under > mapreduce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6523) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated MAPREDUCE-6523: Status: Patch Available (was: Open) > hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 > literals > --- > > Key: MAPREDUCE-6523 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6523 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6523-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > Yarn changes needed for IPv6 support for work > It seems I cannot assign JIRA under YARN (see YARN-4283) so creating under > mapreduce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6519) hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated MAPREDUCE-6519: Attachment: MAPREDUCE-6519-HADOOP-11890.1.patch map-reduce portion of change for IPv6 support. > hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 > literals > -- > > Key: MAPREDUCE-6519 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6519 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6519-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > mapreduce portion of patch in HADOOP-12122 that couldn't run due to number of > components touched. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6519) hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nemanja Matkovic updated MAPREDUCE-6519: Status: Patch Available (was: Open) > hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 > literals > -- > > Key: MAPREDUCE-6519 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6519 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Attachments: MAPREDUCE-6519-HADOOP-11890.1.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > mapreduce portion of patch in HADOOP-12122 that couldn't run due to number of > components touched. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6523) hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals
Nemanja Matkovic created MAPREDUCE-6523: --- Summary: hadoop-yarn Avoid unsafe split and append on fields that might be IPv6 literals Key: MAPREDUCE-6523 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6523 Project: Hadoop Map/Reduce Issue Type: Task Reporter: Nemanja Matkovic Assignee: Nemanja Matkovic Yarn changes needed for IPv6 support for work It seems I cannot assign JIRA under YARN (see YARN-4283) so creating under mapreduce -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6519) hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals
[ https://issues.apache.org/jira/browse/MAPREDUCE-6519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14967550#comment-14967550 ] Nemanja Matkovic commented on MAPREDUCE-6519: - According to comment from HADOOP-12122 yarn needs to go in first then mapreduce > hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 > literals > -- > > Key: MAPREDUCE-6519 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6519 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Nemanja Matkovic >Assignee: Nemanja Matkovic > Labels: ipv6 > Original Estimate: 48h > Remaining Estimate: 48h > > mapreduce portion of patch in HADOOP-12122 that couldn't run due to number of > components touched. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6519) hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals
Nemanja Matkovic created MAPREDUCE-6519: --- Summary: hadoop-mapreduce - Avoid unsafe split and append on fields that might be IPv6 literals Key: MAPREDUCE-6519 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6519 Project: Hadoop Map/Reduce Issue Type: Task Reporter: Nemanja Matkovic Assignee: Nemanja Matkovic mapreduce portion of patch in HADOOP-12122 that couldn't run due to number of components touched. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
