[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-24 Thread Junping Du (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450067#comment-16450067
 ] 

Junping Du commented on HADOOP-15385:
-

Thanks [~shahrs87] for reporting the issue and [~jlowe] for fixing. I also 
verified locally that the patch work well to resolve previous UT problems and 
Jenkins report UT failure cannot reproduce locally too. ASF warning are related 
to environment but not our code.
+1. [~shahrs87], would you like to do the honor?
btw, because this is only test problem but not serious production issue, 
downgrade this from blocker to critical.

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.2
>Reporter: Rushabh S Shah
>Assignee: Jason Lowe
>Priority: Blocker
> Attachments: HADOOP-15385-branch-2.001.patch
>
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-24 Thread Jason Lowe (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450026#comment-16450026
 ] 

Jason Lowe commented on HADOOP-15385:
-

The failures are unrelated and pass for me locally on branch-2 with or without 
the patch applied.

The ASF warnings are for hs_err pid files, so apparently the JVM was crashing 
on the system at least twice during the tests.  That could easily explain why 
the test was failing with timeouts and other errors if the launched distcp 
didn't complete properly.  I'm not sure that's so much a problem with the test 
as it is with the setup of the Jenkins host.


> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.2
>Reporter: Rushabh S Shah
>Assignee: Jason Lowe
>Priority: Blocker
> Attachments: HADOOP-15385-branch-2.001.patch
>
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-24 Thread Rushabh S Shah (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449855#comment-16449855
 ] 

Rushabh S Shah commented on HADOOP-15385:
-

Thanks [~jlowe] for the patch and [~tasanuma0829] for the review.
+1 (binding).
There are 3 test failures in {{TestDistCpSystem}}.
Since this patch is just a test code change, I can't think this patch broke 
tests in {{TestDistCpSystem}}.
Jason, can you please verify that these failures are unrelated and if possible 
create a tracking jira.
I will commit this patch shortly.

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.2
>Reporter: Rushabh S Shah
>Assignee: Jason Lowe
>Priority: Blocker
> Attachments: HADOOP-15385-branch-2.001.patch
>
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-23 Thread genericqa (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449234#comment-16449234
 ] 

genericqa commented on HADOOP-15385:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
12s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
23s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
29s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
18s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}423m 39s{color} 
| {color:red} hadoop-distcp in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
27s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}438m 38s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.tools.TestDistCpSystem |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:f667ef1 |
| JIRA Issue | HADOOP-15385 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12920074/HADOOP-15385-branch-2.001.patch
 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux d0c3fd34b579 3.13.0-137-generic #186-Ubuntu SMP Mon Dec 4 
19:09:19 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2 / 99e82e2 |
| maven | version: Apache Maven 3.3.9 
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_171 |
| unit | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/14517/artifact/out/patch-unit-hadoop-tools_hadoop-distcp.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/14517/testReport/ |
| asflicense | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/14517/artifact/out/patch-asflicense-problems.txt
 |
| Max. process+thread count | 241 (vs. ulimit of 1) |
| modules | C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/14517/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-22 Thread Takanobu Asanuma (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447535#comment-16447535
 ] 

Takanobu Asanuma commented on HADOOP-15385:
---

Thanks for the explanation of the cause and uploading the patch quickly, 
[~jlowe]!

+1 (non-binding) for the latest patch. I confirmed that the tests succeeded 
with it in branch-2.8.

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.2
>Reporter: Rushabh S Shah
>Assignee: Jason Lowe
>Priority: Blocker
> Attachments: HADOOP-15385-branch-2.001.patch
>
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-19 Thread Takanobu Asanuma (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445334#comment-16445334
 ] 

Takanobu Asanuma commented on HADOOP-15385:
---

Thanks for reporting it, [~shahrs87], and thanks for working on the release of 
2.9.1, [~Sammi].

I looked into this issue a little. Seems MAPREDUCE-6909 broke the tests. They 
succeeded with {{9f9d554edfd83cbd2249c780124a75feebc52ef3}} and failed with 
{{71f49406f291038ef5772f216001c9e5abb14c8d in branch-2.8}}.

I'd like to ping [~jlowe] and [~ajisakaa].

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.3
>Reporter: Rushabh S Shah
>Priority: Blocker
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-19 Thread Rushabh S Shah (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445219#comment-16445219
 ] 

Rushabh S Shah commented on HADOOP-15385:
-

{quote} Are you going on work on this JIRA?  
{quote}
Hi Sammi, I am _not_ planning to work on this soon. I am occupied with some 
other stuff.
Maybe we can ping who broke/introduced these failures and request them to take 
a look ?
These failures are not seen in trunk or branch-3.1. Haven't checked branch-3.0.

bq.   Otherwise, I might consider leave it to next release. Your thoughts?
Since the junits are failing, I think there is a hidden bug somewhere and 
distcp is too important tool to ignore the failures. If there is a bug, then it 
can result in data loss/corruption.

Would like to hear from other community members viewpoints.


> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.3
>Reporter: Rushabh S Shah
>Priority: Blocker
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-19 Thread SammiChen (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445109#comment-16445109
 ] 

SammiChen commented on HADOOP-15385:


Hi [~shahrs87], thanks for ping me. Are you going on work on this JIRA?  I 
fully agree that we should better resolve the issue before the release. And on 
the other hand, there are some customers who are waiting eagerly to try the 
enhanced features in 2.9.  So If we can resolve the issue in a shot time 
window, that would be great.  Otherwise, I might consider leave it to next 
release. Your thoughts?

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.3
>Reporter: Rushabh S Shah
>Priority: Blocker
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-19 Thread Rushabh S Shah (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444808#comment-16444808
 ] 

Rushabh S Shah commented on HADOOP-15385:
-

Added blocker for 2.9.1 also. Please remove if you feel its not a blocker.

 

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.3
>Reporter: Rushabh S Shah
>Priority: Blocker
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | head -n3
> c4ea1c8bb73 HADOOP-14970. MiniHadoopClusterManager doesn't respect lack of 
> format option. Contributed by Erik Krogen
> 1548205a845 YARN-8147. TestClientRMService#testGetApplications sporadically 
> fails. Contributed by Jason Lowe
> c01b425ba31 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom 
> leveldb logger. Contributed by Jason Lowe.
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-15385) Many tests are failing in hadoop-distcp project in branch-2.8

2018-04-19 Thread Rushabh S Shah (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444805#comment-16444805
 ] 

Rushabh S Shah commented on HADOOP-15385:
-

These tests are failing in branch-2.9 also.
{noformat}
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
expected:<4> but was:<5>
[ERROR]   
TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
expected:<2> but was:<3>
[ERROR]   
TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
expected:<6> but was:<8>
[ERROR]   
TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
expected:<4> but was:<5>
[ERROR]   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
expected:<4> but was:<5>
[ERROR]   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
expected:<4> but was:<5>
[ERROR]   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
expected:<4> but was:<5>
[ERROR]   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
expected:<2> but was:<3>
[ERROR]   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
expected:<2> but was:<3>
[ERROR]   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
expected:<2> but was:<3>
[ERROR]   
TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
expected:<6> but was:<8>
[ERROR]   
TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
expected:<6> but was:<8>
[ERROR]   
TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
expected:<6> but was:<8>
[ERROR]   
TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
expected:<4> but was:<5>
[ERROR]   
TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
expected:<4> but was:<5>
[ERROR]   
TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
expected:<4> but was:<5>
[INFO] 
[ERROR] Tests run: 73, Failures: 16, Errors: 0, Skipped: 0
{noformat}

{noformat}
C02QD8LYG8WP-lm:hadoop-distcp rushabhs$ git branch
* branch-2.9.1

C02QD8LYG8WP-lm:hadoop-distcp rushabhs$ git status
On branch branch-2.9.1
Your branch is up-to-date with 'origin/branch-2.9.1'.
nothing to commit, working tree clean
{noformat}
Cc [~Sammi].

> Many tests are failing in hadoop-distcp project in branch-2.8
> -
>
> Key: HADOOP-15385
> URL: https://issues.apache.org/jira/browse/HADOOP-15385
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: tools/distcp
>Affects Versions: 2.8.3
>Reporter: Rushabh S Shah
>Priority: Blocker
>
> Many tests are failing in hadoop-distcp project in branch-2.8
> Below are the failing tests.
> {noformat}
> Failed tests: 
>   
> TestDistCpViewFs.testUpdateGlobTargetMissingSingleLevel:326->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingMultiLevel:346->checkResult:428 
> expected:<4> but was:<5>
>   TestDistCpViewFs.testGlobTargetMissingSingleLevel:306->checkResult:428 
> expected:<2> but was:<3>
>   TestDistCpViewFs.testUpdateGlobTargetMissingMultiLevel:367->checkResult:428 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
>   TestIntegration.testUpdateGlobTargetMissingSingleLevel:431->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingMultiLevel:454->checkResult:577 
> expected:<4> but was:<5>
>   TestIntegration.testGlobTargetMissingSingleLevel:408->checkResult:577 
> expected:<2> but was:<3>
>   TestIntegration.testUpdateGlobTargetMissingMultiLevel:478->checkResult:577 
> expected:<6> but was:<8>
> Tests run: 258, Failures: 16, Errors: 0, Skipped: 0
> {noformat}
> {noformat}
> rushabhs$ pwd
> /Users/rushabhs/hadoop/apacheHadoop/hadoop/hadoop-tools/hadoop-distcp
> rushabhs$ git branch
>  branch-2
>   branch-2.7
> * branch-2.8
>   branch-2.9
>   branch-3.0
>  rushabhs$ git log --oneline | he