[ https://issues.apache.org/jira/browse/HDFS-15128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021712#comment-17021712 ]
Hadoop QA commented on HDFS-15128: ---------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 33m 49s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 38s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 14m 37s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 34s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}126m 48s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 2m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}232m 52s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.tools.TestDFSAdminWithHA | | | hadoop.hdfs.TestEncryptionZonesWithKMS | | | hadoop.hdfs.TestDFSStripedOutputStream | | | hadoop.hdfs.TestDFSStripedInputStream | | | hadoop.hdfs.TestMultipleNNPortQOP | | | hadoop.hdfs.TestFileChecksum | | | hadoop.hdfs.TestDecommissionWithStriped | | | hadoop.hdfs.TestFileChecksumCompositeCrc | \\ \\ || Subsystem || Report/Notes || | Docker | Client=19.03.5 Server=19.03.5 Image:yetus/hadoop:c44943d1fc3 | | JIRA Issue | HDFS-15128 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12991568/HDFS-15128-000.patch | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 4e3db9017c5f 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 9520b2ad | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_232 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/28701/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/28701/testReport/ | | Max. process+thread count | 2686 (vs. ulimit of 5500) | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/28701/console | | Powered by | Apache Yetus 0.8.0 http://yetus.apache.org | This message was automatically generated. > Unit test failing to clean testing data and crashed future Maven test run > ------------------------------------------------------------------------- > > Key: HDFS-15128 > URL: https://issues.apache.org/jira/browse/HDFS-15128 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, test > Affects Versions: 3.2.1 > Reporter: Ctest > Priority: Critical > Labels: easyfix, patch, test > Attachments: HDFS-15128-000.patch, HDFS-15128-000.patch > > > Actively-used test helper function `testVolumeConfig` in > `org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration` > chmod a directory with invalid perm 000 for testing purposes but later failed > to chmod back this directory with a valid perm if the assertion inside this > function failed. Any subsequent `mvn test` command would fail to run if this > test had failed before. It is because Maven failed to build itself as it did > not have permission to clean the temporarily-generated directory that has > perm 000. See below for the code snippet that is buggy. > {code:java} > try { > for (int i = 0; i < volumesFailed; i++) { > prepareDirToFail(dirs[i]); // this will chmod dirs[i] to perm 000 > } > restartDatanodes(volumesTolerated, manageDfsDirs); > } catch (DiskErrorException e) { > ... > } finally { > ... > } > > assertEquals(expectedBPServiceState, bpServiceState); > > for (File dir : dirs) { > FileUtil.chmod(dir.toString(), "755"); > } > } > {code} > The failure of the statement `assertEquals(expectedBPServiceState, > bpServiceState)` caused function to terminate without executing > `FileUtil.chmod(dir.toString(), "755")` for each temporary directory with > invalid perm 000 the test has created. > > *Consequence* > Any subsequent `mvn test` command would fail to run if this test had failed > before. It is because Maven failed to build itself since it does not have > permission to clean this temporarily-generated directory. For details of the > failure, see below: > {noformat} > [INFO] --- maven-antrun-plugin:1.7:run (create-log-dir) @ hadoop-hdfs --- > [INFO] Executing tasks > > main: > [delete] Deleting directory > /home/ctest/app/Ctest-Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 8.349 s > [INFO] Finished at: 2019-12-27T03:53:04-06:00 > [INFO] > ------------------------------------------------------------------------ > [ERROR] Failed to execute > goalorg.apache.maven.plugins:maven-antrun-plugin:1.7:run (create-log-dir) on > project hadoop-hdfs: An Ant BuildException has occured: Unable to delete > directory > /home/ctest/app/Ctest-Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1/current > [ERROR] around Ant part ...<delete > dir="/home/ctest/app/Ctest-Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data"/>... > @ 4:105 in > /home/ctest/app/Ctest-Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/antrun/build-main.xml > [ERROR] -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException{noformat} > > *Root Cause* > The test helper function > `org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration#testVolumeConfig` > purposely set the directory > `/home/ctest/app/Ctest-Hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data1/current` > to have perm 000. And at the end of this function, it changed the perm of > this directory to 755. However, there is an assertion in this function before > the perm was able to changed to 755. Once this assertion fails, the function > terminates before the directory’s perm can be changed to 755. Hence, this > directory was later unable to be removed by Maven for when executing `mvn > test`. > > *Fix* > In > `org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration#testVolumeConfig`, > move the assertion `assertEquals(expectedBPServiceState, bpServiceState)` > to the last line of this function. This fix will fix the bug and will not > change the test outcome. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org