[ https://issues.apache.org/jira/browse/HDFS-16740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17624080#comment-17624080 ]
ASF GitHub Bot commented on HDFS-16740: --------------------------------------- xinglin commented on code in PR #4835: URL: https://github.com/apache/hadoop/pull/4835#discussion_r1005061516 ########## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommissionWithStriped.java: ########## @@ -117,10 +121,8 @@ protected Configuration createConfiguration() { public void setup() throws IOException { conf = createConfiguration(); // Set up the hosts/exclude files. - localFileSys = FileSystem.getLocal(conf); - Path workingDir = localFileSys.getWorkingDirectory(); - decommissionDir = new Path(workingDir, - PathUtils.getTestDirName(getClass()) + "/work-dir/decommission"); + localFileSys = FileSystem.get(baseDir.getRoot().toURI(), conf); Review Comment: Will FileSystem.get() return a LocalFileSystem object here? what scheme does baseDir.getRoot().toURI() return? ########## hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java: ########## @@ -83,37 +83,18 @@ public static void runCmd(DFSAdmin dfsadmin, boolean success, } @Rule - public TemporaryFolder folder = new TemporaryFolder(); - - /** - * Create a default HDFS configuration which has test-specific data directories. This is - * intended to protect against interactions between test runs that might corrupt results. Each - * test run's data is automatically cleaned-up by JUnit. - * - * @return a default configuration with test-specific data directories - */ - public Configuration getHdfsConfiguration() throws IOException { - Configuration conf = new HdfsConfiguration(); - - // Override the file system locations with test-specific temporary folders - conf.set(DFSConfigKeys.DFS_NAMENODE_NAME_DIR_KEY, Review Comment: Why is the case that we no longer need to set these properties anymore? Is it because now NAME_DIR/DATA_DIR/CHECKPOINT_DIR will all defaulted to be within the new baseDir? > Mini cluster test flakiness > --------------------------- > > Key: HDFS-16740 > URL: https://issues.apache.org/jira/browse/HDFS-16740 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, test > Affects Versions: 3.4.0, 3.3.5 > Reporter: Steve Vaughan > Assignee: Steve Vaughan > Priority: Major > Labels: pull-request-available > > Mini clusters used during HDFS unit tests are reporting test failures that do > not appear to be directly related to submitted changes. The failures are the > result of either interactions between tests run in parallel, or tests which > share common disk space for tests. In all cases, the tests can be run > individually serially without any errors. Addressing this issue will > simplify future submissions by eliminating the confusion introduced by these > unrelated test failures. > We can apply lessons recently from TestRollingUpgrade, which was recently > patched to unblock a recent submission. The fixes involved changing the HDFS > configuration to use temporary disk space for each individual tests, and > using try-with-resources to ensure that clusters were shutdown cleanly. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org