[ https://issues.apache.org/jira/browse/HDFS-15578?focusedWorklogId=485008&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-485008 ]
ASF GitHub Bot logged work on HDFS-15578: ----------------------------------------- Author: ASF GitHub Bot Created on: 16/Sep/20 08:36 Start Date: 16/Sep/20 08:36 Worklog Time Spent: 10m Work Description: umamaheswararao commented on a change in pull request #2305: URL: https://github.com/apache/hadoop/pull/2305#discussion_r489262731 ########## File path: hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestViewDistributedFileSystemWithMountLinks.java ########## @@ -61,4 +64,55 @@ public void testCreateOnRoot() throws Exception { public void testMountLinkWithNonExistentLink() throws Exception { testMountLinkWithNonExistentLink(false); } + + @Test + public void testRenameOnInternalDirWithFallback() throws Exception { + Configuration conf = getConf(); + URI defaultFSURI = + URI.create(conf.get(CommonConfigurationKeys.FS_DEFAULT_NAME_KEY)); + final Path hdfsTargetPath1 = new Path(defaultFSURI + "/HDFSUser"); + final Path hdfsTargetPath2 = new Path(defaultFSURI + "/NewHDFSUser/next"); + ViewFsTestSetup.addMountLinksToConf(defaultFSURI.getAuthority(), + new String[] {"/HDFSUser", "/NewHDFSUser/next"}, + new String[] {hdfsTargetPath1.toUri().toString(), + hdfsTargetPath2.toUri().toString()}, conf); + //Making sure parent dir structure as mount points available in fallback. + try (DistributedFileSystem dfs = new DistributedFileSystem()) { + dfs.initialize(defaultFSURI, conf); + dfs.mkdirs(hdfsTargetPath1); + dfs.mkdirs(hdfsTargetPath2); + } + + try (FileSystem fs = FileSystem.get(conf)) { + Path src = new Path("/newFileOnRoot"); + Path dst = new Path("/newFileOnRoot1"); + fs.create(src).close(); + verifyRename(fs, src, dst); + + src = new Path("/newFileOnRoot1"); + dst = new Path("/NewHDFSUser/newFileOnRoot"); + fs.mkdirs(dst.getParent()); + verifyRename(fs, src, dst); + + src = new Path("/NewHDFSUser/newFileOnRoot"); + dst = new Path("/NewHDFSUser/newFileOnRoot1"); + verifyRename(fs, src, dst); + + src = new Path("/NewHDFSUser/newFileOnRoot1"); + dst = new Path("/newFileOnRoot"); + verifyRename(fs, src, dst); + + src = new Path("/HDFSUser/newFileOnRoot1"); + dst = new Path("/HDFSUser/newFileOnRoot"); + fs.create(src).close(); + verifyRename(fs, src, dst); + } + } + + private void verifyRename(FileSystem fs, Path src, Path dst) + throws IOException { + fs.rename(src, dst); + Assert.assertFalse(fs.exists(src)); + Assert.assertTrue(fs.exists(dst)); + } Review comment: @ayushtkn , Thanks a lot for the review! Good findings. Case1: Actually there are two rename behaviors, I think rename2 does expect both(src and dst) to be same type either dir or file. However I modified it to allow internalDir as dst with fallback, rename2 can fail later as usual. But the current rename(src,dst) also works as you verified. Please check the modified code whether it can satisfy you expected conditions. Case2: Yes, this is know fact that, if fallback does not have structure, rename does not create any dirs and it will fail. When user does not have any fallback structure, I think they should not expect all of this this should work very well as they might have created arbitrary mount point, but not created mount points with respective to fallback/default cluster structure. Please note ViewFs rename behave like rename2 so, test assertions are slightly different than rename api from ViewFileSystem. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 485008) Time Spent: 1h 20m (was: 1h 10m) > Fix the rename issues with fallback fs enabled > ---------------------------------------------- > > Key: HDFS-15578 > URL: https://issues.apache.org/jira/browse/HDFS-15578 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: viewfs, viewfsOverloadScheme > Affects Versions: 3.4.0 > Reporter: Uma Maheswara Rao G > Assignee: Uma Maheswara Rao G > Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > When we enable fallback, rename should success if the src.parent or > dst.parent on inernalDir. > {noformat} > org.apache.hadoop.security.AccessControlException: InternalDir of > ViewFileSystem is readonly, operation rename not permitted on path > /newFileOnRoot.org.apache.hadoop.security.AccessControlException: InternalDir > of ViewFileSystem is readonly, operation rename not permitted on path > /newFileOnRoot. > at > org.apache.hadoop.fs.viewfs.ViewFileSystem.readOnlyMountTable(ViewFileSystem.java:95) > at > org.apache.hadoop.fs.viewfs.ViewFileSystem.readOnlyMountTable(ViewFileSystem.java:101) > at > org.apache.hadoop.fs.viewfs.ViewFileSystem.rename(ViewFileSystem.java:683) at > org.apache.hadoop.hdfs.ViewDistributedFileSystem.rename(ViewDistributedFileSystem.java:533) > at > org.apache.hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks.verifyRename(TestViewDistributedFileSystemWithMountLinks.java:114) > at > org.apache.hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks.testRenameOnInternalDirWithFallback(TestViewDistributedFileSystemWithMountLinks.java:90) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at > org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at > org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at > org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at > org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at > org.junit.runner.JUnitCore.run(JUnitCore.java:137) at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) > at > com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33) > at > com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230) > at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58){noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org