[ https://issues.apache.org/jira/browse/HDFS-15532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uma Maheswara Rao G updated HDFS-15532: --------------------------------------- Fix Version/s: 3.4.0 3.3.1 Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available) > listFiles on root/InternalDir will fail if fallback root has file > ----------------------------------------------------------------- > > Key: HDFS-15532 > URL: https://issues.apache.org/jira/browse/HDFS-15532 > Project: Hadoop HDFS > Issue Type: Sub-task > Affects Versions: 3.4.0 > Reporter: Uma Maheswara Rao G > Assignee: Uma Maheswara Rao G > Priority: Major > Labels: pull-request-available > Fix For: 3.3.1, 3.4.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > listFiles implementation gets the RemoteIterator created in > InternalViewFSDirFs as the root is an InternalViewFSDir. > If there is a fallback and a file exist at root level, it would have > collected when collecting locatedStatuses. > When its iterating over to that fallbacks file from RemoteIterator (which > was returned from InternalViewFSDirFs ), iterator's next will will call > getFileBlockLocations if it's a file. > {code:java} > @Override > public LocatedFileStatus next() throws IOException { > System.out.println(this); > if (!hasNext()) { > throw new NoSuchElementException("No more entries in " + f); > } > FileStatus result = stats[i++]; > // for files, use getBlockLocations(FileStatus, int, int) to avoid > // calling getFileStatus(Path) to load the FileStatus again > BlockLocation[] locs = result.isFile() ? > getFileBlockLocations(result, 0, result.getLen()) : > null; > return new LocatedFileStatus(result, locs); > }{code} > > this getFileBlockLocations will be made on InternalViewFSDirFs, as that > Iterator created originally from that fs. > InternalViewFSDirFs#getFileBlockLocations does not handle fallback cases. > It's always expecting "/", this means it always assuming the dir. > But with the fallback and returning Iterator from InternalViewFSDirFs, will > create problems. > Probably we need to handle fallback case in getFileBlockLocations as well.( > Fallback only should be the reason for call coming to InternalViewFSDirFs > with other than "/") > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org