[jira] [Updated] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HDFS-10466: --- Resolution: Won't Fix Status: Resolved (was: Patch Available) Thanks [~andrew.wang] for discussion. I just need a unique ID for DN storage. With https://issues.apache.org/jira/browse/HDFS-8887, BlockLocation already contains those information. no need to add LocatedBlock. Close it. > DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation > instead of BlockLocation > -- > > Key: HDFS-10466 > URL: https://issues.apache.org/jira/browse/HDFS-10466 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: Juan Yu >Assignee: Juan Yu >Priority: Minor > Attachments: HDFS-10466.001.patch, HDFS-10466.patch > > > https://issues.apache.org/jira/browse/HDFS-202 added a new API > listLocatedStatus() to get all files' status with block locations for a > directory. This is great that we don't need to call > FileSystem.getFileBlockLocations() for each file. it's much faster (about > 8-10 times). > However, the returned LocatedFileStatus only contains basic BlockLocation > instead of HdfsBlockLocation, the LocatedBlock details are stripped out. > It should do the similar as DFSClient.getBlockLocations(), return > HdfsBlockLocation which provide full block location details. > The implementation of DistributedFileSystem. listLocatedStatus() retrieves > HdfsLocatedFileStatus which contains all information, but when convert it to > LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and > compatible) change to make to keep the LocatedBlock details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HDFS-10466: --- Attachment: HDFS-10466.001.patch > DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation > instead of BlockLocation > -- > > Key: HDFS-10466 > URL: https://issues.apache.org/jira/browse/HDFS-10466 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: Juan Yu >Assignee: Juan Yu >Priority: Minor > Attachments: HDFS-10466.001.patch, HDFS-10466.patch > > > https://issues.apache.org/jira/browse/HDFS-202 added a new API > listLocatedStatus() to get all files' status with block locations for a > directory. This is great that we don't need to call > FileSystem.getFileBlockLocations() for each file. it's much faster (about > 8-10 times). > However, the returned LocatedFileStatus only contains basic BlockLocation > instead of HdfsBlockLocation, the LocatedBlock details are stripped out. > It should do the similar as DFSClient.getBlockLocations(), return > HdfsBlockLocation which provide full block location details. > The implementation of DistributedFileSystem. listLocatedStatus() retrieves > HdfsLocatedFileStatus which contains all information, but when convert it to > LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and > compatible) change to make to keep the LocatedBlock details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10466: --- Target Version/s: 2.9.0 > DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation > instead of BlockLocation > -- > > Key: HDFS-10466 > URL: https://issues.apache.org/jira/browse/HDFS-10466 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: Juan Yu >Assignee: Juan Yu >Priority: Minor > Attachments: HDFS-10466.patch > > > https://issues.apache.org/jira/browse/HDFS-202 added a new API > listLocatedStatus() to get all files' status with block locations for a > directory. This is great that we don't need to call > FileSystem.getFileBlockLocations() for each file. it's much faster (about > 8-10 times). > However, the returned LocatedFileStatus only contains basic BlockLocation > instead of HdfsBlockLocation, the LocatedBlock details are stripped out. > It should do the similar as DFSClient.getBlockLocations(), return > HdfsBlockLocation which provide full block location details. > The implementation of DistributedFileSystem. listLocatedStatus() retrieves > HdfsLocatedFileStatus which contains all information, but when convert it to > LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and > compatible) change to make to keep the LocatedBlock details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HDFS-10466: --- Status: Patch Available (was: Open) Ran all related unit tests and they passed. Also verified my application can get HdfsBlockLocation successfully. > DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation > instead of BlockLocation > -- > > Key: HDFS-10466 > URL: https://issues.apache.org/jira/browse/HDFS-10466 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: Juan Yu >Assignee: Juan Yu >Priority: Minor > Attachments: HDFS-10466.patch > > > https://issues.apache.org/jira/browse/HDFS-202 added a new API > listLocatedStatus() to get all files' status with block locations for a > directory. This is great that we don't need to call > FileSystem.getFileBlockLocations() for each file. it's much faster (about > 8-10 times). > However, the returned LocatedFileStatus only contains basic BlockLocation > instead of HdfsBlockLocation, the LocatedBlock details are stripped out. > It should do the similar as DFSClient.getBlockLocations(), return > HdfsBlockLocation which provide full block location details. > The implementation of DistributedFileSystem. listLocatedStatus() retrieves > HdfsLocatedFileStatus which contains all information, but when convert it to > LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and > compatible) change to make to keep the LocatedBlock details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Juan Yu updated HDFS-10466: --- Attachment: HDFS-10466.patch > DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation > instead of BlockLocation > -- > > Key: HDFS-10466 > URL: https://issues.apache.org/jira/browse/HDFS-10466 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: Juan Yu >Assignee: Juan Yu >Priority: Minor > Attachments: HDFS-10466.patch > > > https://issues.apache.org/jira/browse/HDFS-202 added a new API > listLocatedStatus() to get all files' status with block locations for a > directory. This is great that we don't need to call > FileSystem.getFileBlockLocations() for each file. it's much faster (about > 8-10 times). > However, the returned LocatedFileStatus only contains basic BlockLocation > instead of HdfsBlockLocation, the LocatedBlock details are stripped out. > It should do the similar as DFSClient.getBlockLocations(), return > HdfsBlockLocation which provide full block location details. > The implementation of DistributedFileSystem. listLocatedStatus() retrieves > HdfsLocatedFileStatus which contains all information, but when convert it to > LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and > compatible) change to make to keep the LocatedBlock details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
