[jira] [Commented] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[ https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15303161#comment-15303161 ] Andrew Wang commented on HDFS-10466: Thanks for the patch Juan, LGTM overall. There are a couple long lines found by checkstyle. Do you want to add a unit test too to make sure you can dig the info you want out of this API? > DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation > instead of BlockLocation > -- > > Key: HDFS-10466 > URL: https://issues.apache.org/jira/browse/HDFS-10466 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: Juan Yu >Assignee: Juan Yu >Priority: Minor > Attachments: HDFS-10466.001.patch, HDFS-10466.patch > > > https://issues.apache.org/jira/browse/HDFS-202 added a new API > listLocatedStatus() to get all files' status with block locations for a > directory. This is great that we don't need to call > FileSystem.getFileBlockLocations() for each file. it's much faster (about > 8-10 times). > However, the returned LocatedFileStatus only contains basic BlockLocation > instead of HdfsBlockLocation, the LocatedBlock details are stripped out. > It should do the similar as DFSClient.getBlockLocations(), return > HdfsBlockLocation which provide full block location details. > The implementation of DistributedFileSystem. listLocatedStatus() retrieves > HdfsLocatedFileStatus which contains all information, but when convert it to > LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and > compatible) change to make to keep the LocatedBlock details. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[
https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301563#comment-15301563
]
Hadoop QA commented on HDFS-10466:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 29s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
33s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 27s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 38s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 17s
{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client: patch generated 3
new + 5 unchanged - 0 fixed = 8 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 41s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 59s
{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 29s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:2c91fd8 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12806291/HDFS-10466.001.patch |
| JIRA Issue | HDFS-10466 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d53ec1fc3267 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 4f513a4 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/15574/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/15574/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-client U:
hadoop-hdfs-project/hadoop-hdfs-client |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15574/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation
> instead of BlockLocation
> --
>
> Key: HDFS-10466
> URL:
[jira] [Commented] (HDFS-10466) DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation instead of BlockLocation
[
https://issues.apache.org/jira/browse/HDFS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301176#comment-15301176
]
Hadoop QA commented on HDFS-10466:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 3s {color}
| {color:red} HDFS-10466 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12806262/HDFS-10466.patch |
| JIRA Issue | HDFS-10466 |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/15571/console |
| Powered by | Apache Yetus 0.2.0 http://yetus.apache.org |
This message was automatically generated.
> DistributedFileSystem.listLocatedStatus() should return HdfsBlockLocation
> instead of BlockLocation
> --
>
> Key: HDFS-10466
> URL: https://issues.apache.org/jira/browse/HDFS-10466
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Reporter: Juan Yu
>Assignee: Juan Yu
>Priority: Minor
> Attachments: HDFS-10466.patch
>
>
> https://issues.apache.org/jira/browse/HDFS-202 added a new API
> listLocatedStatus() to get all files' status with block locations for a
> directory. This is great that we don't need to call
> FileSystem.getFileBlockLocations() for each file. it's much faster (about
> 8-10 times).
> However, the returned LocatedFileStatus only contains basic BlockLocation
> instead of HdfsBlockLocation, the LocatedBlock details are stripped out.
> It should do the similar as DFSClient.getBlockLocations(), return
> HdfsBlockLocation which provide full block location details.
> The implementation of DistributedFileSystem. listLocatedStatus() retrieves
> HdfsLocatedFileStatus which contains all information, but when convert it to
> LocatedFileStatus, it doesn't keep LocatedBlock data. It's a simple (and
> compatible) change to make to keep the LocatedBlock details.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
