[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833668#comment-13833668
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12616017/HDFS-5574.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5590//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/5590//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5590//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833891#comment-13833891
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12616049/HDFS-5574.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
org.apache.hadoop.hdfs.server.datanode.TestFsDatasetCache
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5592//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5592//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837414#comment-13837414 ] Binglin Chang commented on HDFS-5574: - The failed test is unrelated. TestBalancerWithNodeGroup is traced in HDFS-5580 TestFsDatasetCache: I looked into the failed log, the timeout is caused by a race condition, luckily after HDFS-5556 committed, the race condition no longer apply, so there should be no problem now. > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13850907#comment-13850907
]
Colin Patrick McCabe commented on HDFS-5574:
actually, maybe I spoke too soon. This patch removes a buffer copy from
{{RemoteBlockReader2#skip}}, which HDFS-5634 doesn't change. HDFS-5634 only
affects {{BlockReaderLocal}}, not either of the remote block readers. So
Binglin, if you want to prepare a new version, it would be worth looking at.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13850902#comment-13850902
]
Colin Patrick McCabe commented on HDFS-5574:
Let's close this as duplicate since after HDFS-5634, we no longer do a buffer
copy in {{BlockReader#skip}}.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13851424#comment-13851424
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619250/HDFS-5574.v3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
org.apache.hadoop.hdfs.server.namenode.ha.TestHASafeMode
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5755//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5755//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13852068#comment-13852068
]
Colin Patrick McCabe commented on HDFS-5574:
If it's easy, then I would say do {{RemoteBlockReader}} as well, just to avoid
divergence. There are still some people who have to use the old block reader--
for example, if they are using SOCKS sockets.
The main point of the old block reader is that it can operate on sockets that
don't have channels. In the long term, I feel like we could probably wrap
those sockets in something that would provide a channel, or just integrate that
functionality into {{RemoteBlockReader2}}.
Your test doesn't really test what you think. {{DFSInputStream#skip}} is not
necessarily going to call {{BlockReader#skip}}. If you want to play with a
remote block reader directly, you should call the test
{{TestRemoteBlockReader}} or something like that. There is an example in
{{TestBlockTokenWithDFS}}. Also, you should set
{{DFSInputStream#tcpReadsDisabledForTesting}} if you want to make sure you're
really getting a local block reader.
Finally, {{BlockReaderLocal}} lets you skip past the end of the file without
complaining. This behavior is consistent with how
{{FileInputChannel#position}} works. Since the {{DFSInputStream}} knows the
length of the block (reader), it never tries to skip past the end.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13853282#comment-13853282
]
Colin Patrick McCabe commented on HDFS-5574:
Thanks for working on this, Binglin.
{code}
BlockReaderTestUtil util;
byte[] blockData;
BlockReader reader;
{code}
If these aren't needed by the test subclasses, they should be private.
Similarly with {{getBlockReader}}.
{code}
conf.setBoolean(DFSConfigKeys.DFS_CLIENT_USE_LEGACY_BLOCKREADER, true);
{code}
I don't think you want this in {{TestRemoteBlockReader}}.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13853287#comment-13853287
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619595/HDFS-5574.v4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDFSInputStream
org.apache.hadoop.hdfs.TestRemoteBlockReader
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5769//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5769//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13853631#comment-13853631
]
Hadoop QA commented on HDFS-5574:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619705/HDFS-5574.v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/5777//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5777//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13854223#comment-13854223
]
Colin Patrick McCabe commented on HDFS-5574:
bq. I set DFSConfigKeys.DFS_CLIENT_USE_LEGACY_BLOCKREADER to true, so
getBlockReader can get RemoteBlockReader rather than RemoteBlockReader2.
oh, sorry, I was confusing this with {{DFS_CLIENT_USE_LEGACY_BLOCKREADERLOCAL}}.
+1.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032230#comment-14032230
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619705/HDFS-5574.v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.metrics2.impl.TestMetricsSystemImpl
org.apache.hadoop.ha.TestZKFailoverControllerStress
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7127//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7127//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032368#comment-14032368
]
Hadoop QA commented on HDFS-5574:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619705/HDFS-5574.v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7131//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7131//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032389#comment-14032389
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12619705/HDFS-5574.v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.datanode.TestBPOfferService
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7132//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7132//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061607#comment-14061607 ] Binglin Chang commented on HDFS-5574: - Hi [~cmccabe], looks like there are no more comments for a long time, could you help get this committed? Thanks:) > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Attachments: HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239350#comment-14239350
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685973/HDFS-5574.006.patch
against trunk revision db73cc9.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 352 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestDFSInputStream
org.apache.hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8970//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/8970//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/8970//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8970//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.v1.patch,
> HDFS-5574.v2.patch, HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14239922#comment-14239922
]
Hadoop QA commented on HDFS-5574:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12685973/HDFS-5574.006.patch
against trunk revision 82707b4.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 352 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.TestLeaseRecovery2
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/8974//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/8974//artifact/patchprocess/newPatchFindbugsWarningshadoop-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/8974//artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8974//console
This message is automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.v1.patch,
> HDFS-5574.v2.patch, HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240189#comment-14240189 ] Colin Patrick McCabe commented on HDFS-5574: Thanks for your patience. I'll try to look at this later this week. If anyone else wants to check it out before that, that works too > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Attachments: HDFS-5574.006.patch, HDFS-5574.v1.patch, > HDFS-5574.v2.patch, HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510790#comment-14510790 ] Akira AJISAKA commented on HDFS-5574: - Looks good to me. +1 pending Jenkins. Kicked https://builds.apache.org/job/PreCommit-HDFS-Build/10369/ > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Attachments: HDFS-5574.006.patch, HDFS-5574.v1.patch, > HDFS-5574.v2.patch, HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510895#comment-14510895
]
Hadoop QA commented on HDFS-5574:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 41s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:red}-1{color} | javac | 7m 37s | The applied patch generated 125
additional warning messages. |
| {color:green}+1{color} | javadoc | 9m 47s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 3m 58s | The applied patch generated 1
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 34s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 44s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | common tests | 22m 51s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 0m 31s | Tests failed in hadoop-hdfs. |
| | | 66m 42s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.security.ssl.TestReloadingX509TrustManager |
| Failed build | hadoop-hdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12685973/HDFS-5574.006.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / c8d7290 |
| javac |
https://builds.apache.org/job/PreCommit-HDFS-Build/10369/artifact/patchprocess/diffJavacWarnings.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10369/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10369/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10369/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10369/testReport/ |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10369/console |
This message was automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.v1.patch,
> HDFS-5574.v2.patch, HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513697#comment-14513697
]
Akira AJISAKA commented on HDFS-5574:
-
Hi [~decster], would you replace {{DFS_CLIENT_READ_SHORTCIRCUIT_KEY}} with
{{HdfsClientConfigKeys.Read.ShortCircuit.KEY}} in TestDFSInputStream? That will
fix the javac warning.
For checkstyle warnings, would you make {{FSInputChecker#readAndDiscard}} final
and adding {{@return}}, {{@param}}, and {{@throws}} tags in javadoc?
{code}
{code}
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.v1.patch,
> HDFS-5574.v2.patch, HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14514264#comment-14514264
]
Hadoop QA commented on HDFS-5574:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 34s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 7m 28s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 30s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 5m 26s | The applied patch generated 1
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 43s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:green}+1{color} | common tests | 24m 21s | Tests passed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 184m 14s | Tests failed in hadoop-hdfs. |
| | | 252m 50s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.blockmanagement.TestDatanodeManager |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12728364/HDFS-5574.007.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 5e67c4d |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10409/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10409/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10409/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10409/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10409/console |
This message was automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch,
> HDFS-5574.v1.patch, HDFS-5574.v2.patch, HDFS-5574.v3.patch,
> HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516439#comment-14516439
]
Akira AJISAKA commented on HDFS-5574:
-
Thanks [~decster] for updating the patch.
{code}
{code}
Would you add {{final}} to {{FSInputChecker#readAndDiscard}}?
The test failure looks unrelated to the patch.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch,
> HDFS-5574.v1.patch, HDFS-5574.v2.patch, HDFS-5574.v3.patch,
> HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516809#comment-14516809
]
Hadoop QA commented on HDFS-5574:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 17m 23s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 4 new or modified test files. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | javac | 9m 15s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 31s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 26s | The applied patch does
not increase the total number of release audit warnings. |
| {color:red}-1{color} | checkstyle | 4m 46s | The applied patch generated 1
additional checkstyle issues. |
| {color:green}+1{color} | install | 1m 42s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 38s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 5m 27s | The patch does not introduce
any new Findbugs (version 2.0.3) warnings. |
| {color:red}-1{color} | common tests | 22m 32s | Tests failed in
hadoop-common. |
| {color:red}-1{color} | hdfs tests | 174m 54s | Tests failed in hadoop-hdfs. |
| | | 248m 40s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.TestRemoteBlockReader |
| | hadoop.hdfs.TestDFSInputStream |
| | hadoop.hdfs.server.namenode.TestStartup |
| Timed out tests | org.apache.hadoop.ha.TestZKFailoverControllerStress |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12728717/HDFS-5574.008.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / feb68cb |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/10429/artifact/patchprocess/checkstyle-result-diff.txt
|
| hadoop-common test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10429/artifact/patchprocess/testrun_hadoop-common.txt
|
| hadoop-hdfs test log |
https://builds.apache.org/job/PreCommit-HDFS-Build/10429/artifact/patchprocess/testrun_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/10429/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10429/console |
This message was automatically generated.
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch,
> HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516940#comment-14516940
]
Binglin Chang commented on HDFS-5574:
-
Strange, the test error is caused by NoSuchMethodError, which should not happen
if code is compiled successfully, is there any bug in test-patch process?
{code}
java.lang.NoSuchMethodError:
org.apache.hadoop.fs.FSInputChecker.readAndDiscard(I)I
at
org.apache.hadoop.hdfs.RemoteBlockReader.read(RemoteBlockReader.java:128)
at
org.apache.hadoop.hdfs.DFSInputStream$ByteArrayStrategy.doRead(DFSInputStream.java:740)
at
org.apache.hadoop.hdfs.DFSInputStream.readBuffer(DFSInputStream.java:796)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:856)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:899)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:700)
at
org.apache.hadoop.hdfs.TestDFSInputStream.testSkipInner(TestDFSInputStream.java:61)
at
org.apache.hadoop.hdfs.TestDFSInputStream.testSkipWithRemoteBlockReader(TestDFSInputStream.java:76)
{code}
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch,
> HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[
https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521004#comment-14521004
]
Akira AJISAKA commented on HDFS-5574:
-
Looks like jenkins ran the tests in hadoop-hdfs project with
hadoop-common-3.0.0-.jar, which does not have
{{FSInputChecker#readAndDiscard}}. I could reproduce the error by the following
command:
{code}
$ cd hadoop-hdfs-project/hadoop-hdfs
$ mvn test -Dtest=TestDFSInputStream
{code}
> Remove buffer copy in BlockReader.skip
> --
>
> Key: HDFS-5574
> URL: https://issues.apache.org/jira/browse/HDFS-5574
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: Binglin Chang
>Assignee: Binglin Chang
>Priority: Trivial
> Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch,
> HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch,
> HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch
>
>
> BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read
> data to this buffer, it is not necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521253#comment-14521253 ] Akira AJISAKA commented on HDFS-5574: - +1, I ran the failed tests locally and all the tests passed. Committing this. > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521357#comment-14521357 ] Hudson commented on HDFS-5574: -- FAILURE: Integrated in Hadoop-trunk-Commit #7705 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7705/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521387#comment-14521387 ] Hudson commented on HDFS-5574: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2111 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2111/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521397#comment-14521397 ] Hudson commented on HDFS-5574: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #179 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/179/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521407#comment-14521407 ] Hudson commented on HDFS-5574: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #170 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/170/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521453#comment-14521453 ] Hudson commented on HDFS-5574: -- FAILURE: Integrated in Hadoop-Yarn-trunk #913 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/913/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521638#comment-14521638 ] Hudson commented on HDFS-5574: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #180 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/180/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-5574) Remove buffer copy in BlockReader.skip
[ https://issues.apache.org/jira/browse/HDFS-5574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14521711#comment-14521711 ] Hudson commented on HDFS-5574: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2129 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2129/]) HDFS-5574. Remove buffer copy in BlockReader.skip. Contributed by Binglin Chang. (aajisaka: rev e89fc53a1d264fde407dd2c36defab5241cd0b52) * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSInputStream.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader2.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderBase.java * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSInputChecker.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/RemoteBlockReader.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRemoteBlockReader2.java > Remove buffer copy in BlockReader.skip > -- > > Key: HDFS-5574 > URL: https://issues.apache.org/jira/browse/HDFS-5574 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Binglin Chang >Assignee: Binglin Chang >Priority: Trivial > Fix For: 2.8.0 > > Attachments: HDFS-5574.006.patch, HDFS-5574.007.patch, > HDFS-5574.008.patch, HDFS-5574.v1.patch, HDFS-5574.v2.patch, > HDFS-5574.v3.patch, HDFS-5574.v4.patch, HDFS-5574.v5.patch > > > BlockReaderLocal.skip and RemoteBlockReader.skip uses a temp buffer to read > data to this buffer, it is not necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
