[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13615176#comment-13615176
]
Hudson commented on MAPREDUCE-5075:
---
Integrated in Hadoop-Hdfs-0.23-Build #566 (See
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/566/])
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream
does not close the wrapped InputStream. (Chris Nauroth via tgraves) (Revision
1461305)
Result = SUCCESS
tgraves :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1461305
Files :
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
*
/hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
*
/hadoop/common/branches/branch-0.23/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 0.23.7, 2.0.5-beta
>
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608828#comment-13608828
]
Hudson commented on MAPREDUCE-5075:
---
Integrated in Hadoop-Yarn-trunk #162 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/162/])
Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749)
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does
not close the wrapped InputStream. Contributed by Chris Nauroth (Revision
1458741)
Result = SUCCESS
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458749
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458741
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.0.5-beta
>
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607625#comment-13607625
]
Hudson commented on MAPREDUCE-5075:
---
Integrated in Hadoop-Mapreduce-trunk #1378 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1378/])
Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749)
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does
not close the wrapped InputStream. Contributed by Chris Nauroth (Revision
1458741)
Result = SUCCESS
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458749
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458741
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.0.5-beta
>
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607569#comment-13607569
]
Hudson commented on MAPREDUCE-5075:
---
Integrated in Hadoop-Hdfs-trunk #1350 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1350/])
Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749)
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does
not close the wrapped InputStream. Contributed by Chris Nauroth (Revision
1458741)
Result = FAILURE
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458749
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458741
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.0.5-beta
>
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607501#comment-13607501
]
Hudson commented on MAPREDUCE-5075:
---
Integrated in Hadoop-trunk-Commit #3495 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3495/])
Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749)
Result = SUCCESS
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458749
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.0.5-beta
>
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13607491#comment-13607491
]
Hudson commented on MAPREDUCE-5075:
---
Integrated in Hadoop-trunk-Commit #3494 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3494/])
MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream
does not close the wrapped InputStream. Contributed by Chris Nauroth (Revision
1458741)
Result = SUCCESS
szetszwo :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1458741
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Fix For: 2.0.5-beta
>
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603992#comment-13603992
]
Hadoop QA commented on MAPREDUCE-5075:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573975/MAPREDUCE-5075.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-tools/hadoop-distcp.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3421//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3421//console
This message is automatically generated.
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
> Attachments: MAPREDUCE-5075.1.patch
>
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[
https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603903#comment-13603903
]
Chris Nauroth commented on MAPREDUCE-5075:
--
I discovered this while testing on Windows, where file locking is enforced more
strictly. The DistCp tests would fail sporadically due to not being able to
delete the temp files. I have a patch in progress.
> DistCp leaks input file handles
> ---
>
> Key: MAPREDUCE-5075
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
>Affects Versions: 3.0.0
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>
> DistCp wraps the {{InputStream}} for each input file it reads in an instance
> of {{ThrottledInputStream}}. This class does not close the wrapped
> {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the
> {{ThrottledInputStream}} gets closed, but without closing the underlying
> wrapped stream, it still leaks a file handle.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
