[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13608828#comment-13608828 ] Hudson commented on MAPREDUCE-5075: --- Integrated in Hadoop-Yarn-trunk #162 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/162/]) Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749) MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream. Contributed by Chris Nauroth (Revision 1458741) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458749 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.0.5-beta Attachments: MAPREDUCE-5075.1.patch DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13607491#comment-13607491 ] Hudson commented on MAPREDUCE-5075: --- Integrated in Hadoop-trunk-Commit #3494 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3494/]) MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream. Contributed by Chris Nauroth (Revision 1458741) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.0.5-beta Attachments: MAPREDUCE-5075.1.patch DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13607501#comment-13607501 ] Hudson commented on MAPREDUCE-5075: --- Integrated in Hadoop-trunk-Commit #3495 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3495/]) Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458749 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.0.5-beta Attachments: MAPREDUCE-5075.1.patch DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13607569#comment-13607569 ] Hudson commented on MAPREDUCE-5075: --- Integrated in Hadoop-Hdfs-trunk #1350 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1350/]) Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749) MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream. Contributed by Chris Nauroth (Revision 1458741) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458749 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.0.5-beta Attachments: MAPREDUCE-5075.1.patch DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13607625#comment-13607625 ] Hudson commented on MAPREDUCE-5075: --- Integrated in Hadoop-Mapreduce-trunk #1378 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1378/]) Move MAPREDUCE-5075 to 2.0.5-beta in CHANGES.txt. (Revision 1458749) MAPREDUCE-5075. DistCp leaks input file handles since ThrottledInputStream does not close the wrapped InputStream. Contributed by Chris Nauroth (Revision 1458741) Result = SUCCESS szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458749 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1458741 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestIntegration.java DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 2.0.5-beta Attachments: MAPREDUCE-5075.1.patch DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13603903#comment-13603903 ] Chris Nauroth commented on MAPREDUCE-5075: -- I discovered this while testing on Windows, where file locking is enforced more strictly. The DistCp tests would fail sporadically due to not being able to delete the temp files. I have a patch in progress. DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5075) DistCp leaks input file handles
[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13603992#comment-13603992 ] Hadoop QA commented on MAPREDUCE-5075: -- {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12573975/MAPREDUCE-5075.1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 tests included appear to have a timeout.{color} {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-tools/hadoop-distcp. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3421//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3421//console This message is automatically generated. DistCp leaks input file handles --- Key: MAPREDUCE-5075 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 Project: Hadoop Map/Reduce Issue Type: Bug Components: distcp Affects Versions: 3.0.0 Reporter: Chris Nauroth Assignee: Chris Nauroth Attachments: MAPREDUCE-5075.1.patch DistCp wraps the {{InputStream}} for each input file it reads in an instance of {{ThrottledInputStream}}. This class does not close the wrapped {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the {{ThrottledInputStream}} gets closed, but without closing the underlying wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira