[ https://issues.apache.org/jira/browse/MAPREDUCE-5075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Nauroth updated MAPREDUCE-5075: ------------------------------------- Attachment: MAPREDUCE-5075.1.patch Here is a patch with the following changes: # {{RetriableFileCopyCommand}} - This is just code clean-up. The {{copyBytes}} private method accepted a flag as an argument to control whether or not to close the streams after copying. This method was only ever called from {{copyToTmpFile}} with a hard-coded true. I removed the flag from the method signature and changed the code so that it closes the streams unconditionally. # {{ThrottledInputStream}} - Override {{close}} so that it closes the wrapped stream. # {{TestIntegration}} - This code was not creating the target file correctly. {{target}} contains a fully qualified path. Inside {{createFiles}}, it prepends the test root again. This would be 2 fully qualified paths appended to each other. On Windows, the result would look like C:\project\target\C:\project\target. The second ':' makes the filename invalid. With this patch, all DistCp tests pass consistently on Mac and Windows. > DistCp leaks input file handles > ------------------------------- > > Key: MAPREDUCE-5075 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5075 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: distcp > Affects Versions: 3.0.0 > Reporter: Chris Nauroth > Assignee: Chris Nauroth > Attachments: MAPREDUCE-5075.1.patch > > > DistCp wraps the {{InputStream}} for each input file it reads in an instance > of {{ThrottledInputStream}}. This class does not close the wrapped > {{InputStream}}. {{RetriableFileCopyCommand}} guarantees that the > {{ThrottledInputStream}} gets closed, but without closing the underlying > wrapped stream, it still leaks a file handle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira