Wei-Chiu Chuang created HDFS-9612: ------------------------------------- Summary: DistCp worker threads are not terminated after jobs are done. Key: HDFS-9612 URL: https://issues.apache.org/jira/browse/HDFS-9612 Project: Hadoop HDFS Issue Type: Bug Components: distcp Affects Versions: 2.8.0 Reporter: Wei-Chiu Chuang Assignee: Wei-Chiu Chuang
In HADOOP-11827, a producer-consumer style thread pool was introduced to parallelize the task of listing files/directories. We have a use case where a distcp job is run during the commit phase of a MR2 job. However, it was found distcp does not terminate ProducerConsumer thread pools properly. Because threads are not terminated, those MR2 jobs never finish. In a more typical use case where distcp is run as a standalone job, those threads are terminated forcefully when the java process is terminated. So these leaked threads did not become a problem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)