Jiandan Yang created YARN-7168: ----------------------------------- Summary: The size of dataQueue and ackQueue in DataStreamer has no limit when writer thread is interrupted Key: YARN-7168 URL: https://issues.apache.org/jira/browse/YARN-7168 Project: Hadoop YARN Issue Type: Bug Components: client Reporter: Jiandan Yang
In our cluster, when found NodeManager frequently FullGC when decommissioning NodeManager, and we found the biggest object is dataQueue of DataStreamer, it has almost 6w DFSPacket, and every DFSPacket is about 64k. !mat.jpg|memory_analysis! The root reason is that the size of dataQueue and ackQueue in DataStreamer has no limit when writer thread is interrupted. I know NodeManager may stop writing when interruped, but DFSOutputStream also could do something to avoid fullgc -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org