[ https://issues.apache.org/jira/browse/HDFS-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14183664#comment-14183664 ]
stack commented on HDFS-7276: ----------------------------- bq. Yes, limiting the number of arrays also limits the number of outstanding packets. Thinking on it, the patch only limits the outstanding number of Packets if Packets always allocate same size buffers, right? If the objective is a limit on Packet count, perhaps a more direct choke on Packet allocations would be more effective? Meantime, I like the buffer reuse that this patch brings. > Limit the number of byte arrays used by DFSOutputStream > ------------------------------------------------------- > > Key: HDFS-7276 > URL: https://issues.apache.org/jira/browse/HDFS-7276 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client > Reporter: Tsz Wo Nicholas Sze > Assignee: Tsz Wo Nicholas Sze > Attachments: h7276_20141021.patch, h7276_20141022.patch, > h7276_20141023.patch, h7276_20141024.patch > > > When there are a lot of DFSOutputStream's writing concurrently, the number of > outstanding packets could be large. The byte arrays created by those packets > could occupy a lot of memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)