[ https://issues.apache.org/jira/browse/HADOOP-16242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16827238#comment-16827238 ]
Da Zhou commented on HADOOP-16242: ---------------------------------- Opened PR using github: https://github.com/apache/hadoop/pull/776 > ABFS: add bufferpool to AbfsOutputStream > ---------------------------------------- > > Key: HADOOP-16242 > URL: https://issues.apache.org/jira/browse/HADOOP-16242 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure > Affects Versions: 3.2.0 > Reporter: Da Zhou > Assignee: Da Zhou > Priority: Major > Attachments: HADOOP-16242-001.patch, > WritePerformanceWithBufferPool.pdf > > > In current AbfsOutPutStream, new temporary buffers always get allocated when > uploading the data to server. These big chunk creates huge pressure on the > GC, and in some extreme cases that GC doesn't happen in time(observed in some > customer's env), it also lead to OutOfMemory issue easily. > Hence we need to replace the current implementation with a bufferpool. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org