[ https://issues.apache.org/jira/browse/MAPREDUCE-6488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14901860#comment-14901860 ]
He Tianyi commented on MAPREDUCE-6488: -------------------------------------- Went through this again, buffer size has nothing to do with this. Please mark as Invalid. > Make buffer size in PipeMapRed configurable > ------------------------------------------- > > Key: MAPREDUCE-6488 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6488 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Reporter: He Tianyi > Assignee: He Tianyi > > Default value of buffer size is 128K in {{PipeMapRed}}. > When mapper input record is large enough that it won't fit in buffer, > {{MapRunner}} blocks until written. If child process and input reader are > both slow (due to calculation and decompress), then process of decoding and > reading will rarely overlap with each other, hurting performance. > I suppose we should make the buffer size configurable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)