He Tianyi created MAPREDUCE-6488:
------------------------------------
Summary: Make buffer size in PipeMapRed configurable
Key: MAPREDUCE-6488
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6488
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: He Tianyi
Assignee: He Tianyi
Default value of buffer size is 128K in {{PipeMapRed}}.
When mapper input record is large enough that it won't fit in buffer,
{{MapRunner}} blocks until written. If child process and input reader are both
slow (due to calculation and decompress), then process of decoding and reading
will rarely overlap with each other, hurting performance.
I suppose we should make the buffer size configurable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
