Travis Woodruff created PIG-4002:
------------------------------------
Summary: Disable combiner when map-side aggregation is used
Key: PIG-4002
URL: https://issues.apache.org/jira/browse/PIG-4002
Project: Pig
Issue Type: Improvement
Affects Versions: 0.13.0
Reporter: Travis Woodruff
Assignee: Travis Woodruff
Priority: Minor
This may be controversial, so I'd like others' opinions on this.
It is not currently possible to disable the combiner and use map-side
aggregation at the same time. This is a problematic since map-side aggregation
effectively combines in the mapper, so running the combiner adds expensive
combiner execution (combiner requires deserialization & reserialization) for
little to no value.
PIG-2829 had a patch to disable the combiner when map-side aggregation is used
(along with some other changes). This was never integrated because the map-side
aggregation code was redone while this was in progress.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
