Combiner not used when group elements referred to in tuple notation instead of
flatten.
---------------------------------------------------------------------------------------
Key: PIG-490
URL: https://issues.apache.org/jira/browse/PIG-490
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Alan Gates
Fix For: types_branch
Given a query like:
{code}
A = load 'myfile';
B = group A by ($0, $1);
C = foreach B generate group.$0, group.$1, COUNT(A);
{code}
The combiner will not be invoked. But if the last line is changed to:
{code}
C = foreach B generate flatten(group), COUNT(A);
{code}
it will be. The reason for the discrepancy is because the CombinerOptimizer
checks that all of the projections are simple. If not, it does not use the
combiner. group.$0 is not a simple projection, so this is failed. However,
this is a common enough case that the CombinerOptimizer should detect it and
still use the combiner.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
