[
https://issues.apache.org/jira/browse/TEZ-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054201#comment-14054201
]
Bikas Saha commented on TEZ-1260:
---------------------------------
bq. add something similar to hash based partial aggregation for join like what
we do for groupby
Can you please elaborate on that?
> Allow KeyValueWriter to support writing list of values also
> -----------------------------------------------------------
>
> Key: TEZ-1260
> URL: https://issues.apache.org/jira/browse/TEZ-1260
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Rohini Palaniswamy
>
> TEZ-1228 adds support to IFile for storing K,L<V>. Currently KeyValueWriter
> allows write of K,V
> public void write(Object key, Object value) throws IOException;
> We should add support for
> public void write(Object key, Iterable<Object> values) throws IOException;
> taking advantage of TEZ-1228. In few cases, pig unwraps key, list<values> and
> writes them as separate K,V pairs. This can avoid that overhead. That may
> enable us to even add something similar to hash based partial aggregation for
> join like what we do for groupby.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
