Rohini Palaniswamy created TEZ-1260:
---------------------------------------
Summary: Allow KeyValueWriter to support writing list of values
also
Key: TEZ-1260
URL: https://issues.apache.org/jira/browse/TEZ-1260
Project: Apache Tez
Issue Type: Improvement
Reporter: Rohini Palaniswamy
TEZ-1228 adds support to IFile for storing K,L<V>. Currently KeyValueWriter
allows write of K,V
public void write(Object key, Object value) throws IOException;
We should add support for
public void write(Object key, Iterable<Object> values) throws IOException;
taking advantage of TEZ-1228. In few cases, pig unwraps key, list<values> and
writes them as separate K,V pairs. This can avoid that overhead. That may
enable us to even add something similar to hash based partial aggregation for
join like what we do for groupby.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
