it seems that if I put too many records into the same mapper output
key, all these records are grouped into one key one one reducer,
then the reducer became out of memory.
but the reducer interface is:
public void reduce(K key, Iterator<V> values,
OutputCollector<K, V> output,
Reporter reporter)
so all the values belonging to the key can be iterated, so
theoretically they can be iterated from disk, and does not have to be
in memory at the same time,
so why am I getting out of heap error? is there some param I could
tune (apart from -Xmx since my box is ultimately bounded in memory
capacity)
thanks
Yang
