Hi Shixiong, > The Iterable from cogroup is CompactBuffer, which is already > materialized. It's not a lazy Iterable. So now Spark cannot handle > skewed data that some key has too many values that cannot be fit into > the memory.
Cool, thanks for the confirmation. - Stephen --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
