into the disk temporarily and use disk files to do the join.
Best Regards,
Shixiong Zhu
2015-03-13 9:37 GMT+08:00 Soila Pertet Kavulya skavu...@gmail.com:
Does Spark support skewed joins similar to Pig which distributes large
keys over multiple partitions? I tried using the RangePartitioner
.
Best Regards,
Shixiong Zhu
2015-03-13 9:37 GMT+08:00 Soila Pertet Kavulya skavu...@gmail.com:
Does Spark support skewed joins similar to Pig which distributes large
keys over multiple partitions? I tried using the RangePartitioner but
I am still experiencing failures because some keys are too
9:37 GMT+08:00 Soila Pertet Kavulya skavu...@gmail.com:
Does Spark support skewed joins similar to Pig which distributes large
keys over multiple partitions? I tried using the RangePartitioner but
I am still experiencing failures because some keys are too large to
fit in a single partition. I
Does Spark support skewed joins similar to Pig which distributes large
keys over multiple partitions? I tried using the RangePartitioner but
I am still experiencing failures because some keys are too large to
fit in a single partition. I cannot use broadcast variables to
work-around this because