Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
'The ShuffleWriter should treat RangePartitioner specially and consume the
sampled data in RangePartitioner instead of the input iterator.' This idea is
good, maybe we can cache both the K and V when doing sample.
I will have a try on this idea.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
