Hello Ashok, I found three sources of how shuffle works (and what transformations trigger it) instructive and illuminative. After learning from it, you should be able to extrapolate how your particular and practical use case would work.
http://www.slideshare.net/databricks/strata-sj-everyday-im-shuffling-tips-for-writing-better-spark-programs <http://www.slideshare.net/databricks/strata-sj-everyday-im-shuffling-tips-for-writing-better-spark-programs> https://spark-summit.org/2014/talk/a-deeper-understanding-of-spark-internals <https://spark-summit.org/2014/talk/a-deeper-understanding-of-spark-internals> http://www.slideshare.net/databricks/spark-summit-east-2015-advdevopsstudentslides <http://www.slideshare.net/databricks/spark-summit-east-2015-advdevopsstudentslides> Associated with each presentation or training are videos from Spark Summit that elaborate it. I would view them. Cheers Jules > On Mar 14, 2016, at 2:59 PM, Ashok Kumar <ashok34...@yahoo.com.INVALID> wrote: > > experts, > > please I need to understand how shuffling works in Spark and which parameters > influence it. > > I am sorry but my knowledge of shuffling is very limited. Need a practical > use case if you can. > > regards
