Re: How does HashPartitioner distribute data in Spark?

2017-06-25 Thread Russell Spitzer
t;>> In both the cases data is distributed uniformaly. >>>> I do have following questions on the basis of above observation: >>>> >>>> 1. In case of rdd1, hash partitioning should calculate hashcode of key >>>> (i.e. "aa" in

Re: How does HashPartitioner distribute data in Spark?

2017-06-24 Thread Russell Spitzer
t;>> I do have following questions on the basis of above observation: >>> >>> 1. In case of rdd1, hash partitioning should calculate hashcode of key >>> (i.e. "aa" in this case), so all records should go to single partition >>> instead of uniform dist

Re: How does HashPartitioner distribute data in Spark?

2017-06-24 Thread Vikash Pareek
calculate hashcode of key >> (i.e. "aa" in this case), so all records should go to single partition >> instead of uniform distribution? >> 2. In case of rdd2, there is no key value pair so how hash partitoning >> going to work i.e. what is the key to calculate hashcode? &

Re: How does HashPartitioner distribute data in Spark?

2017-06-23 Thread Vadim Semenov
instead of uniform distribution? > 2. In case of rdd2, there is no key value pair so how hash partitoning > going to work i.e. what is the key to calculate hashcode? > > I have followed @zero323 answer but not getting answer of these. > https://stackoverflow.com/questions/314243

How does HashPartitioner distribute data in Spark?

2017-06-23 Thread Vikash Pareek
late hashcode? I have followed @zero323 answer but not getting answer of these. https://stackoverflow.com/questions/31424396/how-does-hashpartitioner-work ----- __Vikash Pareek -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-does-HashPartitioner-d