Shuffle Spill (Memory) greater than Shuffle Spill (Disk)

Hello! In my spark job, I see that Shuffle Spill (Memory) is greater than Shuffle Spill (Disk). spark.shuffle.compress parameter is left to default(true?). I would expect the size on disk to be smaller which isn't the case here. I've been having some performance issues as well and I suspect this

Re: subscribe

You should email users-subscr...@kafka.apache.org if you are trying to subscribe. On 3 January 2016 at 11:52, Rajdeep Dua wrote: > >