To: Praveen Garg praveen.g...@guavus.commailto:praveen.g...@guavus.com
Cc: user@spark.apache.orgmailto:user@spark.apache.org
user@spark.apache.orgmailto:user@spark.apache.org
Subject: Re: Shuffle read/write issue in spark 1.2
Even I observed the same issue.
On Fri, Feb 6, 2015 at 12:19 AM, Praveen
...@guavus.com
Cc: user@spark.apache.org user@spark.apache.org
Subject: Re: Shuffle read/write issue in spark 1.2
Even I observed the same issue.
On Fri, Feb 6, 2015 at 12:19 AM, Praveen Garg praveen.g...@guavus.com
wrote:
Hi,
While moving from spark 1.1 to spark 1.2, we are facing an issue
@spark.apache.org
Subject: Re: Shuffle read/write issue in spark 1.2
Did the problem go away when you switched to lz4? There was a change from the
default compression codec fro 1.0 to 1.1, where we went from LZF to Snappy. I
don't think there was any such change from 1.1 to 1.2, though.
On Fri, Feb 6
Even I observed the same issue.
On Fri, Feb 6, 2015 at 12:19 AM, Praveen Garg praveen.g...@guavus.com
wrote:
Hi,
While moving from spark 1.1 to spark 1.2, we are facing an issue where
Shuffle read/write has been increased significantly. We also tried running
the job by rolling back to
