Can you provide a bit more detail ? values of the parameters you have tuned log snippets from executors snippet of your code
Thanks On Thu, Feb 4, 2016 at 9:48 AM, <arun.bong...@cognizant.com> wrote: > Hi Sir/madam, > Greetings of the day. > > I am working on Spark 1.6.0 with AWS EMR(Elastic Map Reduce). I'm facing > some issues in reading large(500 mb) file in spark-sql. > Sometimes i get heap space error and sometimes the executors fail. > i have increased the driver memory, executor memory, kryo serializer > buffer size.. etc.. but nothing helps. > > I Kindly request your help in resolving this issue. > > Thanks > Arun. > This e-mail and any files transmitted with it are for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. If you are not the intended recipient(s), please reply to the > sender and destroy all copies of the original message. Any unauthorized > review, use, disclosure, dissemination, forwarding, printing or copying of > this email, and/or any action taken in reliance on the contents of this > e-mail is strictly prohibited and may be unlawful. Where permitted by > applicable law, this e-mail and other e-mail communications sent to and > from Cognizant e-mail addresses may be monitored. >
