On Fri, Oct 30, 2015 at 4:31 AM, karthik kadiyam <
> karthik.kadiyam...@gmail.com
> > wrote:
>
>> Hi,
>>
>> In spark streaming job i had the following setting
>>
>> this.jsc.getConf().set("spark.driver.maxResultSize", “0”);
>>
Did any one had issue setting spark.driver.maxResultSize value ?
On Friday, October 30, 2015, karthik kadiyam
wrote:
> Hi Shahid,
>
> I played around with spark driver memory too. In the conf file it was set
> to " --driver-memory 20G " first. When i changed the spark
guess you need to increase spark driver memory as well. But that should
> be set in conf files
> Let me know if that resolves
> On Oct 30, 2015 7:33 AM, "karthik kadiyam"
> wrote:
>
>> Hi,
>>
>> In spark streaming job i had the following setting
>
Hi,
In spark streaming job i had the following setting
this.jsc.getConf().set("spark.driver.maxResultSize", “0”);
and i got the error in the job as below
User class threw exception: Job aborted due to stage failure: Total size of
serialized results of 120 tasks (1082.2 MB) is bigger
Hi,
In spark streaming job i had the following setting
this.jsc.getConf().set("spark.driver.maxResultSize", “0”);
and i got the error in the job as below
User class threw exception: Job aborted due to stage failure: Total size of
serialized results of 120 tasks (1082.2 MB) is bigge
