The program does not fail consistently. We are executing the spark driver as a
standalone Java program. If the cause was a null value in a field, I would
assume the program will fail every time we execute it . But, it fails with
Nullpointer exception and Task failure only on certain executions even if the
input remains the same and we restart the program.
Thanks
Bijuna
Sent from my iPhone
> On Feb 17, 2016, at 3:18 AM, Sudhanshu Janghel
> <sudhanshu.jang...@cloudwick.com> wrote:
>
> I think that the value in the data frame is null for some field. Why not
> insert a check for it . For e.g. Say field is not null then convert to a df.
>
> Kind Regards,
> Sudhanshu
>
>> On 16 Feb 2016, at 8:03 pm, Bijuna <bij...@gmail.com> wrote:
>>
>> Spark Experts,
>>
>> We are using Spark 1.6 Java version in standalone mode in Windows . Our
>> program reads data from text files using javasparkcontext and SQLServer
>> using SQLContext.
>>
>> We use DataFrame API to filter, join and aggregate data read from above
>> sources. The collectAsList operation on the DataFrame fails intermittently
>> with Nullpointer and Task failure exception. I have attached two screenshots
>> of the errors we see.
>>
>> Any help in debugging and fixing this error will be much appreciated.
>>
>> Thank you,
>> Bijuna
>>
>>
>> Sent from my iPad
>>
>>
>>
>>>
>>>
>>>
>>> Sent from my iPad
>>
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
