Re: NPE while reading ORC file using Spark 1.4 API

I think this is fixed in 1.5 (release soon), by https://github.com/apache/spark/pull/8407 On Tue, Sep 8, 2015 at 11:39 AM, unk1102 wrote: > Hi I read many ORC files in Spark and process it those files are basically > Hive partitions. Most of the times processing goes well

NPE while reading ORC file using Spark 1.4 API

Hi I read many ORC files in Spark and process it those files are basically Hive partitions. Most of the times processing goes well but for few files I get the following exception dont know why? These files are working fine in Hive using Hive queries. Please guide. Thanks in advance. DataFrame df

Re: NPE while reading ORC file using Spark 1.4 API

Hi Zhan, thanks for the reply. Yes schema should be same actually I am reading Hive table partitions as ORC format into Spark. So I believe it should be same. I am new to Hive so dont know if schema can be different in Hive partitioned table. On Wed, Sep 9, 2015 at 12:16 AM, Zhan Zhang