Re: Spark Sql 2.0 throws null pointer exception

2016-10-24 Thread Selvam Raman
​Why i could not able to access sparksession instance within foreachpartition(i have created sparksession instance within main fucntion). spark.sql("select 1").count or any sql queries which return within foreachpartition throws nullpointer exception. please give me some idea if you have faced the

Spark Sql 2.0 throws null pointer exception

2016-10-24 Thread Selvam Raman
Hi All, Please help me. I have 10 (tables data) parquet file in s3. I am reading and storing as Dataset then registered as temp table. One table driving whole flow so i am doing below.(When i am triggering query from Code Base: SparkSession spark = SparkSession.builder().appName("Test").getO