Re: [spark1.5.1] HiveQl.parse throws org.apache.spark.sql.AnalysisException: null

Hi, Sebastian, To use private APIs, you have to be very familiar with the code path; otherwise, it is very easy to hit an exception or a bug. My suggestion is to use IntelliJ to step-by-step step in the function hiveContext.sql until you hit the parseSql API. Then, you will know if you have to

Re: [spark1.5.1] HiveQl.parse throws org.apache.spark.sql.AnalysisException: null

What we're trying to achieve is a fast way of testing the validity of our SQL queries within Unit tests without going through the time consuming task of setting up an Hive Test Context. If there is any way to speed this step up, any help would be appreciated. Thanks, Sebastian *Sebastian Nadorp*

[spark1.5.1] HiveQl.parse throws org.apache.spark.sql.AnalysisException: null

Hello, when upgrading to spark 1.5.1 from 1.4.1 the following code crashed on runtime. It is mainly used to parse HiveQL queries and check that they are valid. package org.apache.spark.sql.hive val sql = "CREATE EXTERNAL TABLE IF NOT EXISTS `t`(`id` STRING, `foo` INT) PARTITIONED BY (year INT,

Re: [spark1.5.1] HiveQl.parse throws org.apache.spark.sql.AnalysisException: null

Just curious why you are using parseSql APIs? It works well if you use the external APIs. For example, in your case: val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) hiveContext.sql("CREATE EXTERNAL TABLE IF NOT EXISTS `t`(`id` STRING, `foo` INT) PARTITIONED BY (year INT, month

Re: [spark1.5.1] HiveQl.parse throws org.apache.spark.sql.AnalysisException: null

Thats not really intended to be a public API as there is some internal setup that needs to be done for Hive to work. Have you created a HiveContext in the same thread? Is there more to that stacktrace? On Tue, Oct 20, 2015 at 2:25 AM, Ayoub wrote: > Hello, > >