Re: Spark 2.2.0 - Odd Hive SQL Warnings

2017-09-03 Thread Liang-Chi Hsieh
Hi Don, There is a new SQL config `spark.sql.hive.caseSensitiveInferenceMode` which sets the action to take when a case-sensitive schema cannot be read from a Hive table's properties. The default setting of this config is `INFER_AND_SAVE` that goes toinfer the case-sensitive schema from the

Spark 2.2.0 - Odd Hive SQL Warnings

2017-09-01 Thread Don Drake
I'm in the process of migrating a few applications from Spark 2.1.1 to Spark 2.2.0 and so far the transition has been smooth. One odd thing is that when I query a Hive table that I do not own, but have read access, I get a very long WARNING with a stack trace that basically says I do not have