Teng Qiu created SPARK-13983:
--------------------------------
Summary: HiveThriftServer2 can not get "--hiveconf" or
''--hivevar" variables since 1.6 version (both multi-session and single session)
Key: SPARK-13983
URL: https://issues.apache.org/jira/browse/SPARK-13983
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.6.1, 1.6.0
Environment: ubuntu, spark 1.6.0 standalone, spark 1.6.1 standalone
(tried spark branch-1.6 snapshot as well)
Reporter: Teng Qiu
HiveThriftServer2 should be able to get "\--hiveconf" or ''\-\-hivevar"
variables from JDBC client, either from command line parameter of beeline, such
as
{{beeline --hiveconf hive.stats.autogather=false --hivevar db_name=somelog}}
or from JDBC connection string, like
{{jdbc:hive2://localhost:10000?hive.stats.autogather=false#db_name=somelog}}
this works in spark version 1.5.x, but after upgraded to 1.6, it doesn't work.
to reproduce this issue, try to connect to HiveThriftServer2 with beeline:
{code}
bin/beeline -u jdbc:hive2://localhost:10000 \
--hiveconf hive.stats.autogather=false \
--hivevar db_name=somelog
{code}
or
{code}
bin/beeline -u
jdbc:hive2://localhost:10000?hive.stats.autogather=false#db_name=somelog
{code}
will get following results:
{code}
0: jdbc:hive2://localhost:10000> set hive.stats.autogather;
+-----------------------+--------------+--+
| key | value |
+-----------------------+--------------+--+
| hive.stats.autogather | <undefined> |
+-----------------------+--------------+--+
1 row selected (0.01 seconds)
0: jdbc:hive2://localhost:10000> use ${db_name};
Error: org.apache.spark.sql.AnalysisException: cannot recognize input near '$'
'{' 'db_name' in switch database statement; line 1 pos 4 (state=,code=0)
{code}
-
but this bug does not affect current versions of spark-sql CLI, following
commands works:
{code}
bin/spark-sql --master local[2] \
--hiveconf hive.stats.autogather=false \
--hivevar db_name=somelog
spark-sql> set hive.stats.autogather;
hive.stats.autogather false
Time taken: 1.037 seconds, Fetched 1 row(s)
spark-sql> use ${db_name};
OK
Time taken: 1.697 seconds
{code}
so I think it may caused by this change: [SPARK-10810] [SPARK-10902] [SQL]
Improve session management in SQL https://github.com/apache/spark/pull/8909
perhaps by calling {{hiveContext.newSession}}, the variables from
{{sessionConf}} were not loaded into the new session?
(https://github.com/apache/spark/pull/8909/files#diff-8f8b7f4172e8a07ff20a4dbbbcc57b1dR69)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
