I registered it in a new Spark SQL CLI. Yeah I thought so too about how the
temp tables were accessible across different applications without using a
job-server. I see that running*
HiveThriftServer2.startWithContext(hiveContext) *within the spark app
starts up a thrift server.
On Tue, Aug 25, 2015 at 5:32 PM, Cheng, Hao hao.ch...@intel.com wrote:
Did you register temp table via the beeline or in a new Spark SQL CLI?
As I know, the temp table cannot cross the HiveContext.
Hao
*From:* Udit Mehta [mailto:ume...@groupon.com]
*Sent:* Wednesday, August 26, 2015 8:19 AM
*To:* user
*Subject:* Spark thrift server on yarn
Hi,
I am trying to start a spark thrift server using the following command on
Spark 1.3.1 running on yarn:
* ./sbin/start-thriftserver.sh --master yarn://resourcemanager.snc1:8032
--executor-memory 512m --hiveconf
hive.server2.thrift.bind.host=test-host.sn1 --hiveconf
hive.server2.thrift.port=10001 --queue public*
It starts up fine and is able to connect to the hive metastore.
I now need to view some temporary tables using this thrift server so I
start up SparkSql and register a temp table.
But the problem is that I am unable to view the temp table using the
beeline client. I am pretty sure I am going wrong somewhere and the spark
documentation does not clearly say how to run the thrift server in yarn
mode or maybe I missed something.
Could someone tell me how this is to be done or point me to some
documentation?
Thanks in advance,
Udit