Re: Spark-hive integration on HDInsight
Hey jay How you are making your cluster are you using spark cluster All this thing should be set up automatically Sent from my iPhone > On Feb 21, 2019, at 12:12 PM, Felix Cheung wrote: > > You should check with HDInsight support > > From: Jay Singh > Sent: Wednesday, February 20, 2019 11:43:23 PM > To: User > Subject: Spark-hive integration on HDInsight > > I am trying to integrate spark with hive on HDInsight spark cluster . > I copied hive-site.xml in spark/conf directory. In addition I added hive > metastore properties like jdbc connection info on Ambari as well. But still > the database and tables created using spark-sql are not visible in hive. > Changed ‘spark.sql.warehouse.dir’ value also to point to hive warehouse > directory. > Spark does work with hive not having LLAP ON. What am I missing in the > configuration to integrate spark with hive ? Any pointer will be appreciated. > > thx
Re: Spark-hive integration on HDInsight
You should check with HDInsight support From: Jay Singh Sent: Wednesday, February 20, 2019 11:43:23 PM To: User Subject: Spark-hive integration on HDInsight I am trying to integrate spark with hive on HDInsight spark cluster . I copied hive-site.xml in spark/conf directory. In addition I added hive metastore properties like jdbc connection info on Ambari as well. But still the database and tables created using spark-sql are not visible in hive. Changed ‘spark.sql.warehouse.dir’ value also to point to hive warehouse directory. Spark does work with hive not having LLAP ON. What am I missing in the configuration to integrate spark with hive ? Any pointer will be appreciated. thx
Spark-hive integration on HDInsight
I am trying to integrate spark with hive on HDInsight spark cluster . I copied hive-site.xml in spark/conf directory. In addition I added hive metastore properties like jdbc connection info on Ambari as well. But still the database and tables created using spark-sql are not visible in hive. Changed 'spark.sql.warehouse.dir' value also to point to hive warehouse directory. Spark does work with hive not having LLAP ON. What am I missing in the configuration to integrate spark with hive ? Any pointer will be appreciated. thx