Hi Ayan, Thank you for replying. But I wanna create a table in Zeppelin and store the metadata in Alluxio like I tried to do set hive.metastore.warehouse.dir=alluxio://master1:19998/metadb <alluxio://master1:19998/metadb> <>So I can share data with STS.
The way you’ve mentioned through JDBC I already did and it works but I can’t create table in Spark way easily. Regards, Chanh > On Jul 13, 2016, at 12:06 PM, ayan guha <guha.a...@gmail.com> wrote: > > HI > > I quickly tried with available hive interpreter > > <image.png> > > Please try similarly. > > I will try with jdbc interpreter but I need to add it to zeppelin :) > > Best > Ayan > > On Wed, Jul 13, 2016 at 1:53 PM, Chanh Le <giaosu...@gmail.com > <mailto:giaosu...@gmail.com>> wrote: > Hi Ayan, > How to set hive metastore in Zeppelin. I tried but not success. > The way I do I add into Spark Interpreter > > <Screen Shot 2016-07-13 at 10.50.53 AM.png> > > And also try in a notebook by > %sql > set hive.metastore.metadb.dir=alluxio://master1:19998/metadb <> > > %sql > set hive.metastore.warehouse.dir=alluxio://master1:19998/metadb <> > > %spark > sqlContext.setConf("hive.metastore.warehouse.dir", > "alluxio://master1:19998/metadb <>") > sqlContext.setConf("hive.metastore.metadb.dir", > "alluxio://master1:19998/metadb <>") > sqlContext.read.parquet("alluxio://master1:19998/etl_info/WEBSITE > <>").saveAsTable("tests_5”) > > But It’s <Screen Shot 2016-07-13 at 10.53.10 AM.png> > >> On Jul 11, 2016, at 1:26 PM, ayan guha <guha.a...@gmail.com >> <mailto:guha.a...@gmail.com>> wrote: >> >> Hi >> >> When you say "Zeppelin and STS", I am assuming you mean "Spark Interpreter" >> and "JDBC interpreter" respectively. >> >> Through Zeppelin, you can either run your own spark application (by using >> Zeppelin's own spark context) using spark interpreter OR you can access STS, >> which is a spark application ie separate Spark Context using JDBC >> interpreter. There should not be any need for these 2 contexts to coexist. >> >> If you want to share data, save it to hive from either context, and you >> should be able to see the data from other context. >> >> Best >> Ayan >> >> >> >> On Mon, Jul 11, 2016 at 3:00 PM, Chanh Le <giaosu...@gmail.com >> <mailto:giaosu...@gmail.com>> wrote: >> Hi Ayan, >> I tested It works fine but one more confuse is If my (technical) users want >> to write some code in zeppelin to apply thing into Hive table? >> Zeppelin and STS can’t share Spark Context that mean we need separated >> process? Is there anyway to use the same Spark Context of STS? >> >> Regards, >> Chanh >> >> >>> On Jul 11, 2016, at 10:05 AM, Takeshi Yamamuro <linguin....@gmail.com >>> <mailto:linguin....@gmail.com>> wrote: >>> >>> Hi, >>> >>> ISTM multiple sparkcontexts are not recommended in spark. >>> See: https://issues.apache.org/jira/browse/SPARK-2243 >>> <https://issues.apache.org/jira/browse/SPARK-2243> >>> >>> // maropu >>> >>> >>> On Mon, Jul 11, 2016 at 12:01 PM, ayan guha <guha.a...@gmail.com >>> <mailto:guha.a...@gmail.com>> wrote: >>> Hi >>> >>> Can you try using JDBC interpreter with STS? We are using Zeppelin+STS on >>> YARN for few months now without much issue. >>> >>> On Mon, Jul 11, 2016 at 12:48 PM, Chanh Le <giaosu...@gmail.com >>> <mailto:giaosu...@gmail.com>> wrote: >>> Hi everybody, >>> We are using Spark to query big data and currently we’re using Zeppelin to >>> provide a UI for technical users. >>> Now we also need to provide a UI for business users so we use Oracle BI >>> tools and set up a Spark Thrift Server (STS) for it. >>> >>> When I run both Zeppelin and STS throw error: >>> >>> INFO [2016-07-11 09:40:21,905] ({pool-2-thread-4} >>> SchedulerFactory.java[jobStarted]:131) - Job >>> remoteInterpretJob_1468204821905 started by scheduler >>> org.apache.zeppelin.spark.SparkInterpreter835015739 >>> INFO [2016-07-11 09:40:21,911] ({pool-2-thread-4} >>> Logging.scala[logInfo]:58) - Changing view acls to: giaosudau >>> INFO [2016-07-11 09:40:21,912] ({pool-2-thread-4} >>> Logging.scala[logInfo]:58) - Changing modify acls to: giaosudau >>> INFO [2016-07-11 09:40:21,912] ({pool-2-thread-4} >>> Logging.scala[logInfo]:58) - SecurityManager: authentication disabled; ui >>> acls disabled; users with view permissions: Set(giaosudau); users with >>> modify permissions: Set(giaosudau) >>> INFO [2016-07-11 09:40:21,918] ({pool-2-thread-4} >>> Logging.scala[logInfo]:58) - Starting HTTP Server >>> INFO [2016-07-11 09:40:21,919] ({pool-2-thread-4} >>> Server.java[doStart]:272) - jetty-8.y.z-SNAPSHOT >>> INFO [2016-07-11 09:40:21,920] ({pool-2-thread-4} >>> AbstractConnector.java[doStart]:338) - Started >>> SocketConnector@0.0.0.0:54818 <http://SocketConnector@0.0.0.0:54818/> >>> INFO [2016-07-11 09:40:21,922] ({pool-2-thread-4} >>> Logging.scala[logInfo]:58) - Successfully started service 'HTTP class >>> server' on port 54818. >>> INFO [2016-07-11 09:40:22,408] ({pool-2-thread-4} >>> SparkInterpreter.java[createSparkContext]:233) - ------ Create new >>> SparkContext local[*] ------- >>> WARN [2016-07-11 09:40:22,411] ({pool-2-thread-4} >>> Logging.scala[logWarning]:70) - Another SparkContext is being constructed >>> (or threw an exception in its constructor). This may indicate an error, >>> since only one SparkContext may be running in this JVM (see SPARK-2243). >>> The other SparkContext was created at: >>> >>> Is that mean I need to setup allow multiple context? Because It’s only test >>> in local with local mode If I deploy on mesos cluster what would happened? >>> >>> Need you guys suggests some solutions for that. Thanks. >>> >>> Chanh >>> --------------------------------------------------------------------- >>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >>> <mailto:user-unsubscr...@spark.apache.org> >>> >>> >>> >>> >>> -- >>> Best Regards, >>> Ayan Guha >>> >>> >>> >>> -- >>> --- >>> Takeshi Yamamuro >> >> >> >> >> -- >> Best Regards, >> Ayan Guha > > > > > -- > Best Regards, > Ayan Guha
