Re: How spark and hive integrate in long term?

2014-11-22 Thread Patrick Wendell
; two projects are undergoing fast improvement and changes. Currently, my >>>> understanding is that spark hive sql part relies on hive meta store and >>>> basic parser to operate, and the thrift-server intercept hive query and >>>> replace it with its own engi

Re: How spark and hive integrate in long term?

2014-11-22 Thread Zhan Zhang
the metastore part, we may possibly replace it with hcatalog. But given >>> the dependency of other parts on hive, e.g., metastore, thriftserver, >>> hcatlog may not be able to help much. >>> >>> Does anyone have any insight or idea in mind? >>> >&g

Re: How spark and hive integrate in long term?

2014-11-22 Thread Cheng Lian
catlog may not be able to help much. Does anyone have any insight or idea in mind? Thanks. Zhan Zhang -- View this message in context:http://apache-spark-developers-list.1001551.n3.nabble.com/How-spark-and-hive-integrate-in-long-term-tp9482.html Sent from the Apache Spark Developers List mailin

Re: How spark and hive integrate in long term?

2014-11-22 Thread Cheng Lian
server, hcatlog may not be able to help much. Does anyone have any insight or idea in mind? Thanks. Zhan Zhang -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/How-spark-and-hive-integrate-in-long-term-tp9482.html Sent from the Apache Spark Developers List m

Re: How spark and hive integrate in long term?

2014-11-21 Thread Ted Yu
; given > >> the dependency of other parts on hive, e.g., metastore, thriftserver, > >> hcatlog may not be able to help much. > >> > >> Does anyone have any insight or idea in mind? > >> > >> Thanks. > >> > >> Zhan Zhang &g

Re: How spark and hive integrate in long term?

2014-11-21 Thread Zhan Zhang
gt;> >> Does anyone have any insight or idea in mind? >> >> Thanks. >> >> Zhan Zhang >> >> >> >> -- >> View this message in context: >> http://apache-spark-developers-list.1001551.n3.nabble.com/How-spark-and-hive-integr

Re: How spark and hive integrate in long term?

2014-11-21 Thread Dean Wampler
the metastore part, we may possibly replace it with hcatalog. But given > the dependency of other parts on hive, e.g., metastore, thriftserver, > hcatlog may not be able to help much. > > Does anyone have any insight or idea in mind? > > Thanks. > > Zhan Zhang > > > >

How spark and hive integrate in long term?

2014-11-21 Thread Zhan Zhang
on hive, e.g., metastore, thriftserver, hcatlog may not be able to help much. Does anyone have any insight or idea in mind? Thanks. Zhan Zhang -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/How-spark-and-hive-integrate-in-long-term-tp9482.html Sent