Interesting. So all the planning & execution is done by Hive, and Beam will process the results of the query?
On Wed, Dec 7, 2016 at 8:24 AM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > Hi > > The HiveIO will directly use the native API and HiveQL. That's the plan on > which we are working right now. > > Regards > JB > > On Dec 7, 2016, 17:18, at 17:18, Vinoth Chandar <vin...@uber.com> wrote: > >Hi, > > > >I am not looking for a way to actually execute the query on Hive. I > >would > >like to do something similar to Spark SQL/HiveContext, but with Beam. > >Just > >have a HiveIO that reads metadata from Hive metastore, and then later > >use a > >Spark runner to execute the query. So, HiveJDBC is not an option I > >would > >like to pursue. Thanks for the pointer, though! > > > >And does the HiveIO that is being planned, work similarly as above? > > > > > >Thanks > >Vinoth > > > > > > > >On Tue, Dec 6, 2016 at 4:55 AM, Ismaël Mejía <ieme...@gmail.com> wrote: > > > >> Hello, > >> > >> If you really need to read/write via Hive, remember that you can use > >the > >> Hive Jdbc driver, and achieve this with Beam using the JdbcIO (this > >is > >> probably less efficient for the streaming case but still a valid > >solution). > >> > >> Ismaël > >> > >> > >> On Tue, Dec 6, 2016 at 12:04 PM, Vinoth Chandar <vin...@uber.com> > >wrote: > >> > >> > Great. Thanks! > >> > > >> > Thanks, > >> > Vinoth > >> > > >> > > On Dec 6, 2016, at 2:06 AM, Jean-Baptiste Onofré > ><j...@nanthrax.net> > >> > wrote: > >> > > > >> > > Hi, > >> > > > >> > > Ismaël and I started HiveIO. > >> > > > >> > > I have several IOs ready to propose as PR, but, in order to limit > >the > >> > number of open PRs, I would like to merge the pending ones. > >> > > > >> > > I will let you know when the branches/PRs will be available. > >> > > > >> > > Regards > >> > > JB > >> > > > >> > >> On 12/05/2016 11:40 PM, Vinoth Chandar wrote: > >> > >> Hi guys, > >> > >> > >> > >> Saw a post around HiveIO on the users list with a PR followup. I > >am > >> > >> interested in this too and can pitch in on developement and > >testing.. > >> > >> > >> > >> Who & where is this work happening? > >> > >> > >> > >> Thanks > >> > >> VInoth > >> > >> > >> > > > >> > > -- > >> > > Jean-Baptiste Onofré > >> > > jbono...@apache.org > >> > > http://blog.nanthrax.net > >> > > Talend - http://www.talend.com > >> > > >> >
