Great to see that there is progress on this! Like written on the user mailing list, I would also be interested in contributing to this.
On Thu, Dec 15, 2016 at 4:44 AM, Ismaël Mejía <ieme...@gmail.com> wrote: > For ref, I just created a JIRA so people can track the progress/contribute > to the progress of HiveIO. > > https://issues.apache.org/jira/browse/BEAM-1158 > > On Wed, Dec 7, 2016 at 5:39 PM, Jean-Baptiste Onofré <j...@nanthrax.net> > wrote: > >> Yes that's the first idea ;) >> >> Regards >> JB >> >> On Dec 7, 2016, 17:27, at 17:27, Vinoth Chandar <vin...@uber.com> wrote: >> >Interesting. So all the planning & execution is done by Hive, and Beam >> >will >> >process the results of the query? >> > >> >On Wed, Dec 7, 2016 at 8:24 AM, Jean-Baptiste Onofré <j...@nanthrax.net> >> >wrote: >> > >> >> Hi >> >> >> >> The HiveIO will directly use the native API and HiveQL. That's the >> >plan on >> >> which we are working right now. >> >> >> >> Regards >> >> JB >> >> >> >> On Dec 7, 2016, 17:18, at 17:18, Vinoth Chandar <vin...@uber.com> >> >wrote: >> >> >Hi, >> >> > >> >> >I am not looking for a way to actually execute the query on Hive. I >> >> >would >> >> >like to do something similar to Spark SQL/HiveContext, but with >> >Beam. >> >> >Just >> >> >have a HiveIO that reads metadata from Hive metastore, and then >> >later >> >> >use a >> >> >Spark runner to execute the query. So, HiveJDBC is not an option I >> >> >would >> >> >like to pursue. Thanks for the pointer, though! >> >> > >> >> >And does the HiveIO that is being planned, work similarly as above? >> >> > >> >> > >> >> >Thanks >> >> >Vinoth >> >> > >> >> > >> >> > >> >> >On Tue, Dec 6, 2016 at 4:55 AM, Ismaël Mejía <ieme...@gmail.com> >> >wrote: >> >> > >> >> >> Hello, >> >> >> >> >> >> If you really need to read/write via Hive, remember that you can >> >use >> >> >the >> >> >> Hive Jdbc driver, and achieve this with Beam using the JdbcIO >> >(this >> >> >is >> >> >> probably less efficient for the streaming case but still a valid >> >> >solution). >> >> >> >> >> >> Ismaël >> >> >> >> >> >> >> >> >> On Tue, Dec 6, 2016 at 12:04 PM, Vinoth Chandar <vin...@uber.com> >> >> >wrote: >> >> >> >> >> >> > Great. Thanks! >> >> >> > >> >> >> > Thanks, >> >> >> > Vinoth >> >> >> > >> >> >> > > On Dec 6, 2016, at 2:06 AM, Jean-Baptiste Onofré >> >> ><j...@nanthrax.net> >> >> >> > wrote: >> >> >> > > >> >> >> > > Hi, >> >> >> > > >> >> >> > > Ismaël and I started HiveIO. >> >> >> > > >> >> >> > > I have several IOs ready to propose as PR, but, in order to >> >limit >> >> >the >> >> >> > number of open PRs, I would like to merge the pending ones. >> >> >> > > >> >> >> > > I will let you know when the branches/PRs will be available. >> >> >> > > >> >> >> > > Regards >> >> >> > > JB >> >> >> > > >> >> >> > >> On 12/05/2016 11:40 PM, Vinoth Chandar wrote: >> >> >> > >> Hi guys, >> >> >> > >> >> >> >> > >> Saw a post around HiveIO on the users list with a PR >> >followup. I >> >> >am >> >> >> > >> interested in this too and can pitch in on developement and >> >> >testing.. >> >> >> > >> >> >> >> > >> Who & where is this work happening? >> >> >> > >> >> >> >> > >> Thanks >> >> >> > >> VInoth >> >> >> > >> >> >> >> > > >> >> >> > > -- >> >> >> > > Jean-Baptiste Onofré >> >> >> > > jbono...@apache.org >> >> >> > > http://blog.nanthrax.net >> >> >> > > Talend - http://www.talend.com >> >> >> > >> >> >> >> >> >>
