Thanks! Sent from my iPhone
> On Jan 13, 2019, at 18:00, Thomas Weise <t...@apache.org> wrote: > > For the upgrade it is probably best when you change the version number in > [1] and then work through the build issues. > > I would expect a number of dependency changes since the client classpath > now has the Hadoop transitive dependencies shaded away [2]. > > Regarding the development process, see [3]. > > Thomas > > [1] > https://github.com/apache/apex-core/blob/1f5425dc5ac1d3452a3b37db0e4c61485af9c6e1/pom.xml#L109 > [2] https://hadoop.apache.org/docs/r3.0.0/ > [3] http://apex.apache.org/contributing.html > > > >> On Fri, Jan 11, 2019 at 5:14 PM Aaron Bossert <aa...@punchcyber.com> wrote: >> >> at first blush, I would think, that the YARN node tagging and native Docker >> support would be particularly intriguing...the GPU node support would be a >> good implementation trick to be able to put operators that do deep learning >> tasks... >> >> On Fri, Jan 11, 2019 at 2:42 PM Pramod Immaneni <pramod.imman...@gmail.com >>> >> wrote: >> >>> I agree we should just try changing the dependency first. I am in favor >> of >>> upgrading. It will be good to explore if and how we can leverage some of >>> the new features in 3.x. >>> >>>> On Fri, Jan 11, 2019 at 11:38 AM Vlad Rozov <vro...@apache.org> wrote: >>>> >>>> Hi Aaron, >>>> >>>> I’d recommend to start with upgrading Hadoop dependency to 3.x, >>>> recompiling and running unit test. Also, before you put significant >>> effort >>>> into Hadoop 3.x upgrade, let’s discuss and vote on dev@apex. Once >>>> everyone agrees and vote passes, please open JIRA for tracking the >>> changes. >>>> >>>> Thank you, >>>> >>>> Vlad >>>> >>>>> On Jan 10, 2019, at 07:58, Aaron Bossert <aa...@punchcyber.com> >> wrote: >>>>> >>>>> As requested by Thomas, I am hoping we can kick off a discussion >> about >>> a >>>>> short-term goal of adding support for Hadoop 3.X, though, if I am not >>>>> mistaken, there was a pretty large change from 3.0 to 3.1...but will >>>>> confirm, in which case, the ask is to support 3.1+ specifically. >>>>> >>>>> I program in Scala for the most part, but have not problem switching >> to >>>>> pure Java as needed. That being said, I am very unfamiliar with the >>> Apex >>>>> code base and also unsure of the scope and level of effort required >> to >>>> get >>>>> this done. I look to the group for a few pointers on where to begin >> as >>>>> well as code style and such...I don't think I will need a lot of >>>>> hand-holding, just a good shove in the right direction... >>>>> >>>>> Thanks again for the discussion and support on this...for background >>> here >>>>> is my personal and professional motivation for pressing forward: >>>>> >>>>> - Doing research work for clients as well as internal R&D for >>> streaming >>>>> data processing >>>>> - very specific requirement to support the ability to alter >>>>> topologies programatically and manually on the fly, hence the >>>> interest in >>>>> Apex, which is the only platform I know of that has this support >>>> built >>>>> in...I don't want to re-invent that wheel unless absolutely >>>> necessary >>>>> - General interest in supporting the open-source community, both >> as a >>>>> company and individually >>>>> >>>>> -- >>>>> >>>>> M. Aaron Bossert >>>>> (202) 594-1703 >>>>> Punch Cyber Analytics Group >>>> >>>> -- >>> Thanks, >>> Pramod >>> http://ts.la/pramod3443 >>> >> >> >> -- >> >> M. Aaron Bossert >> (202) 594-1703 >> Punch Cyber Analytics Group >>
