at first blush, I would think, that the YARN node tagging and native Docker support would be particularly intriguing...the GPU node support would be a good implementation trick to be able to put operators that do deep learning tasks...
On Fri, Jan 11, 2019 at 2:42 PM Pramod Immaneni <pramod.imman...@gmail.com> wrote: > I agree we should just try changing the dependency first. I am in favor of > upgrading. It will be good to explore if and how we can leverage some of > the new features in 3.x. > > On Fri, Jan 11, 2019 at 11:38 AM Vlad Rozov <vro...@apache.org> wrote: > > > Hi Aaron, > > > > I’d recommend to start with upgrading Hadoop dependency to 3.x, > > recompiling and running unit test. Also, before you put significant > effort > > into Hadoop 3.x upgrade, let’s discuss and vote on dev@apex. Once > > everyone agrees and vote passes, please open JIRA for tracking the > changes. > > > > Thank you, > > > > Vlad > > > > > On Jan 10, 2019, at 07:58, Aaron Bossert <aa...@punchcyber.com> wrote: > > > > > > As requested by Thomas, I am hoping we can kick off a discussion about > a > > > short-term goal of adding support for Hadoop 3.X, though, if I am not > > > mistaken, there was a pretty large change from 3.0 to 3.1...but will > > > confirm, in which case, the ask is to support 3.1+ specifically. > > > > > > I program in Scala for the most part, but have not problem switching to > > > pure Java as needed. That being said, I am very unfamiliar with the > Apex > > > code base and also unsure of the scope and level of effort required to > > get > > > this done. I look to the group for a few pointers on where to begin as > > > well as code style and such...I don't think I will need a lot of > > > hand-holding, just a good shove in the right direction... > > > > > > Thanks again for the discussion and support on this...for background > here > > > is my personal and professional motivation for pressing forward: > > > > > > - Doing research work for clients as well as internal R&D for > streaming > > > data processing > > > - very specific requirement to support the ability to alter > > > topologies programatically and manually on the fly, hence the > > interest in > > > Apex, which is the only platform I know of that has this support > > built > > > in...I don't want to re-invent that wheel unless absolutely > > necessary > > > - General interest in supporting the open-source community, both as a > > > company and individually > > > > > > -- > > > > > > M. Aaron Bossert > > > (202) 594-1703 > > > Punch Cyber Analytics Group > > > > -- > Thanks, > Pramod > http://ts.la/pramod3443 > -- M. Aaron Bossert (202) 594-1703 Punch Cyber Analytics Group
