Re: What next with NiFi

2016-09-06 Thread Tijo Thomas
Hi Joe, I am not sure whether I sounds stupid. I was also researching on the option 2. I felt nifi lack the capability to process the big data jobs where data need to be processed with locality. Initially I was thinking about making the modification to run nifi on yarn. But this approach

Re: What next with NiFi

2016-09-06 Thread Bryan Bende
t; > Ram > > > > *From:* Bryan Bende [mailto:bbe...@gmail.com] > *Sent:* Tuesday, September 06, 2016 10:20 AM > > *To:* users@nifi.apache.org > *Subject:* Re: What next with NiFi > > > > Ram, > > > > Are you talking about isolation from a security perspective?

RE: What next with NiFi

2016-09-06 Thread Nathamuni, Ramanujam
ent: Tuesday, September 06, 2016 9:12 AM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: What next with NiFi Gunjan The best indicator of areas of focus I think are listed here https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals In a roadmap discuss

Re: What next with NiFi

2016-09-06 Thread Bryan Bende
ptember 06, 2016 9:12 AM > To: users@nifi.apache.org > Subject: Re: What next with NiFi > > Gunjan > > The best indicator of areas of focus I think are listed here > https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals > > In a roadmap discussion thread

RE: What next with NiFi

2016-09-06 Thread Nathamuni, Ramanujam
AM To: users@nifi.apache.org Subject: Re: What next with NiFi Gunjan The best indicator of areas of focus I think are listed here https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals In a roadmap discussion thread back in January of this year the items mentioned specifically

Re: What next with NiFi

2016-09-06 Thread Bryan Bende
Gunjan/Joe, I was looking into this at one point... I think the idea would be to implement a Beam source/sink (whatever their terminology is) for NiFi, and this would allow someone to write a single Beam job pulling/pushing data to/from NiFi, and then that single job could be executed in Flink, S

Re: What next with NiFi

2016-09-06 Thread Joe Witt
Gunjan No plans at this point. What sort of beam related integration are you envisioning? There are a couple scenarios that come to mind: 1) NiFi as a beam runner. Probably not a great fit as by design we'd not be attempting to address and important cross section of the types of processing you

Re: What next with NiFi

2016-09-06 Thread Gunjan Dave
Thank Joe, w.r.t to NiFi externals, like we currently have NiFi spark and storm externals, any plans for NiFi beam external? Reason is I saw your name as one of the contributors there in incubation proposal. I was studying beam in more depth hence the question. On Tue, Sep 6, 2016, 6:42 PM Joe Wit

Re: What next with NiFi

2016-09-06 Thread Joe Witt
Gunjan The best indicator of areas of focus I think are listed here https://cwiki.apache.org/confluence/display/NIFI/NiFi+Feature+Proposals In a roadmap discussion thread back in January of this year the items mentioned specifically as trailing the 1.0 release were extension and variable registri