Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-11 Thread Sumanth Chinthagunta
Good idea. There will be many possibilities if we can make MiNIFi run on android / iOS or other embedded devices. Wonder how back-pressure works in this kind of distributed setup. I was reading about reactivesocket project, This project is trying to solve reactive / back-pressure problem over

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-10 Thread Joe Witt
Andre Wow! Lots of good points/considerations in there. Let me try to respond to a few and these are good things we should be sure to have included in a more formal proposal document. Centralized Management: Yeah this would not limit or compete with the use of automated deployment tools

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-10 Thread Joshua Davis
Some thoughts as well. - This will really differentiate NIFI from other open source projects that have a central model similar to Nifi but are too large to place on small devices. - This will enable anyone creating a project with devices whether they are refrigerators, security systems, power

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-10 Thread Joe Witt
Joshua, Definitely agree with the bullet points you listed. As for the differentiation it is not, in my view, so much a differentiation concern but a 'there should be a better dataflow management and provenance story' concern. My personal view is with MiNiFi, just as in NiFi, we should not

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-10 Thread Mark Payne
I definitely love the idea, and I think it will provide NiFi a lot more opportunity to accomplish some of its goals. With this being a sub-project of NiFi, I am guessing that it would have its own code repository, separate from the NiFI git repository, similarly to how nifi-site and

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-10 Thread Joe Witt
Mark I am not quite sure what the best practices are around those questions yet. Have been doing some looking around at how other projects do it. Spark/Hadoop/Camel/etc... there are lots of examples to check into. Thanks Joe On Sun, Jan 10, 2016 at 10:10 AM, Mark Payne

[DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-09 Thread Joe Witt
NiFi Community, I'd like to initiate discussion around a proposal to create our first sub-project of NiFi. A possible name for it is "MiNiFi" a sort of play on Mini-NiFi. The idea is to provide a complementary data collection agent to NiFi's current approach of dataflow management. As noted in

Re: [DISCUSS] Proposal for an Apache NiFi sub project - MiNiFi

2016-01-09 Thread Tony Kurc
Some thoughts - so on the roadmap Joe posted a couple days ago, HA was a big focus (in 1.X). NiFi becomes a bigger better "hub", but likely those capabilities don't scale down quite as well. Thanks for kicking off this discussion, Joe, I think now is the time think about what we need to do about