I definitely love the idea, and I think it will provide NiFi a lot more opportunity to accomplish some of its goals.
With this being a sub-project of NiFi, I am guessing that it would have its own code repository, separate from the NiFI git repository, similarly to how nifi-site and nifi-nar-plugin are their own repositories? How would we keep track of JIRA's for MiNiFi? Would it have its own JIRA, or would it perhaps just be a new 'Component' of the NiFi JIRA? Thanks -Mark > On Jan 9, 2016, at 7:29 PM, Joe Witt <[email protected]> wrote: > > NiFi Community, > > I'd like to initiate discussion around a proposal to create our first > sub-project of NiFi. A possible name for it is "MiNiFi" a sort of > play on Mini-NiFi. > > The idea is to provide a complementary data collection agent to NiFi's > current approach of dataflow management. As noted in our ASF TLP > resolution NiFi is to provide "an automated and durable data broker > between systems providing interactive command and control and detailed > chain of custody for data." MiNiFi would be consistent with that > scope with a specific focus on the first-mile challenge so common in > dataflow. > > Specific goals of MiNiFi would be to provide a small, lightweight, > centrally managed agent that natively generates data provenance and > seamlessly integrates with NiFi for follow-on dataflow management and > maintenance of the chain of custody provided by the powerful data > provenance features of NiFi. > > MiNiFi should be designed to operate directly on or adjacent to the > source sensor, system, server generating the events as a resource > sensitive tenant. There are numerous agent models in existence today > but they do not offer the command and control or provenance that is so > important to the philosophy and scope of NiFi. > > These agents would necessarily have a different interactive command > and control model than NiFi as you'd not expect consistent behavior, > capability, or accessibility of all instances of the agents at any > given time. > > Multiple implementations of MiNiFi are envisioned including those that > operate on the JVM and those that do not. > > As the discussion advances we can put together wiki pages, concept > diagrams, and requirements to help better articulate how this might > evolve. We should also discuss the mechanics of how this might work > in terms of infrastructure, code repository, and more. > > Thanks > Joe
