NiFi Community, Originally discussed in January [1], the MiNiFi agent model was met with positive feedback. I would like to propose a concerted effort toward the execution on the ideas presented and establish a basis for incorporation of the feedback received from, and collaboration with, the community to move toward our goals of helping with dataflow from the point of its origin.
To that end, I would like to propose the creation of: - a separate repository (nifi-minifi), - establishment of a MiNiFi JIRA (MINIFI), and - production of an associated feature proposals and design documentation within our Confluence Wiki spaces beyond the initial points outlined by Joe with some additional proposals architecture and roadmap The separate JIRA and Git repo map to the existing ASF infrastructure for projects with similar efforts and will aid in a cleaner release and issue management process. Central to the aims of MiNiFi, the tenets of its operation and execution of dataflow are the same as NiFi itself: security, provenance, and management of dataflow; helping bring information to NiFi while maintaining the full extent of its provenance. Some clarifying points based on the discussion that existed previously: - While there may be reuse of NiFi components and some overlap, MiNiFi is a separate effort that is complementary to but not necessarily directly compatible with existing components and extensions. Obviously there has been a lot of great effort which we can reuse, but in striving to be a smaller footprint, we should not find ourselves beholden to the existing, core, NiFi architecture. - There will exist scenarios where there is an inherent need to go smaller and closer to the source system. This will take the form of native code that builds upon the same efforts and items originally developed under the Java ecosystem. - Design should take consideration of disparate execution environments and provide ways to robustly handle varying means of communication and exchange. Accordingly, communications both in management of agents and the transference of data should be neutral to technologies, providing the same flexibility and adaptation that allows NiFi to communicate with a wide breadth of systems, protocols, schemas, and formats. [1] http://apache-nifi-developer-list.39713.n7.nabble.com/DISCUSS-Proposal-for-an-Apache-NiFi-sub-project-MiNiFi-td6141.html
