Hello These questions were asked in forum on flow based programming so I thought it best to redirect it here to the apache nifi (incubating) mailing list.
Question: How does it fare with GC Gitter and use of off heap memory to prevent GC Gitter? Response: Considerable effort has gone into and will continue to go into ensuring we are as complimentary to the Java GC processes as possible. We've done a great deal of design consideration to make this work well and have considerable operational experience to help us understand if we're on the right track in this regard. That said, it is a framework and folks can build their own processes. As such for processors built which do not consider/honor Java memory pressures it can create challenges. We do also some interesting things. While we generally try to keep in-memory the pointers and the metadata/attributes about the objects we're proccessing (not their content) we do at times swap them out to some underlying off-heap storage. This greatly reduces Java memory pressure even in conditions of massive backlogs which can at times occur. Question: Will you have a web editor which can be used in web based projects? Response: NiFi itself runs as a full application - it is not meant to be embedded into another application. NiFi is controlled and observed through a web-client we provide but any client that can talk to our HTTP-based API will be sufficient. Question: Will you able to use NoFlo UI with this as an alternative editor? Will you support this in the future? Response: I simply do not know enough about this to give you a good answer but if it can talk to our HTTP API then possibly. Question: Will the editor be a projectional editor / structure editor? Response: Those were not terms I was familiar with. But some googling has me thinking it is more of a projectional editor. I am prepared to learn I am wrong here though ;-) Question: Is it possible to extend this to more general Dataflow programming using both Visual and text based language? Like: Microsoft Dryad, Netflix Mantis, Apache Flink, Verilog, VHDL, CAL Actor Language, etc. Response: As of yet it is unclear if there is a good fit. One important difference between NiFi and many of the other systems I am aware of is the notion of runtime/live modification versus design and deploy. For many other systems there is a language in which you design a flow and compile/deploy. With NiFi it is more like molding clay. As you press in, the clay changes shape. This difference is important as it lends itself to a vastly different set of possible user experiences. Question: Will you support data connection and feed handlers for important domains out of the box? E.g FIX Protocol handlers for trading, OpenMAMA support, etc. Response: Support for connectors will presumably occur at the will of the community and as licensing permits. We would love to see the range of connectors provided grow based on contributions and feedback. Question: Any plans to use a language work bench ( https://www.jetbrains.com/mps/, http://mbeddr.com/, https://github.com/JetBrains/Nitra) so this can be ported outside the JVM while maintaining the same code base? Response: There are no plans for support outside the JVM at this time. Question: Will this support data flow spreadsheet paradigm? E.g. http://www.ankhor.com/en/ Response: I am not sure it applies but as we do a better job of communicating, describing, and making NiFi available it will be interesting to learn more. Question: Will this support real time / streaming DataFrames? E.g. http://ddf.io/, borrowing concepts from Pandas, R DataFrame, https://www.quantrix.com/en/ Response: It is unclear if DataFrames specifically makes sense for us to support. It appears they have this already well supported on ideal platforms like Apache Spark and others. We do not replace or compete with complex event processing engines - we compliment them through orchestration/data flow management into and out of these environments and the wide range of simple event processing requirements which are best done in more simplified models. Thanks Joe
