Come on with a subject like that people must want to read on... The model of configuration hierarchy nifi supports with Process Groups/ports today enforces a very explicit connection from component or group to another component or group. This is great in that it is very intentional and makes it easy for operator to understand precisely how the data flows. However, this means that a given process group cannot be easily reused - you have to make copies of it.
What we propose then is to introduce a concept of a 'Function Group'. It would be similar in structure to a Process Group yet it would have a very important difference. These 'Function Groups' would be reference-able from anywhere in the flow. They would have zero or one input ports and they would have zero or more output ports. Once a function group is defined then references to it can be placed anywhere onto the flow it is needed just like dragging a new processor onto the flow works. The framework would automatically retain the context of the where the objects actually came from in the flow and thus where they need to come back out. Similar in concept, but for different use cases then is the idea of a 'wormhole' type input port and output port. These would be virtual ports whereby you could refer to that port anywhere on the graph that you need to and the actual port would be somewhere specific. In this sense you would not be required to have explicit connections at all times. I am guessing this is a little tough to follow with both my wording and the lack of visuals. Will turn this into a wiki page with some screenshots/concept drawings to help communicate this better. However, wanted to put this out to start finding if there are any strong opinions on this concept. Our current model is explicit but can be too rigid and cause people to create flows that have way more connections and duplication than are truly necessary or intended. By offering these implicit and explicit models with some UX enhancements I believe we can retain the ability for an operator to understand and follow how the flow works but reduce the complexity that can grow in a completely explicit approach as we have today. Thanks Joe
