Aldrin, I got what you mean, and it could actually work in my scenario but I need some time to look into it to evaluate pros and cons as I feel that, even if it could help reduce maintenance cost by centralising common functionalities, it also adding complexity to the overall design and debug of flows because of all the connections that would have to be created (as you said, organisation is key here).
Thanks for your support, Yari On 27 December 2016 at 17:02, Aldrin Piri <aldrinp...@gmail.com> wrote: > Hi Yari, > > I think your assumptions may be a little off. You can use input/output > ports to help traverse the nested levels of PGs. To help illustrate, I > created a template [1] you can evaluate as to one way this could be managed > for your scenario. As noted, the number of connections can become > unwieldy, so organization is key with large numbers of groups making use of > a central PG of functionality! :) > > Please let us know if that clarifies or if there are any additional > questions. > > > [1] https://gist.github.com/apiri/69b40d46d29a15641962dae649093298 > > Thanks! > --aldrin > > > On Tue, Dec 27, 2016 at 10:40 AM, Yari Marchetti < > yari.marche...@buongiorno.com> wrote: > >> Hi Aldrin, >> the problem I see with your suggestion, at least in my specific case, >> it's that it requires to have all the referencing groups (in your example >> A,B) inside the same process group (otherwise you cannot link them, right?) >> but that's not the case for us because we've many of them so they're >> organised in a hierarchy of PG for better manageability and control (Please >> let me know if my assumptions are correct, because I may be totally off :) >> That's why I was looking for something that could be referenced from >> anywhere (like a RPG). >> >> Yari >> >> On 27 December 2016 at 16:14, Aldrin Piri <aldrinp...@gmail.com> wrote: >> >>> Hi Yari, >>> >>> The concept you are looking for has been discussed a bit and a draft >>> feature proposal was created for referenceable process groups [1], although >>> is a common pattern and implementable with the current functionality >>> provided; common cases of this are standard tagging or enrichment sets of >>> functionality. >>> >>> What you want to attempt seems like you wouldn't necessarily need to do >>> an RPG (unless you were looking to cut down on connection clutter), but >>> could perform routing to a single process group via attributes and tagging >>> and then using a fan out to send data back to its respective group. >>> >>> Flow of data would be something such as: >>> >>> * Source Group {A, B, ...} >>> * Update Attribute >>> - Tag Data as Source Group {A, B, ...} (source) >>> * Singleton Process group input port for common processing >>> * Route on Attribute (source) >>> - Create relationships for each of the groups that are transmitting >>> data to this common block >>> * Source Group >>> >>> >>> [1] https://cwiki.apache.org/confluence/display/NIFI/Referen >>> ce-able+Process+Groups >>> >>> On Tue, Dec 27, 2016 at 9:49 AM, Yari Marchetti < >>> yari.marche...@buongiorno.com> wrote: >>> >>>> Hello, >>>> we're actively using Nifi on production but there's a recurring problem >>>> popping out every now and then: maintenance of process groups created from >>>> templates, as we're spending quite some time looking for all the instances >>>> of a specific template and updating them as required. >>>> >>>> What we would like to do it's something like a singleton instance of a >>>> process group that could just be referenced by all other the flows in Nifi >>>> without having different instances of a template. Is there any way to >>>> achieve this? The only thing that came to my mind is using a RPG to the >>>> local cluster (which it's made up of 3 servers), don't know though if it >>>> would work... >>>> >>>> Thanks, >>>> Yari >>>> >>> >>> >> >
