Yes, I should have clarified this. Thanks Bryan. This is the solution for the generic use case. The original question was about reducing the controller to only a single instance of a specific controller service implementation, which is how the tangent got started.
Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 23, 2020, at 3:27 PM, Bryan Bende <bbe...@gmail.com> wrote: > > If you use registry >= 0.5.0 And nifi >= 1.10.0, then it will auto select > external controller services with the same name as long as there is only one > of the same type with same name (name is not unique). > > On Sat, May 23, 2020 at 3:34 PM Andy LoPresto <alopre...@apache.org > <mailto:alopre...@apache.org>> wrote: > My position is that we don’t claim completely automated deployment as a > feature, so manually setting the controller service IDs is not exposed. > Technically, they are defined in the flow.xml.gz and could be modified by an > administrator to be static after generation. This would require frequent > manual manipulation of the flow.xml.gz in various environments and frequent > restarts of the NiFi service. I do not recommend this. > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > He/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On May 23, 2020, at 11:05 AM, Andrew Grande <apere...@gmail.com >> <mailto:apere...@gmail.com>> wrote: >> >> Aren't those IDs generated? How can one enforce it? >> >> Andrew >> >> On Sat, May 23, 2020, 10:53 AM Andy LoPresto <alopre...@apache.org >> <mailto:alopre...@apache.org>> wrote: >> If you want the process to be completely automated, you would have to >> enforce the controller service IDs to be identical across environments. >> Otherwise deployment would need a manual intervention to reference the >> specific controller service in the proper component. >> >> Andy LoPresto >> alopre...@apache.org <mailto:alopre...@apache.org> >> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> >> He/Him >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >>> On May 22, 2020, at 3:57 PM, Eric Secules <esecu...@gmail.com >>> <mailto:esecu...@gmail.com>> wrote: >>> >>> Hi Andy, >>> >>> Given that you have a flow which operates on two different S3 accounts for >>> example, how would you do deployment automation? Do you mandate that the >>> controller service with the same ID must exist in both a development and >>> production environment rather than try to connect a processor to a matching >>> controller service? >>> >>> -Eric >>> >>> On Fri, May 22, 2020 at 3:44 PM Andy LoPresto <alopre...@apache.org >>> <mailto:alopre...@apache.org>> wrote: >>> Eric, >>> >>> I can’t answer all these questions but I would definitely have hesitations >>> around building an expectation that there is only one instance of any given >>> controller service type in an entire canvas. I can think of numerous flows >>> (this may not affect your particular flows, but the concepts still apply) >>> which require multiple instances of the same controller service type to be >>> available: >>> >>> * A flow which invokes a mutually-authenticated TLS HTTP API, consumes >>> data, transforms it, and posts it to another mTLS API >>> * A flow which retrieves objects from one S3 bucket and puts them into an >>> S3 bucket in a different AWS account >>> * A flow which connects to one database and retrieves data, transforms it, >>> and persists it to another database >>> >>> If there is only _one_ StandardSSLContextService, >>> AWSCredentialsProviderControllerService, or DBCPConnectionPool available in >>> the entire controller, these flows cannot exist. >>> >>> I am not saying the retrieval of new flow versions and the matching of >>> referenced controller services cannot be improved, but I would definitely >>> advise caution before going too far down this path without considering all >>> possible side effects and potential constraints on future flow development. >>> >>> >>> >>> Andy LoPresto >>> alopre...@apache.org <mailto:alopre...@apache.org> >>> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> >>> He/Him >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>>> On May 22, 2020, at 3:01 PM, Eric Secules <esecu...@gmail.com >>>> <mailto:esecu...@gmail.com>> wrote: >>>> >>>> Hello everyone, >>>> >>>> I am running into an issue with automated deployment using nipyapi >>>> <https://nipyapi.readthedocs.io/en/latest/>. We would like to be able to >>>> pull down flows from a registry and have them ready to go once all their >>>> controller services have been turned on. But there are a few issues. >>>> Sometimes the flows that we download from the registry reference >>>> controller service IDs that don't exist on this machine because the flow >>>> was developed in a different environment. That's easy enough to fix if >>>> there is just one applicable controller service, but not when there are >>>> two or more. >>>> >>>> We have taken the decision to put all our controller services at the top >>>> level and have one of each kind we need, rather than have multiple of the >>>> same controller service attached to individual process groups. >>>> >>>> We are running into a problem where some processors can either connect to >>>> a JSONTreeReader or a CSVReader and there's no indication in the >>>> ProcessorDTO object which type it was originally connected to, just a GUID >>>> of a controller service that doesn't exist in this deployment. >>>> >>>> Would it be possible to include the type or name of the controller service >>>> in the component.config.descriptors section? Are we going about it the >>>> wrong way trying to simplify down to the least number of controller >>>> services? >>>> >>>> Thanks, >>>> Eric >>> >> > > -- > Sent from Gmail Mobile
