If you want the process to be completely automated, you would have to enforce the controller service IDs to be identical across environments. Otherwise deployment would need a manual intervention to reference the specific controller service in the proper component.
Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com He/Him PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 22, 2020, at 3:57 PM, Eric Secules <esecu...@gmail.com> wrote: > > Hi Andy, > > Given that you have a flow which operates on two different S3 accounts for > example, how would you do deployment automation? Do you mandate that the > controller service with the same ID must exist in both a development and > production environment rather than try to connect a processor to a matching > controller service? > > -Eric > > On Fri, May 22, 2020 at 3:44 PM Andy LoPresto <alopre...@apache.org > <mailto:alopre...@apache.org>> wrote: > Eric, > > I can’t answer all these questions but I would definitely have hesitations > around building an expectation that there is only one instance of any given > controller service type in an entire canvas. I can think of numerous flows > (this may not affect your particular flows, but the concepts still apply) > which require multiple instances of the same controller service type to be > available: > > * A flow which invokes a mutually-authenticated TLS HTTP API, consumes data, > transforms it, and posts it to another mTLS API > * A flow which retrieves objects from one S3 bucket and puts them into an S3 > bucket in a different AWS account > * A flow which connects to one database and retrieves data, transforms it, > and persists it to another database > > If there is only _one_ StandardSSLContextService, > AWSCredentialsProviderControllerService, or DBCPConnectionPool available in > the entire controller, these flows cannot exist. > > I am not saying the retrieval of new flow versions and the matching of > referenced controller services cannot be improved, but I would definitely > advise caution before going too far down this path without considering all > possible side effects and potential constraints on future flow development. > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > He/Him > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On May 22, 2020, at 3:01 PM, Eric Secules <esecu...@gmail.com >> <mailto:esecu...@gmail.com>> wrote: >> >> Hello everyone, >> >> I am running into an issue with automated deployment using nipyapi >> <https://nipyapi.readthedocs.io/en/latest/>. We would like to be able to >> pull down flows from a registry and have them ready to go once all their >> controller services have been turned on. But there are a few issues. >> Sometimes the flows that we download from the registry reference controller >> service IDs that don't exist on this machine because the flow was developed >> in a different environment. That's easy enough to fix if there is just one >> applicable controller service, but not when there are two or more. >> >> We have taken the decision to put all our controller services at the top >> level and have one of each kind we need, rather than have multiple of the >> same controller service attached to individual process groups. >> >> We are running into a problem where some processors can either connect to a >> JSONTreeReader or a CSVReader and there's no indication in the ProcessorDTO >> object which type it was originally connected to, just a GUID of a >> controller service that doesn't exist in this deployment. >> >> Would it be possible to include the type or name of the controller service >> in the component.config.descriptors section? Are we going about it the wrong >> way trying to simplify down to the least number of controller services? >> >> Thanks, >> Eric >
