Matt and Bryan, Thanks for the detailed explanation. The class loading aspect makes perfect sense, and yes, importing as disabled is understood as well.
I do think I've come across a small bug with automatically creating Controller Services on template imports though. NiFi is not automatically creating Controller Services for Processors inside of Process Groups on template imports. Can one of you check me on this before we write up a Jira ticket? Steps to reproduce: 1. Start fresh (clean canvas, no controller services) 2. Create Process Group and go into it 3. Add DetectDuplicate processor to Process Group 4. Configure DetectDuplicate processor with a new DistributedMapCacheClientService and enable that service so that the DetectDuplicate processor is valid and ready to run 5. Navigate to root canvas (see the Process Group) 6. Create template from root canvas (name: "Controller Service Test") 7. Delete everything from the canvas and disable + delete the controller service so that we're essentially back to the start 8. Drag and drop the "Controller Service Test" template onto the canvas (expecting that everything will be auto-created) 9. Navigate into the Process Group and notice that the DetectDuplicate processor is invalid because the DistributedMapCacheClientService ID cannot be found As an additional test, I created a template with a processor at the top-level and one inside the Process Group that are using the same Controller Service, and that works just fine. My guess is that the top-level processor is used to create the Controller Service and the inner-processor picks it up by that same ID. I can provide a sample template in a Jira ticket if that would help. Thanks, Brian On Fri, Aug 14, 2015 at 8:24 AM, Bryan Bende <bbe...@gmail.com> wrote: > Brian, > > Glad it worked! > > I think it was just a combination of things... I had previously spent a lot > of time studying the existing dependency setup in nifi-standard-bundle, as > well as the dev > guide section on NARs [1]. Also, someone else had ran into a similar > problem recently and it turned out to be a dependency problem as well. > > There should be a Maven archetype for Controller Services [2] coming soon > which might help nail down the correct dependency setup, also after seeing > your email last night I put together a bare-bones project [3] that has a > custom processor using a custom controller service (both created using > the archetypes) to serve as an example. > > -Bryan > > [1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#nars > [2] https://issues.apache.org/jira/browse/NIFI-736 > [3] https://github.com/bbende/nifi-dependency-example > > > > On Fri, Aug 14, 2015 at 8:17 AM, Matt Gilman <matt.c.gil...@gmail.com> > wrote: > > > Brian, > > > > The NAR dependency stuff all breaks down to providing classpath isolation > > and not polluting the classpath of downstream NARs with dependencies > > required for Controller Service implementation. We try to keep Controller > > Service interfaces lean by not depending on external libs in the > interface > > (generally a good practice anyways). Both the implementation of the > > Controller Service and the any Processor NAR that want to consume the > > Controller Service needs to depend on the Controller Service API NAR > (where > > the interface lives). This established a common parent ClassLoader which > > allows the service to be access across NARs. If your building Processor > > NARs that depend on custom Controller Services and standard Controller > > Services, you'll want to have your custom Controller Services API NAR > > depend on the standard Controller Services API NAR. The NAR dependency > tree > > (if you will) maps to a ClassLoader hierarchy within NiFi. > > > > Controller Services being added in a disabled state is remaining > consistent > > with Processors (and other components) in template. Also, because > sensitive > > properties are not included in templates, they may require the user to > > complete the configuration prior to enabling. > > > > Matt > > > > On Fri, Aug 14, 2015 at 8:01 AM, Brian Ghigiarelli <briang...@gmail.com> > > wrote: > > > > > Bryan, > > > > > > Thank you!! Adding a NAR dependency to my standard-processors NAR did > the > > > trick to get NiFi to recognize the service for these processors and > > > auto-create a new instance of the Controller Service on template > > import. I > > > owe ya a beer or six the next time our paths cross. > > > > > > How did you figure out the required NAR dependency? Any great debugging > > > tricks to share, or was it your existing depth of NiFi knowledge that > led > > > you to that solution? > > > > > > As a follow-on to that, I'm seeing that Controller Services created via > > > Template instantiation are created in "Disabled" state. Is that > > expected? > > > Is there any way to drag-and-drop a template with controller services > > onto > > > a canvas and have the newly created controller services enabled? > > > > > > Thanks again, > > > Brian > > > > > > On Thu, Aug 13, 2015 at 5:57 PM, Bryan Bende <bbe...@gmail.com> wrote: > > > > > > > Brian, > > > > > > > > I just ran into this exact scenario today while testing a custom > > > controller > > > > service. > > > > > > > > The cause of the problem was that my custom service NAR, did not > have a > > > NAR > > > > dependency on the service API NAR because I had accidentally marked > the > > > API > > > > NAR as provided in the pom file. Once I removed the provided scope > > > > everything worked. > > > > > > > > You can check the MANIFEST file in your custom service NAR... under > > your > > > > NIFI_HOME/work/nar/extensions/{your-nar}/META-INF/MANIFEST.MF it > should > > > > have a "Nar-Dependency-Id" on your API NAR. If it is missing that, > that > > > is > > > > likely the problem. > > > > > > > > -Bryan > > > > > > > > > > > > > > > > On Thu, Aug 13, 2015 at 5:22 PM, Brian Ghigiarelli < > > briang...@gmail.com> > > > > wrote: > > > > > > > > > NiFi Gurus, > > > > > > > > > > Regarding Controller Services, I'm seeing some strange behavior > > > > regarding a > > > > > custom Controller Service only showing its ID in a Processor and > was > > > > > wondering if anyone's encountered similar issues. Seeing this on > the > > > > > latest develop branch, 0.3.0-SNAPSHOT. > > > > > > > > > > Steps taken: > > > > > 1. Create instance of Controller Service in the Controller > Settings > > > > > > Controller Services tab > > > > > 1a. Enable that instance of the Controller Service > > > > > 2. Add new Processor on canvas that will use Controller Service > > > > > 3. Configure Processor > > > > > 3a. Existing Controller Service does not show in property options. > > Only > > > > "No > > > > > value" and "Create new service..." > > > > > 3b. Clicking "Create new service..." opens modal with the custom > > > > Controller > > > > > Service as the only item in the dropdown > > > > > 3c. Clicking "Create" button succeeds, but puts the UUID of the > > > > Controller > > > > > Service in the Value of the Processor Property. > > > > > > > > > > Observed state: > > > > > - Processor is still invalid > > > > > - The ID in the Processor Property matches the ID of the second > > > > (disabled) > > > > > instance of the Controller Service in the Controller Settings > modal. > > > > > > > > > > There are no errors in nifi-app.log - only contains log messages > from > > > the > > > > > StandardControllerServiceProvider that it "Created Controller > Service > > > of > > > > > type [my-implementation-class] with identifier > > > > > [UUID-that-shows-in-the-GUI]" > > > > > > > > > > Adding a DetectDuplicate processor with the > > > > > DistributedMapCacheClientService works just fine, so it'd hard to > > point > > > > > fingers at the framework. Any thoughts on where to begin debugging > > > > and/or > > > > > loggers to turn up would be greatly appreciated! > > > > > > > > > > Thanks, > > > > > Brian > > > > > > > > > > > > > > > > > > > > > -- > > > Brian Ghigiarelli > > > 570-878-9139 > > > > > > -- Brian Ghigiarelli 570-878-9139
