Mike, Just FYI, ControllerService's are configurable via the UI since 0.2.0 [1].
Also, I would also +1 for the second of Mark's options. Matt [1] https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#Controller_Services_and_Reporting_Tasks On Thu, Oct 22, 2015 at 6:42 AM, Michael D. Coon <mdco...@yahoo.com.invalid> wrote: > Mark, > I went down the path of your "ControllerService" approach only to find > that the service configuration is tied to an XML file in the nifi > installation versus dynamically configurable via the UI (unless I'm missing > how to configure a controller service on the UI somewhere). It also means I > have a SINGLE instance of my module for the NiFi flow. This is not the case > when I have different modules needing multiple instances inside of my flow. > For these reasons I abandoned the idea of using a ControllerService for > this scenario. > I do think that option 2 would be good on its own even outside of this > particular scenario. And if option 2 were implemented, it would, at a > minimum, help support conditional properties and fix the issues I have > today with properties sticking around after I change the module to use at > runtime in the flow. > Mike > > > > On Thursday, October 22, 2015 9:16 AM, Jennifer Barnabee < > jennifer.barna...@gmail.com> wrote: > > > +1 for the second option. > > On Thu, Oct 22, 2015 at 9:08 AM, Mark Payne <marka...@hotmail.com> wrote: > > > I two thoughts on this concept. Firstly is the notion of Controller > > Services. Rather than having properties that > > change significantly depending on the strategy selected, each strategy > > could be implemented as a separate > > Controller Service. Then, in the Processor you'd just choose which > > Controller Service to use. This allows the > > properties that need to be configured for that strategy to move into the > > Controller Service itself. > > > > The second thought, is one that I had proposed a while back, but it never > > picked up any traction. The idea is > > to have properties "depend" on other properties. So, for instance, you > > could say Property B is only applicable > > if Property A is set to Allowable Value 1 or Allowable Value 2. If > > Property A is not set or is set to Allowable Value > > 3, for instance, then Property B is not even shown in the UI. > > > > The nice thing about the second option is that it would make the UI > > cleaner for existing Processors. For instance, > > we could indicate in CompressContent that the "Compression Level" > property > > should not even be shown if the > > mode is set to Decompress. Currently, we just document that "This > property > > is ignored unless mode is Compress." > > I would much prefer to not even show the property at all. > > > > Any thoughts? > > > > -Mark > > > > > > > > > On Oct 21, 2015, at 10:11 PM, Joe Witt <joe.w...@gmail.com> wrote: > > > > > > Not to trivialize this but curious if we added an 'apply' type option > > > there would that be a sufficient user experience? The core point is > > > once you select a new strategy we need to send that fact to the server > > > to see what effect that has. That happens when you hit ok but as you > > > note that closes the dialogue and requires you to reopen it which is > > > just sort of awkward. So would an 'ok' option which applies the > > > changes and talks to the server and gets new info but doesn't close > > > the dialogue make this good enough? > > > > > > Thanks > > > Joe > > > > > > On Wed, Oct 21, 2015 at 6:09 PM, Michael D. Coon > > > <mdco...@yahoo.com.invalid> wrote: > > >> Mike, > > >> Thanks, that is EXACTLY what I'm doing as well :) My current > > version just allows them to set the properties up once…if they need to > > change the selection, they unfortunately have to create a whole new > > processor configuration and delete the old one. Kinda stinks but, as you > > said, current UI doesn't support this particular use case. At least I'm > not > > alone! :) > > >> Mike > > >> > > >> > > >> > > >> On Wednesday, October 21, 2015 6:02 PM, Michael Moser < > > moser...@gmail.com> wrote: > > >> > > >> > > >> Mike, > > >> > > >> I have implemented a NiFi processor very similar to what you describe. > > A > > >> ServiceLoader discovers the plugins that are available to the > processor > > and > > >> populates the allowableValues of a PropertyDescriptor. > > >> > > >> In the end I don't think there is a way to resolve your > > questions/issues in > > >> the UI. After you select a plugin, the user will have to save the > > >> processor config, then go back into it to see the new properties that > > you > > >> made available. And if you change your plugin selection, then the > > >> processor will stay invalid until the user deletes the old properties > > that > > >> are no longer needed. The NiFi framework just doesn't support this > edge > > >> case. > > >> > > >> In the end, for my processor, I just made all properties from all > > plugins > > >> available all the time. I named the properties to begin with the > > plugin's > > >> name to help users keep track of which plugin they applied to. > > >> > > >> -- Mike > > >> > > >> > > >> > > >> On Wed, Oct 21, 2015 at 5:50 PM, Michael D. Coon > > <mdco...@yahoo.com.invalid> > > >> wrote: > > >> > > >>> Hey Tony, > > >>> I do have working code but not in a public repo. I am, in fact, > > trying > > >>> to use NiFi as a backing engine. I think the bottom line is that I > > want to > > >>> be able to add/remove some configuration properties based on the > > settings > > >>> of other configuration properties. For example, I might specify > > "ClassA" > > >>> for property "PluginClass" and the implementation "ClassA" has its > own > > set > > >>> of parameters required to operate. I want to expose those on the UI. > It > > >>> works as I have it now, but if after setting up "ClassA" as my > plugin I > > >>> change the "PluginClass" setting to "ClassB", all of ClassA's > > properties > > >>> stick around even though they are no longer what I'm returning from > > >>> "getSupportedProperties". It looks like, at least on the version I'm > > >>> using, that NiFi is caching the properties internally and not syncing > > them > > >>> with the properties that I provide through the > "getSupportedProperties" > > >>> method. More specifically, if I actually set a value on any of the > > "ClassA" > > >>> required properties, they stick around forever. I would think that > > whatever > > >>> I return from the "getSupportedProperties" method should, in fact, be > > the > > >>> set of properties exposed on the UI. But they fall out of sync if I > > change > > >>> my settings and add/remove properties. > > >>> Mike > > >>> > > >>> > > >>> > > >>> On Wednesday, October 21, 2015 5:42 PM, Tony Kurc < > > trk...@gmail.com> > > >>> wrote: > > >>> > > >>> > > >>> Mike, > > >>> I'm not quite following what you're trying to accomplish. The use > case > > you > > >>> described was a bit too meta for me to wrap my head around. It sounds > > like > > >>> a dataflow abstraction layer with nifi as a backing engine. It sounds > > like > > >>> you have some code, is it in a public repo so we could take a look at > > what > > >>> you have? > > >>> > > >>> Tony > > >>> On Oct 20, 2015 8:58 AM, "Michael D. Coon" <mdco...@yahoo.com.invalid > > > > >>> wrote: > > >>> > > >>>> All, > > >>>> I'm building a set of NiFi processors that allow data flow managers > > to > > >>>> dynamically configure reusable data processing modules wrapped > inside > > >>> NiFi > > >>>> processors. Imagine that we might want to use NiFi in some cases and > > >>> Storm > > >>>> or Spark in others. The business logic in the processing modules > > would be > > >>>> the same regardless of what framework was feeding the data to the > > >>>> modules...it allows us to reuse capability in different data flow > > >>>> environments. > > >>>> > > >>>> I have this working to some extent using ServiceLoader to discover > > the > > >>>> plugin modules and making those "allowableValues" of a > > >>> PropertyDescriptor. > > >>>> Now it gets tricky. When the user selects one of the modules, the > > module > > >>>> itself specifies its own set of configuration properties that I'd > > like to > > >>>> expose to the data flow manager on the UI. That too works as I can > > grab > > >>> the > > >>>> properties from the module and expose them as NiFi > > PropertyDescriptors. > > >>> So > > >>>> far, so good. But when I change my mind and choose a different > module > > >>> from > > >>>> the allowableValues set I created, the old properties remain set > with > > an > > >>>> error that they are not supported properties. > > >>>> > > >>>> So how do I get rid of the old properties once a new module has > been > > >>>> selected? Also, is there any way that the UI can dynamically update > > the > > >>>> required properties once I make a selection of which module I want > to > > >>> use? > > >>>> As it stands, I have to close the settings window and reopen to get > > the > > >>> new > > >>>> properties list. > > >>>> Thanks for any advice! > > >>>> Mike > > >>>> > > >>>> > > >>> > > >>> > > >>> > > >>> > > >> > > >> > > >> > > > > > > >
