Yeah understood. So let's dig into this more. We need to avoid over exposure of internal state which one might want to crawl through because that introduces some multi-threaded challenges and could limit our ability to evolve internals. However, if we understand the questions you'd like to be able to ask of certain things better perhaps we can better expose those results.
Can you try stating what you're looking for in a bit more specific examples. For instance you said "want to iterate over the processor collections...to look for performance thresholds" - What sorts of performance threshold questions? On Fri, Apr 22, 2016 at 2:20 PM, Joe Skora <jsk...@gmail.com> wrote: > Joe Witt - Not really, this kind of went sideways from where I was > originally headed. > > I'm looking for a way for a controller service to iterate over the > processor and queue collections (maybe others as well) to look for > performance thresholds or other issues and then provide feedback somehow or > report externally. > > If it can be done through the REST API, seems like it should be possible > from within the framework as well. > > On Fri, Apr 22, 2016 at 1:32 PM, Joe Witt <joe.w...@gmail.com> wrote: > >> Joe Skora - does Jeremy's JIRA cover your use case needs? >> >> On Fri, Apr 22, 2016 at 12:44 PM, Jeremy Dyer <jdy...@gmail.com> wrote: >> > Mark, >> > >> > ok that makes sense. I have created a jira for this improvement >> > https://issues.apache.org/jira/browse/NIFI-1805 >> > >> > On Fri, Apr 22, 2016 at 12:27 PM, Mark Payne <marka...@hotmail.com> >> wrote: >> > >> >> Jeremy, >> >> >> >> It should be relatively easy. In FlowController, we would have to update >> >> getGroupStatus() to set the values on ConnectionStatus >> >> and of course update ConnectionStatus to have getters & setters for the >> >> new values. That should be about it, I think. >> >> >> >> -Mark >> >> >> >> >> >> > On Apr 22, 2016, at 12:17 PM, Jeremy Dyer <jdy...@gmail.com> wrote: >> >> > >> >> > Mark, >> >> > >> >> > What would the process look like for doing that? Would that be >> something >> >> > trivial or require some reworking? >> >> > >> >> > On Fri, Apr 22, 2016 at 10:26 AM, Mark Payne <marka...@hotmail.com> >> >> wrote: >> >> > >> >> >> I definitely don't think we should be exposing the FlowController to >> a >> >> >> Reporting Task. >> >> >> However, I think exposing information about whether or not >> backpressure >> >> is >> >> >> being applied >> >> >> (or even is configured) is a very reasonable idea. >> >> >> >> >> >> -Mark >> >> >> >> >> >> >> >> >>> On Apr 22, 2016, at 10:22 AM, Jeremy Dyer <jdy...@gmail.com> wrote: >> >> >>> >> >> >>> I could see the argument for not making that available. What about >> some >> >> >>> sort of reference that would allow the ReportingTask to to >> determine if >> >> >>> backpressure is being applied to a Connection? It currently seems >> you >> >> can >> >> >>> see the number of bytes and/or objects count queued in a connection >> but >> >> >>> don't have any reference to the values a user has setup for >> >> backpressure >> >> >> in >> >> >>> the UI. Is there a way to get those values in the scope of the >> >> >>> ReportingTask? >> >> >>> >> >> >>> On Fri, Apr 22, 2016 at 10:03 AM, Bryan Bende <bbe...@gmail.com> >> >> wrote: >> >> >>> >> >> >>>> I think the only way you could do it directly without the REST API >> is >> >> by >> >> >>>> having access to the FlowController, >> >> >>>> but that is purposely not exposed to extension points... actually >> >> >>>> StandardFlowController is what implements the >> >> >>>> EventAccess interface which ends up providing the path way to the >> >> status >> >> >>>> objects. >> >> >>>> >> >> >>>> I would have to defer to Joe, Mark, and others about whether we >> would >> >> >> want >> >> >>>> to expose direct access to components >> >> >>>> through controller services, or some other extension point. >> >> >>>> >> >> >>>> On Fri, Apr 22, 2016 at 9:46 AM, Jeremy Dyer <jdy...@gmail.com> >> >> wrote: >> >> >>>> >> >> >>>>> Bryan, >> >> >>>>> >> >> >>>>> The ReportingTask enumeration makes sense and was helpful for >> >> something >> >> >>>>> else I am working on as well. >> >> >>>>> >> >> >>>>> Like Joe however I'm looking for a way to not just get the *Status >> >> >>>> objects >> >> >>>>> but rather start and stop processors. Is there a way to do that >> from >> >> >> the >> >> >>>>> ReportContext scope? I imagine you could pull the Processor "Id" >> from >> >> >> the >> >> >>>>> ProcessorStatus and then use the REST API but was looking for >> >> something >> >> >>>>> more direct than having to use the REST API >> >> >>>>> >> >> >>>>> >> >> >>>>> On Fri, Apr 22, 2016 at 9:23 AM, Bryan Bende <bbe...@gmail.com> >> >> wrote: >> >> >>>>> >> >> >>>>>> Hi Joe, >> >> >>>>>> >> >> >>>>>> I'm not sure if a controller service can do this, but a >> >> ReportingTask >> >> >>>> has >> >> >>>>>> access to similar information. >> >> >>>>>> >> >> >>>>>> A ReportingTask gets access to a ReportingContext, which can >> access >> >> >>>>>> EventAccess which can access ProcessGroupStatus. >> >> >>>>>> >> >> >>>>>> From ProcessGroupStatus you are at the root process group and can >> >> >>>>> enumerate >> >> >>>>>> the flow: >> >> >>>>>> >> >> >>>>>> private Collection<ConnectionStatus> connectionStatus = new >> >> >>>>> ArrayList<>(); >> >> >>>>>> private Collection<ProcessorStatus> processorStatus = new >> >> >>>> ArrayList<>(); >> >> >>>>>> private Collection<ProcessGroupStatus> processGroupStatus = new >> >> >>>>>> ArrayList<>(); >> >> >>>>>> private Collection<RemoteProcessGroupStatus> >> >> remoteProcessGroupStatus >> >> >> = >> >> >>>>> new >> >> >>>>>> ArrayList<>(); >> >> >>>>>> private Collection<PortStatus> inputPortStatus = new >> ArrayList<>(); >> >> >>>>>> private Collection<PortStatus> outputPortStatus = new >> ArrayList<>(); >> >> >>>>>> >> >> >>>>>> Not sure if that is what you were looking for. >> >> >>>>>> >> >> >>>>>> -Bryan >> >> >>>>>> >> >> >>>>>> >> >> >>>>>> On Fri, Apr 22, 2016 at 8:25 AM, Joe Skora <jsk...@gmail.com> >> >> wrote: >> >> >>>>>> >> >> >>>>>>> Is it possible and if so what is the best way for a controller >> >> >>>> service >> >> >>>>> to >> >> >>>>>>> get the collection of all processors or queues? >> >> >>>>>>> >> >> >>>>>>> The goal being to iterate over the collection of processors or >> >> queues >> >> >>>>> to >> >> >>>>>>> gather information or make adjustments to the flow. >> >> >>>>>>> >> >> >>>>>> >> >> >>>>> >> >> >>>> >> >> >> >> >> >> >> >> >> >> >>
