Re: Enumerating processors, queues, etc.

Ok cool. From the the 'gathering of the raw materials of knowledge perspective' - If the things you want internal access to are already exposed via the REST API then we're definitely safe on the threading side in so far as the things I was worried about when considering internal access. If we

Re: Enumerating processors, queues, etc.

Joe W, The use case that originally got me thinking about this was a processor highlighter that looks for processors above or below configured thresholds, possibly by instance or type. I think that requires the ability to 1. enumerate the processor and/or queue collections, 2. query each

Re: Enumerating processors, queues, etc.

Yeah understood. So let's dig into this more. We need to avoid over exposure of internal state which one might want to crawl through because that introduces some multi-threaded challenges and could limit our ability to evolve internals. However, if we understand the questions you'd like to be

Re: Enumerating processors, queues, etc.

Joe Witt - Not really, this kind of went sideways from where I was originally headed. I'm looking for a way for a controller service to iterate over the processor and queue collections (maybe others as well) to look for performance thresholds or other issues and then provide feedback somehow or

Re: Enumerating processors, queues, etc.

Joe Skora - does Jeremy's JIRA cover your use case needs? On Fri, Apr 22, 2016 at 12:44 PM, Jeremy Dyer wrote: > Mark, > > ok that makes sense. I have created a jira for this improvement > https://issues.apache.org/jira/browse/NIFI-1805 > > On Fri, Apr 22, 2016 at 12:27 PM,

Re: Enumerating processors, queues, etc.

Mark, ok that makes sense. I have created a jira for this improvement https://issues.apache.org/jira/browse/NIFI-1805 On Fri, Apr 22, 2016 at 12:27 PM, Mark Payne wrote: > Jeremy, > > It should be relatively easy. In FlowController, we would have to update >

Re: Enumerating processors, queues, etc.

Jeremy, It should be relatively easy. In FlowController, we would have to update getGroupStatus() to set the values on ConnectionStatus and of course update ConnectionStatus to have getters & setters for the new values. That should be about it, I think. -Mark > On Apr 22, 2016, at 12:17 PM,

Re: Enumerating processors, queues, etc.

Mark, What would the process look like for doing that? Would that be something trivial or require some reworking? On Fri, Apr 22, 2016 at 10:26 AM, Mark Payne wrote: > I definitely don't think we should be exposing the FlowController to a > Reporting Task. > However, I

Re: Enumerating processors, queues, etc.

For an extensible system like NiFi, what is the rational behind not allowing the system introspection into the processor and queue collections? On Fri, Apr 22, 2016 at 10:26 AM, Mark Payne wrote: > I definitely don't think we should be exposing the FlowController to a >

Re: Enumerating processors, queues, etc.

I'm fine with the REST API being a better answer or part of the answer. A separate system that can monitor and manage an instance through the REST API might be the right way to go, plus it separates the concerns of monitor and monitored. But to me, a controller service seems like a logical

Re: Enumerating processors, queues, etc.

I definitely don't think we should be exposing the FlowController to a Reporting Task. However, I think exposing information about whether or not backpressure is being applied (or even is configured) is a very reasonable idea. -Mark > On Apr 22, 2016, at 10:22 AM, Jeremy Dyer

Re: Enumerating processors, queues, etc.

I could see the argument for not making that available. What about some sort of reference that would allow the ReportingTask to to determine if backpressure is being applied to a Connection? It currently seems you can see the number of bytes and/or objects count queued in a connection but don't

Re: Enumerating processors, queues, etc.

I think the only way you could do it directly without the REST API is by having access to the FlowController, but that is purposely not exposed to extension points... actually StandardFlowController is what implements the EventAccess interface which ends up providing the path way to the status

Re: Enumerating processors, queues, etc.

Bryan, The ReportingTask enumeration makes sense and was helpful for something else I am working on as well. Like Joe however I'm looking for a way to not just get the *Status objects but rather start and stop processors. Is there a way to do that from the ReportContext scope? I imagine you

Re: Enumerating processors, queues, etc.

Hi Joe, I'm not sure if a controller service can do this, but a ReportingTask has access to similar information. A ReportingTask gets access to a ReportingContext, which can access EventAccess which can access ProcessGroupStatus. >From ProcessGroupStatus you are at the root process group and

Enumerating processors, queues, etc.

Is it possible and if so what is the best way for a controller service to get the collection of all processors or queues? The goal being to iterate over the collection of processors or queues to gather information or make adjustments to the flow.