Re: [EXT] Re: OnPrimaryNodeStateChange vs Primary Only configuration

2019-08-16 Thread Bryan Bende
Ah sorry I confused "which node the processor is on" with "where it is scheduled to run". Sounds like you would need the fix Mark suggested. I was just looking at the code of HandleHttpRequest and I think it avoids the problem I was talking about... I was thinking of the case where the

Re: [EXT] Re: OnPrimaryNodeStateChange vs Primary Only configuration

2019-08-16 Thread Mark Payne
It seems reasonable to me to add a `ExecutionNode getExecutionNode()` method to ProcessContext. This enum already exists in nifi-api, but I don't believe that it's exposed anywhere to the Processor itself. > On Aug 16, 2019, at 11:32 AM, Peter Wicks (pwicks) wrote: > > Bryan, > > I'm

RE: [EXT] Re: OnPrimaryNodeStateChange vs Primary Only configuration

2019-08-16 Thread Peter Wicks (pwicks)
Bryan, I'm familiar with the getNodeTypeProvider method. Unfortunately, this does not differentiate between processors that are scheduled to run only on the Primary node and those that are scheduled to run on all of them. So you're saying, a better fix would be to properly call

increased metrics window size from 5 minutes

2019-08-16 Thread Aaron Rich
Hi all, I'm trying to report metrics from a NiFi instance to an external system. I'm trying to get total messages processed, queue sizes, errors, etc. I have a reporting task that is close to doing what I need but I only get the metrics for the last 5 minutes. Some of the data only processes

Re: OnPrimaryNodeStateChange vs Primary Only configuration

2019-08-16 Thread Bryan Bende
AbstractSessionFactoryProcessor has a method getNodeTypeProvider().isPrimary() The ultimate fix for your problem is that a processor shouldn't have it's onScheduled called at all unless it is actually schedule to run on that node. Currently it calls onScheduled on all nodes, but then never calls

OnPrimaryNodeStateChange vs Primary Only configuration

2019-08-16 Thread Peter Wicks (pwicks)
I'm working on a bug fix for HandleHttpRequest and need to check if a processor is configured to run only on primary node (and not if a processor has the attribute that ONLY allows it to run on primary node). Here is the scenario for background: * NiFi cluster, but all nodes are on the same

Using ValidateRecord to validate schemas

2019-08-16 Thread Phillip Grenier
Would it be reasonable to add the details of the failures to the flow file attributes? I know they exist on the provenance event, but that can not be persisted easily to file for later analysis and correction. This also seems harder to locate for non-developers and flow designers. Thanks,