Re: flowfiles stuck in load balanced queue; nifi 1.8

2018-12-26 Thread Mark Payne
Ok great, thanks for the info! This at least tells me where to be investigating. Thanks! -Mark Sent from my iPhone On Dec 26, 2018, at 10:31 PM, dan young mailto:danoyo...@gmail.com>> wrote: Hello Mark, I just stopped the destination processor, and then disconnected the node in question (ni

Re: flowfiles stuck in load balanced queue; nifi 1.8

2018-12-26 Thread dan young
Hello Mark, I just stopped the destination processor, and then disconnected the node in question (nifi1-1). Once I disconnected the node, the flow file in the load balance connection disappeared from the queue. After that, I reconnected the node (with the downstream processor disconnected) and on

Re: flowfiles stuck in load balanced queue; nifi 1.8

2018-12-26 Thread Mark Payne
Ok, I just wanted to confirm that when you said “once it rejoins the cluster that flow file is gone” that you mean “the flowfile did not exist on the system” and NOT “the queue size was 0 by the time that I looked at the UI.” I.e., is it possible that the FlowFile did exist, was restored, and th

Re: flowfiles stuck in load balanced queue; nifi 1.8

2018-12-26 Thread dan young
Heya Mark, If we restart the node, that "stuck" flowfile will disappear. This is the only way so far to clear out the flowfile. I usually disconnect the node, then once it's disconnected I restart nifi, and then once it rejoins the cluster that flow file is gone. If we try to empty the queue, it w

Re: flowfiles stuck in load balanced queue; nifi 1.8

2018-12-26 Thread Mark Payne
Hey Dan, Thanks, this is super useful! So, the following section is the damning part of the JSON: { "totalFlowFileCount": 1, "totalByteCount": 975890, "nodeIdentifier": "nifi1-1:9443", "localQueuePartition": { "totalFlowFile

ConsumeKafka demarcator

2018-12-26 Thread Boris Tyukin
Hello, I would like to batch Kafka messages using demarcator property but I am not sure which one to use. In my case I get JSON messages and some message values can contain special characters and even newline delimiters in both Windows and Unix flavors (data is coming from GoldenGate, which stream

Re: InvokeHTTP failure to schedule fro CRON?

2018-12-26 Thread Andy LoPresto
Jeff did a really good job explaining what I glossed over earlier. Assuming the site you are trying to reach doesn’t have a certificate that is signed by a known CA (for example, a company service that is only accessed from within the organization), you can follow these instructions to obtain th

Re: InvokeHTTP failure to schedule fro CRON?

2018-12-26 Thread Jeff
There was a bit of a grammar issue with my previous message... InvokeHTTP should be *presented* with a cert that's signed by one of the CAs in cacerts. You can use your browser to go to the website/URL that you've configured in InvokeHTTP, and take a look at the certificate for the site. For exa