Re: Replacing a base64-encoded field in a JSON-document with its decoded/converted value

2020-06-24 Thread Andy LoPresto
Hello Bjørn, If the size of the encoded XML document is small (under ~1 KB), you can extract the Base64-encoded value to a flowfile attribute using EvaluateJSONPath, perform the decoding using the base64Decode Expression Language function [1], and then replace it into the flowfile JSON content

Re: MergeContent resulting in corrupted JSON

2020-06-24 Thread Jason Iannone
Exactly my thought, and we've been combing through the code but nothing significant has jumped out. Something that does are Nifi JIRA's, NIFI-6923, NIFI-6924, and NIFI-6846. Considering we're on 1.10.0 I've requested upgrading to 1.11.4. Thanks, Jason On Tue, Jun 23, 2020 at 9:05 AM Mark Payne w

Re: MergeContent resulting in corrupted JSON

2020-06-24 Thread Darren Govoni
Just lurking this thread. You wrote earlier. The next time that a processor needs to write to the content of a FlowFile, it may end up appending to that same file on disk, but the FlowFile that the content corresponds to will keep track of the byte offset into the file where its content begins

FetchS3Object throwing com.amazonaws.SdkClientException

2020-06-24 Thread Wyll Ingersoll
Using Nifi 1.11.4, the FetchS3Object processor occasionally throws the following error: routing to failure: com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond. The full stack trace looks like: com.amazonaws.SdkClientException: Unable to execut

Indications in the UI of which cluster node hosts a “stuck” thread?

2020-06-24 Thread James McMahon
Our production nifi cluster is exhibiting repeated problems with threads that do not end. It is happening with processors that have complex configurations and dependencies (ConsumeAMQP), and - more troubling - it is also occurring periodically for simple processors like ControlRate. I’ll have a Con

Re: Indications in the UI of which cluster node hosts a “stuck” thread?

2020-06-24 Thread Matt Gilman
Hi Jim, If you open the Summary page from the global menu you should see the active threads in parentheses next to the scheduled state. Find the row in question and click the cluster icon from the actions column. This will open a dialog with a node-wise breakdown. I believe that the thread coun