Thanks everyone. While I’m naturally disappointed that this doesn’t exist, I am hyper charged about the responsiveness and enthusiasm of the NiFi community!
From: Matt Burgess <mattyb...@gmail.com<mailto:mattyb...@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Monday, March 21, 2016 at 1:58 PM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: Help on creating that flow that requires processing attributes in a flow content but need to preserve the original flow content One way (in NiFi 0.5.0+) is to use the ExecuteScript processor, which gives you full control over the session and flowfile(s). For example if you had JSON in your "kafka.key" attribute such as "{"data": {"myKey": "myValue"}}" , you could use the following Groovy script to parse out the value of the 'data.myKey' field: def flowfile = session.get() if(!flowfile) return def json = new groovy.json.JsonSlurper().parseText(flowfile.getAttribute('kafka.key')) flowfile = session.putAttribute(flowfile, 'myKey', json.data.myKey) session.transfer(flowfile, REL_SUCCESS) I put an example of this up as a Gist (https://gist.github.com/mattyb149/478864017ec70d76f74f) A possible improvement could be to add a "jsonPath" function to Expression Language, which could take any value (including an attribute) along with a JSONPath expression to evaluate against it... Regards, Matt On Mon, Mar 21, 2016 at 1:48 PM, McDermott, Chris Kevin (MSDU - STaTS/StorefrontRemote) <chris.mcderm...@hpe.com<mailto:chris.mcderm...@hpe.com>> wrote: Joe, Thanks for the reply. I think I was not clear. The JSON I need to evaluate is in a FlowFile attribute (kafka.key) which I need to be able to evaluate without modifying the original FlowFile content (which was read from the Kafka topic). What I can’t figure out is how to squirrel away the flowfile content so that I can write the value of the kafka.key attribute to the FlowFile content, so that I can process it with EvaluateJsonPath, and then read content I squirreled away back into the FlowFile content. I considered using the the DistributedMapCache, but there would be no guarantee what I added to the cache would still be there when I needed it back. On 3/21/16, 1:37 PM, "Joe Witt" <joe.w...@gmail.com<mailto:joe.w...@gmail.com>> wrote: >Chris, > >Sounds like you have the right flow in mind already. EvaluateJSONPath >does not write content. It merely evaluates the given jsonpath >expression against the content of the flowfile and if appropriate >creates a flowfile attribute of what it finds. > >For example if you have JSON from Twitter you can use EvaluateJsonPath >and add a property with a name >'twitter.user' and a value of '$.user.name<http://user.name>' > >Once you run the tweets through each flow file will have an attribute >called 'twitter.user' with the name found in the message. No >manipulation of content at all. Just promotes things it finds to flow >file attributes. > >Thanks >Joe > >On Mon, Mar 21, 2016 at 1:34 PM, McDermott, Chris Kevin (MSDU - >STaTS/StorefrontRemote) ><chris.mcderm...@hpe.com<mailto:chris.mcderm...@hpe.com>> wrote: >> What I need to do is read a file from Kafka. The Kafka key contains a JSON >> string which I need to turn in FlowFile attributes while preserving the >> original FlowFile content. Obviously I can use EvaluteJsonPath but that >> necessitates replacing the FlowFile content with the kaka.key attribute, >> thus loosing the original FlowFile content. I feel like I’m missing >> something fundamental.
