Well now that is pretty cool. Thanks James.
On Mon, Jan 9, 2017 at 5:27 PM, James Wing <jvw...@gmail.com> wrote:
> Nick,
>
> You could use ExecuteScript to manipulate the JSON. The sample ECMAScript
> below assumes that you already have the transformed timestamp as an
> attribute "timestamp":
>
> var flowFile = session.get();
> if (flowFile !== null) {
>
> var StreamCallback = Java.type("org.apache.nifi.
> processor.io.StreamCallback");
> var IOUtils = Java.type("org.apache.commons.io.IOUtils");
> var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");
>
> flowFile = session.write(flowFile, new
> StreamCallback(function(inputStream,
> outputStream) {
>
> var inputJSON = IOUtils.toString(inputStream,
> StandardCharsets.UTF_8);
> var contentObj = JSON.parse(inputJSON);
>
> contentObj.time = flowFile.getAttribute("timestamp");
>
> outputStream.write(JSON.stringify(contentObj).
> getBytes(StandardCharsets.UTF_8));
> }));
>
> session.transfer(flowFile, REL_SUCCESS);
> }
>
> You could also do the date conversion directly in Javascript with
> something like:
>
> contentObj.time = new Date(contentObj.time).valueOf() / 1000;
>
>
> Thanks,
>
> James
>
> On Mon, Jan 9, 2017 at 1:52 PM, Nick Carenza <
> nick.care...@thecontrolgroup.com> wrote:
>
>> Hey folks,
>>
>> I am having a hard time figuring out how to work with date values in json
>> documents using the standard processors available in Nifi.
>>
>> example flowfile:
>>
>> {
>> "time": "2017-01-01T01:14:55+00:00",
>> "any": {
>> "nested": "data"
>> }
>> }
>>
>> what i want:
>>
>> {
>> "time": 1483233295,
>> "any": {
>> "nested": "data"
>> }
>> }
>>
>> I can extract the time field into an attribute with EvaluateJsonPath and
>> then using nifi expression language I can get a timestamp in seconds:
>>
>> ${time:toDate("yyyy-MM-dd'T'hh:mm:ss'+00:00'"):toNumber():divide(1000)}
>>
>> I can't find a way to get that value back into my document without losing
>> nested data structures (AttributesToJson).
>>
>> I can't find anything in jolt-core that enables working with dates.
>>
>> Has anyone else been able to solve a transformation like this or know of
>> a way to do it with standard nifi processors?
>>
>> If not? What options for enhancement seem best?
>>
>> - Enhance EvaluateJsonPath to be able to write to individual properties
>> in a json document. Currently it can only overwrite the flowfile with the
>> value of a single jsonpath expression.
>>
>> example properties:
>> $.time : ${time:toDate("yyyy-MM-dd'T'hh:mm:ss'+00:00'"):toNumber():di
>> vide(1000)}
>>
>> - Extend Jolt with a customer transformer class
>>
>> - Enhance AttributesToJson to allow writing nested data structures.
>> Behind a flag, each attribute value would be tested to see if it is valid
>> json.
>>
>> Thanks,
>> Nick
>>
>
>
