Re: attributes to json

[Bryan Bende]

Hello,

You could use ReplaceText to build a new Json document from flow file
attributes. Something like...

Replacement Text = { "myField" : "${attribute1}" }

-Bryan

On Saturday, October 3, 2015, Sumanth Chinthagunta 
wrote:

> do we have attributes to json processor ?
> I am thinking to use it along with ExtractText where the matching data is
> stored in attributes. now I need to convert those attributes into json
> flowFile.
> if we don’t have such processor, any ideas to use, existing processors to
> compose what I needed?
> thanks
> sumo



-- 
Sent from Gmail Mobile

Re: attributes to json

[Sumanth Chinthagunta]

Thanks Bryan. 
I guess I am not reading docs. wish we have more examples and blog posts for 
each existing processor.
developers prefer to learn from examples then reading docs :-)

I am trying to consolidate all flow examples on GitHub. 
I just finished a flow that connect NiFi to streaming dashboard:  
https://github.com/xmlking/nifi-examples/tree/master/collect-stream-logs 


sumo

> On Oct 3, 2015, at 5:46 PM, Bryan Bende  wrote:
> 
> Hello,
> 
> You could use ReplaceText to build a new Json document from flow file
> attributes. Something like...
> 
> Replacement Text = { "myField" : "${attribute1}" }
> 
> -Bryan
> 
> On Saturday, October 3, 2015, Sumanth Chinthagunta 
> wrote:
> 
>> do we have attributes to json processor ?
>> I am thinking to use it along with ExtractText where the matching data is
>> stored in attributes. now I need to convert those attributes into json
>> flowFile.
>> if we don’t have such processor, any ideas to use, existing processors to
>> compose what I needed?
>> thanks
>> sumo
> 
> 
> 
> -- 
> Sent from Gmail Mobile

Re: attributes to json

[Joe Witt]

Sumanth

This feature proposal captures where we think we'll be able to address
this desire for examples really well [1].

Thanks
Joe

[1] 
https://cwiki.apache.org/confluence/display/NIFI/Extension+and+Template+Registry

On Sun, Oct 4, 2015 at 10:47 PM, Sumanth Chinthagunta  wrote:
> Thanks Bryan.
> I guess I am not reading docs. wish we have more examples and blog posts for 
> each existing processor.
> developers prefer to learn from examples then reading docs :-)
>
> I am trying to consolidate all flow examples on GitHub.
> I just finished a flow that connect NiFi to streaming dashboard:  
> https://github.com/xmlking/nifi-examples/tree/master/collect-stream-logs 
> 
>
> sumo
>
>> On Oct 3, 2015, at 5:46 PM, Bryan Bende  wrote:
>>
>> Hello,
>>
>> You could use ReplaceText to build a new Json document from flow file
>> attributes. Something like...
>>
>> Replacement Text = { "myField" : "${attribute1}" }
>>
>> -Bryan
>>
>> On Saturday, October 3, 2015, Sumanth Chinthagunta 
>> wrote:
>>
>>> do we have attributes to json processor ?
>>> I am thinking to use it along with ExtractText where the matching data is
>>> stored in attributes. now I need to convert those attributes into json
>>> flowFile.
>>> if we don’t have such processor, any ideas to use, existing processors to
>>> compose what I needed?
>>> thanks
>>> sumo
>>
>>
>>
>> --
>> Sent from Gmail Mobile
>

Re: Attributes To JSON question

[Chris Sampson]

You could probably use a JOLT Transform on the NiFi side (although I’m no JOLT 
expert).

From an Elasticsearch point of view, if you create an index template [1] for 
the destination index with mappings to store the fields as numbers, I think 
Elasticsearch will do the conversion for you (pretty sure I’m doing this in 
places).


[1] 
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html
 



Cheers,

-- 
Chris Sampson

t: +44 7867 843675
e: chris.samp...@naimuri.com

> On 3 Jun 2020, at 14:12, DAVID SMITH  
> wrote:
> 
> Hi
> I am using the attributesToJSON processor to create a JSON flowfile which I 
> then send to Elastic, I have noticed that some of my attributes which ar 
> numbers such as file size always come out of the attributesToJSON processor 
> as string values (ie with double quotes around them), therefore when I send 
> my JSON string to Elastic it indexes all the fields as Strings and therefore 
> won't let me do a cumulative count of all filesizes.I have tried to cast my 
> filesize toNumber just before the attributes go into the attributesToJSON 
> processor and it makes no difference, do you know how I can correct this?
> Many thanksDave

Re: Attributes To JSON question

[Mark Payne]

David,

Attributes are always Strings. I don’t think you can change how they are 
represented in AttributesToJSON.
My recommendation would be to not use that processor and instead use 
ReplaceText with a Replacement Strategy as Always. Then you can build the JSON 
to look however you want by doing something like:

{
  “abc”: “xyz”,
  “filename": “${filename}”,
  “size”: ${fileSize},
  “otherAttribute”: “${someAttribute}”
}

Thanks
-Mark



> On Jun 3, 2020, at 9:12 AM, DAVID SMITH  
> wrote:
> 
> Hi
> I am using the attributesToJSON processor to create a JSON flowfile which I 
> then send to Elastic, I have noticed that some of my attributes which ar 
> numbers such as file size always come out of the attributesToJSON processor 
> as string values (ie with double quotes around them), therefore when I send 
> my JSON string to Elastic it indexes all the fields as Strings and therefore 
> won't let me do a cumulative count of all filesizes.I have tried to cast my 
> filesize toNumber just before the attributes go into the attributesToJSON 
> processor and it makes no difference, do you know how I can correct this?
> Many thanksDave