Hello
I followed the instructions here
https://flume.apache.org/FlumeUserGuide.html#elasticsearchsink and I am using a
0.90.7 jar on the flume server against an elastic search cluster running 0.90.7.
Also sending plain text through the netcat source works fine. The only problem
is when the text that is sent is formatted as json
If I send the words plain text over the netcat source it ends up like this in
elasticsearch
{"@message":"plain
text\r","@timestamp":"2014-04-10T14:16:38.113Z","@fields":{"timestamp":"1397139398113"}}
If I send { “plain” : “text” }
{"@message":"org.elasticsearch.common.xcontent.XContentBuilder@4710ad01","@timestamp":"2014-04-10T14:20:00.871Z","@fields":{"timestamp":"1397139600871"}}
Also sending just {} over the netcat source results in the same error.
Thanks,
Simeon
On Thursday, April 10, 2014 2:48:49 AM, Mike Percy <[email protected]> wrote:
Not sure either but make sure you're using a compatible version of
ElasticSearch.
Sent from my iPhone
On Apr 9, 2014, at 9:43 PM, Hari Shreedharan <[email protected]> wrote:
Then I really don't know what the issue is. Someone more familiar with elastic
search sink will need to look at it.
>
>
>Hari
>
>On Wednesday, April 9, 2014, Simeon Zaharici <[email protected]> wrote:
>
>Hello
>>
>>
>>This is to be able to easily integrate log centralization clients such as
>>nxlog with flume
>>
>>
>>The problem is not related to the splitting, or the size, even a message like
>>{ "blah": "blah" } will result in the exception
>>"org.elasticsearch.common.xcontent.XContentBuilder"
>>
>>
>>Simeon
>>On Wednesday, April 9, 2014 9:09:18 PM, Hari Shreedharan
>><[email protected]> wrote:
>>
>>Why exactly do you want to send JSON over netcat? Why don't you simply use
>>the Avro Source with Flume SDK? I am not entirely sure about the Elastic
>>search sink. The reason your JSON is not working is likely because netcat is
>>splitting the events per newline or when it reaches a fixed size.
>>
>>
>>
>>
>>Hari
>>
>>
>>
>>On Wed, Apr 9, 2014 at 5:58 PM, Simeon Zaharici <[email protected]>
>>wrote:
>>
>>Hello
>>>
>>>
>>>No, I did not find any solution to this
>>>
>>>
>>>Simeon
>>>On Monday, April 7, 2014 6:09:21 AM, Deepak Subhramanian
>>><[email protected]> wrote:
>>>
>>>Hi Simeon,
>>>I am facing the same problem. Did you find any solution ?
>>>
>>>
>>>Thanks, Deepak
>>>
>>>
>>>
>>>On Wed, Mar 5, 2014 at 5:52 PM, Simeon Zaharici <[email protected]>
>>>wrote:
>>>
>>>Hi guys
>>>>
>>>>
>>>>I would like to send json data to a flume netcat source and then send it to
>>>>an elasticsearch sink that uses the
>>>>org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer.
>>>>
>>>>It works to send plain text on the netcat source, however when I send json
>>>>the event ends up in elasticsearch as
>>>>'org.elasticsearch.common.xcontent.XContentBuilder'.
>>>>
>>>>Which would be the easiest way to go about this ? I saw from reading docs
>>>>that creating a morphline interceptor could be the way to go but I did not
>>>>fully understand how that
works.
>>>>
>>>>Thanks a lot
>>>>
>>>
>>>
>>>
>>>--
>>>Deepak Subhramanian
>>>
>>>
>>
>>
>>
