Re: Json over netcat source

2014-05-13 Thread Deepak Subhramanian
Sorry. My mistake. It is loading JSON data properly after the temporary fix. On Thu, May 8, 2014 at 6:24 PM, Deepak Subhramanian deepak.subhraman...@gmail.com wrote: Hi Ashish, Thanks for the solution. I made the changes and I can see the JSON message now. There is a JIRA raised on the

Re: Json over netcat source

2014-05-12 Thread Deepak Subhramanian
Hi Ashish, Thanks for the solution. I made the changes and I can see the JSON message now. There is a JIRA raised on the same issue. https://issues.apache.org/jira/browse/FLUME-2126 From Hive when I load JSON data it automatically splits JSON fields to different columns. For some reason the

Re: Json over netcat source

2014-04-13 Thread Ashish
little more on the issue builder.field(fieldName, tmp); calls the XContentBuilder API where class type is determined and appropriate method is called. Since tmp, which is instance of XContentBuilder, doesn't match any of the defined if conditions it goes to final else where the tmp.toString() is

Re: Json over netcat source

2014-04-11 Thread Deepak Subhramanian
Thanks Simon. I am also struggling with no luck. I tried using the latest flume elastic search sink jar build from 1.5SNAPSHOT ,but still no luck. I will try to see if it is an issue with elastic search api . When I loaded json using hive it loaded JSON properly. But we have to pass a property

Re: Json over netcat source

2014-04-10 Thread Mike Percy
Not sure either but make sure you're using a compatible version of ElasticSearch. Sent from my iPhone On Apr 9, 2014, at 9:43 PM, Hari Shreedharan hshreedha...@cloudera.com wrote: Then I really don't know what the issue is. Someone more familiar with elastic search sink will need to

Re: Json over netcat source

2014-04-10 Thread Simeon Zaharici
Hello I followed the instructions here  https://flume.apache.org/FlumeUserGuide.html#elasticsearchsink and I am using a 0.90.7 jar on the flume server against an elastic search cluster running 0.90.7. Also sending plain text through the netcat source works fine. The only problem is when the

Re: Json over netcat source

2014-04-09 Thread Simeon Zaharici
Hello No, I did not find any solution to this Simeon On Monday, April 7, 2014 6:09:21 AM, Deepak Subhramanian deepak.subhraman...@gmail.com wrote: Hi Simeon, I am facing the same problem. Did you find any solution ? Thanks, Deepak  On Wed, Mar 5, 2014 at 5:52 PM, Simeon Zaharici

Re: Json over netcat source

2014-04-09 Thread Hari Shreedharan
Why exactly do you want to send JSON over netcat? Why don't you simply use the Avro Source with Flume SDK? I am not entirely sure about the Elastic search sink. The reason your JSON is not working is likely because netcat is splitting the events per newline or when it reaches a fixed size. Hari

Re: Json over netcat source

2014-04-07 Thread Deepak Subhramanian
Hi Simeon, I am facing the same problem. Did you find any solution ? Thanks, Deepak On Wed, Mar 5, 2014 at 5:52 PM, Simeon Zaharici simeon_zahar...@yahoo.comwrote: Hi guys I would like to send json data to a flume netcat source and then send it to an elasticsearch sink that uses the

Json over netcat source

2014-03-05 Thread Simeon Zaharici
Hi guys I would like to send json data to a flume netcat source and then send it to an elasticsearch sink that uses the org.apache.flume.sink.elasticsearch.ElasticSearchLogStashEventSerializer. It works to send plain text on the netcat source, however when I send json the event ends up in