Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2015-04-08 Thread Paul Chua
I'm having an issue very similar to this; I'm not sure exactly what you did to get the array contents. I've made a new post here: https://groups.google.com/forum/#!topic/elasticsearch/MpOqKthgqtA -- Paul Chua Data Scientist 317-979-5643 [image: cid:02526A0B-9444-47C7-A3EC-12B05A02CB50] *We

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Elias Abou Haydar
That works fine for me thank you! But I'd also wanted to be able to build and object from the MapWritable values in the mapper. Consider values as MapWritable object. When I try to get a specified value from values.get(title) per example the returning value is null but the field exists in the

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Costin Leau
That's because your MapWritables doesn't use Strings as keys but rather org.apache.hadoop.io.Text In other words, you can see the data is in the map however you cannot retrieve it since you are using the wrong key (try inspecting the map object types). Try values.get(new Text(title)) On

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Elias Abou Haydar
I've tried that. It returns a org.elasticsearch.hadoop.mr.WritableArrayWritable object. How can I get my field content out of that? Le mercredi 3 décembre 2014 14:10:24 UTC+1, Costin Leau a écrit : That's because your MapWritables doesn't use Strings as keys but rather

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Costin Leau
You're getting back an array ([Samsung EF-C]) - a Writable wrapper around org.hadoop.io.ArrayWritable (to actually allow it to be serialized). So call toStrings() or get() to get its content. On 12/3/14 3:30 PM, Elias Abou Haydar wrote: I've tried that. It returns a

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Elias Abou Haydar
I've already tried that. It doesn't work... :/ Le mercredi 3 décembre 2014 16:21:40 UTC+1, Costin Leau a écrit : You're getting back an array ([Samsung EF-C]) - a Writable wrapper around org.hadoop.io.ArrayWritable (to actually allow it to be serialized). So call toStrings() or get()

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Elias Abou Haydar
I've tried to call toStrings() I got this : title : [Ljava.lang.String;@35112ff7 with the get(), i'm getting this: title : [Lorg.apache.hadoop.io.Writable;@666f5678 Le mercredi 3 décembre 2014 16:21:40 UTC+1, Costin Leau a écrit : You're getting back an array ([Samsung EF-C]) -

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Elias Abou Haydar
Ok actually Strings() returns the String[] array that has the contents and that solved my problem. Thanks again Costin! :) Le mercredi 3 décembre 2014 16:29:38 UTC+1, Elias Abou Haydar a écrit : I've tried to call toStrings() I got this : title : [Ljava.lang.String;@35112ff7 with

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Costin Leau
I'm not sure what you are expecting since the results are as expected. See the javadocs [1] for ArrayWritable. toStrings() returns a String[] while get() a Writable[]. In other words you get an array of Strings and Writables and neither implements toString natively. To get the actual content

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-03 Thread Elias Abou Haydar
I have actually went through the API and I get the big picture now. I appreciate your help. Thanks! :) Le mercredi 3 décembre 2014 16:50:33 UTC+1, Costin Leau a écrit : I'm not sure what you are expecting since the results are as expected. See the javadocs [1] for ArrayWritable.

elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-02 Thread Elias Abou Haydar
I'm trying to write a mapreduce job where I can query elasticsearch so it can return to me specific fields. Is there any way to do that? My mapping contains about 30 fields and I will need just 4 of them (_id,title,description,category) The way I was doing it is to process each answer to get

Re: elasticsearch-hadoop - getting specified fields from elasticsearch as an input to a mapreduce job.

2014-12-02 Thread Costin Leau
Simply specify the fields that you are interested in, in the query and you are good to go. [1] http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-fields.html On 12/2/14 12:52 PM, Elias Abou Haydar wrote: I'm trying to write a mapreduce job where I can query