I'm having an issue very similar to this one, but I'm not sure exactly what they did to get the array contents. I can't apply the toStrings() function to the arraywritable because it's in a writable. http://grokbase.com/t/gg/elasticsearch/14c2sb14rk/hadoop-getting-specified-fields-from-elasticsearch-as-an-input-to-a-mapreduce-job/14c3dz4san#14c3dz4san
I've got a map from document IDs to another map from fields to their contents. I'm interested in getting the contents of the concepts field, which is an array of tuples. My goal is to use this in a recommender system using spark. I'm working in scala. Looking up a document ID gives me a map from the field name concepts to the contents. I want the contents of the concepts field in the form of a map from text to relevance. I'd rather not save this as a string and parse it for efficiency reasons. scala> map("396974") warning: there were 1 deprecation warning(s); re-run with -deprecation for detai ls res34: org.apache.hadoop.io.MapWritable = {concepts=[{text=Jujutsu, relevance=0. 953505}, {text=Aikido, relevance=0.87238}, {text=Martial arts, relevance=0.56092 8}, {text=Morihei Ueshiba, relevance=0.535638}, {text=Mixed martial arts, releva nce=0.470634}, {text=Grappling, relevance=0.43726}, {text=Karate, relevance=0.43 5142}, {text=Brazilian Jiu-Jitsu, relevance=0.403991}]} scala> map("396974").get(new Text("concepts")) warning: there were 1 deprecation warning(s); re-run with -deprecation for detai ls res35: org.apache.hadoop.io.Writable = org.elasticsearch.hadoop.mr.WritableArray Writable@1077a7 My understanding is that this is a writable wrapper around an arraywritable object. I want to get the contents of this object as a map. Ultimately, I want is a map from document IDs to the corresponding concept map from text to relevance. The thread linked above suggests calling toStrings on the writablearraywritable object, but this doesn't work: scala> map("396974").get(new Text("concepts")).toStrings <console>:41: error: value toStrings is not a member of org.apache.hadoop.io.Wri table map("396974").get(new Text("concepts")).toStrings ^ Any advice will be greatly appreciated. Thanks, Paul -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/239004a9-81ed-4aef-9a19-35a64655fa50%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.