Hi,
You recommend the native integration instead of MR and I see on the
official documentation that MR is recommended to read/write data to ES
using spark. Spark support Doc
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/spark.html
what would be the basic piece of code to
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/2.1.Beta/spark.html#spark-native
On 12/18/14 8:27 PM, chris wrote:
Hi,
You recommend the native integration instead of MR and I see on the official
documentation that MR is recommended to
read/write data to ES using spark. Spark
am trying to understand how spark and ES work... could someone please help
me answer this question..
val conf = new Configuration()
conf.set(es.resource, radio/artists)
conf.set(es.query, ?q=me*)
val esRDD = sc.newHadoopRDD(conf, classOf[EsInputFormat[Text,
MapWritable]],
Hi,
First off I recommend using the native integration (aka the Java/Scala APIs) instead of MapReduce. The latter works but
the former is better performing and more flexible.
ES works in a similar fashion to the HDFS store - the data doesn't go through the master rather, each task has its own
Great Thanks a lot Costin.
Are people supposed to deploy the Spark workers on the same ES cluster? I
guess it would make sense for data to remain local and avoid network
transfers altogether?
Thanks a lot,
Mohamed.
On Monday, December 8, 2014 10:19:12 AM UTC-5, Costin Leau wrote:
Hi,
