am trying to understand how spark and ES work... could someone please help
me answer this question..
val conf = new Configuration()
conf.set("es.resource", "radio/artists")
conf.set("es.query", "?q=me*")
val esRDD = sc.newHadoopRDD(conf, classOf[EsInputFormat[Text,
MapWritable]],
classOf[Text], classOf[MapWritable]))
val docCount = esRDD.count();
When and where is data being transferred from ES? is it all collected on
the Spark master node, then partitioned and sent to the worker nodes? or is
each worker node talking to ES to somehow get a partition of the data?
How does this effectively work?
Thanks a lot,
Mohamed.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEU_gmf9Nt0xn_0NbzDn_moRWUT96uWYf4cicJdZik3r0Zz8XA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
