Quick follow-up: this works sweetly with spark-1.1.1-bin-hadoop2.4.
On Dec 3, 2014, at 3:31 PM, Ian Wilkinson ia...@me.com wrote:
Hi,
I'm trying the Elasticsearch support for Spark (2.1.0.Beta3).
In the following I provide the query (as query dsl):
import org.elasticsearch.spark
Hi,
I'm trying the Elasticsearch support for Spark (2.1.0.Beta3).
In the following I provide the query (as query dsl):
import org.elasticsearch.spark._
object TryES {
val sparkConf = new SparkConf().setAppName(Campaigns)
sparkConf.set(es.nodes, es_cluster:9200)
.
Unfortunately, unless you can use Spark on EMR, I guess your options are to
hack it by spinning up an EMR cluster and getting the JAR, or maybe fall back
to using boto and rolling your own :(
On Fri, Jul 4, 2014 at 9:28 AM, Ian Wilkinson ia...@me.com wrote:
Trying to discover source
Hi,
I’m trying to run the Spark (1.0.0) shell on EMR and encountering a classpath
issue.
I suspect I’m missing something gloriously obviously, but so far it is eluding
me.
I launch the EMR Cluster (using the aws cli) with:
aws emr create-cluster --name Test Cluster \
--ami-version
a HadoopRDD from that.
On Fri, Jul 4, 2014 at 8:28 AM, Ian Wilkinson ia...@me.com wrote:
Hi,
I noticed mention of DynamoDB as input source in
http://ampcamp.berkeley.edu/wp-content/uploads/2012/06/matei-zaharia-amp-camp-2012-advanced-spark.pdf.
Unfortunately, Google is not coming to my
- inputFormat takes
care of this so should be the easier approach
—
Sent from Mailbox
On Fri, Jul 4, 2014 at 8:51 AM, Ian Wilkinson ia...@me.com wrote:
Excellent. Let me get browsing on this.
Huge thanks,
ian
On 4 Jul 2014, at 16:47, Nick Pentreath nick.pentre...@gmail.com
