Oh. Sorry :-)
On Mon, Sep 15, 2014 at 3:27 AM, Mark Walkom ma...@campaignmonitor.com
wrote:
You probably want to put this in your own thread :)
Regards,
Mark Walkom
Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com
On 15 September
Hi,
I am trying to use Spark and ElasticSearch.
Currently, the RDD contains pipe delimited records.
parsedRDD.saveAsNewAPIHadoopFile(outputLocation,
NullWritable.class,
Text.class,
CustomTextOutputFormat.class,
job.getConfiguration());
Write now I am storing the output in HDFS.
Here is the gradle build I was using originally:
apply plugin: 'java'
apply plugin: 'eclipse'
sourceCompatibility = 1.7
version = '0.0.1'
group = 'com.spark.testing'
repositories {
mavenCentral()
}
dependencies {
compile 'org.apache.spark:spark-core_2.10:1.0.0'
compile
Thanks for the analysis. It looks like Hadoop 1.0.4 POM has an invalid pom
- though it uses Jackson 1.8.8 (see the distro) the pom declares version
1.0.1 for some reason. Hadoop version 1.2 (the latest stable) and higher
has this fixed.
We don't mark the jackson version within our POM since it's
I figured it out, dependency issue in my classpath. Maven was pulling down
a very old version of the jackson jar. I added the following line to my
dependencies and the error went away:
compile 'org.codehaus.jackson:jackson-mapper-asl:1.9.13'
On Friday, July 4, 2014 3:22:30 PM UTC-4, Brian
Hi,
Glad to see you sorted out the problem. Out of curiosity what version of
jackson were you using and what was pulling it in? Can you share you maven
pom/gradle build?
On Sun, Jul 6, 2014 at 10:27 PM, Brian Thomas brianjthoma...@gmail.com
wrote:
I figured it out, dependency issue in my
I am trying to test querying elasticsearch using Apache Spark using
elasticsearch-hadoop. I am just trying to do a query to the elasticsearch
server and return the count of results.
Below is my test class using the Java API:
import org.apache.hadoop.conf.Configuration;
import
