Hi,
Glad to see you sorted out the problem. Out of curiosity what version of
jackson were you using and what was pulling it in? Can you share you maven
pom/gradle build?
On Sun, Jul 6, 2014 at 10:27 PM, Brian Thomas <brianjthoma...@gmail.com>
wrote:
> I figured it out, dependency issue in my classpath. Maven was pulling
> down a very old version of the jackson jar. I added the following line to
> my dependencies and the error went away:
>
> compile 'org.codehaus.jackson:jackson-mapper-asl:1.9.13'
>
>
> On Friday, July 4, 2014 3:22:30 PM UTC-4, Brian Thomas wrote:
>>
>> I am trying to test querying elasticsearch using Apache Spark using
>> elasticsearch-hadoop. I am just trying to do a query to the elasticsearch
>> server and return the count of results.
>>
>> Below is my test class using the Java API:
>>
>> import org.apache.hadoop.conf.Configuration;
>> import org.apache.hadoop.io.MapWritable;
>> import org.apache.hadoop.io.Text;
>> import org.apache.spark.SparkConf;
>> import org.apache.spark.api.java.JavaPairRDD;
>> import org.apache.spark.api.java.JavaSparkContext;
>> import org.apache.spark.serializer.KryoSerializer;
>> import org.elasticsearch.hadoop.mr.EsInputFormat;
>>
>> import scala.Tuple2;
>>
>> public class ElasticsearchSparkQuery{
>>
>> public static int query(String masterUrl, String
>> elasticsearchHostPort) {
>> SparkConf sparkConfig = new SparkConf().setAppName("
>> ESQuery").setMaster(masterUrl);
>> sparkConfig.set("spark.serializer",
>> KryoSerializer.class.getName());
>> JavaSparkContext sparkContext = new JavaSparkContext(sparkConfig);
>>
>> Configuration conf = new Configuration();
>> conf.setBoolean("mapred.map.tasks.speculative.execution", false);
>> conf.setBoolean("mapred.reduce.tasks.speculative.execution",
>> false);
>> conf.set("es.nodes", elasticsearchHostPort);
>> conf.set("es.resource", "media/docs");
>> conf.set("es.query", "?q=*");
>>
>> JavaPairRDD<Text, MapWritable> esRDD =
>> sparkContext.newAPIHadoopRDD(conf, EsInputFormat.class, Text.class,
>> MapWritable.class);
>> return (int) esRDD.count();
>> }
>> }
>>
>>
>> When I try to run this I get the following error:
>>
>>
>> 4/07/04 14:58:07 INFO executor.Executor: Running task ID 0
>> 14/07/04 14:58:07 INFO storage.BlockManager: Found block broadcast_0
>> locally
>> 14/07/04 14:58:07 INFO rdd.NewHadoopRDD: Input split: ShardInputSplit
>> [node=[5UATWUzmTUuNzhmGxXWy_w/S'byll|10.45.71.152:9200],shard=0]
>> 14/07/04 14:58:07 WARN mr.EsInputFormat: Cannot determine task id...
>> 14/07/04 14:58:07 ERROR executor.Executor: Exception in task ID 0
>> java.lang.NoSuchFieldError: ALLOW_UNQUOTED_FIELD_NAMES
>> at org.elasticsearch.hadoop.serialization.json.
>> JacksonJsonParser.<clinit>(JacksonJsonParser.java:38)
>> at org.elasticsearch.hadoop.serialization.ScrollReader.
>> read(ScrollReader.java:75)
>> at org.elasticsearch.hadoop.rest.RestRepository.scroll(
>> RestRepository.java:267)
>> at org.elasticsearch.hadoop.rest.ScrollQuery.hasNext(
>> ScrollQuery.java:75)
>> at org.elasticsearch.hadoop.mr.EsInputFormat$ShardRecordReader.next(
>> EsInputFormat.java:319)
>> at org.elasticsearch.hadoop.mr.EsInputFormat$ShardRecordReader.
>> nextKeyValue(EsInputFormat.java:255)
>> at org.apache.spark.rdd.NewHadoopRDD$$anon$1.hasNext(
>> NewHadoopRDD.scala:122)
>> at org.apache.spark.InterruptibleIterator.hasNext(
>> InterruptibleIterator.scala:39)
>> at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1014)
>> at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:847)
>> at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:847)
>> at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(
>> SparkContext.scala:1080)
>> at org.apache.spark.SparkContext$$anonfun$runJob$4.apply(
>> SparkContext.scala:1080)
>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.
>> scala:111)
>> at org.apache.spark.scheduler.Task.run(Task.scala:51)
>> at org.apache.spark.executor.Executor$TaskRunner.run(
>> Executor.scala:187)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1145)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>> Has anyone run into this issue with the JacksonJsonParser?
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/9c2b2f2e-5196-4a72-bfbc-4cd0fda9edf0%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/9c2b2f2e-5196-4a72-bfbc-4cd0fda9edf0%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJogdmd0fR795J7p3BRMOgN2U%2BTq3hYr9bzCKabmk9q8VOT0VQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
