Hi, list.
We are working on a spark application that sends messages to Druid. For
that, we're using Tranquility core. In my local test, I'm using the
"spark-1.6.1-bin-hadoop2.6" distribution and the following dependencies in
my app:
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.6.1</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.druid</groupId>
<artifactId>tranquility-core_2.10</artifactId>
<version>0.7.4</version>
</dependency>
But i getting the error down below when Tranquility tries to create
Tranquilizer object:
tranquilizer =
DruidBeams.fromConfig(dataSourceConfig).buildTranquilizer(tranquilizerBuider);
The stacktrace is down below:
java.lang.IllegalAccessError: tried to access method
com.fasterxml.jackson.databind.introspect.AnnotatedMember.getAllAnnotations()Lcom/fasterxml/jackson/databind/introspect/AnnotationMap;
from class
com.fasterxml.jackson.databind.introspect.GuiceAnnotationIntrospector
at
com.fasterxml.jackson.databind.introspect.GuiceAnnotationIntrospector.findInjectableValueId(GuiceAnnotationIntrospector.java:39)
at
com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.findInjectableValueId(AnnotationIntrospectorPair.java:269)
at
com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addDeserializerConstructors(BasicDeserializerFactory.java:433)
at
com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:325)
at
com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:266)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:266)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:168)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:399)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:348)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:261)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:241)
at
com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
at
com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:380)
at
com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.construct(PropertyBasedCreator.java:96)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:413)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:292)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:241)
at
com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
at
com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:394)
at
com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3169)
at
com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2767)
at
com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700)
at
com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:192)
at
com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:119)
at com.metamx.tranquility.druid.DruidBeams.fromConfig(DruidBeams.scala)
Does someone faced that problem too?
I know that it's related to jackson lib conflict but could anyone please
shed some light? I created a jar with dependencies and when I submit a job
for spark, does it run with just with the libraries inside the jar, right?
Where is the conflict between jacksons libraries?
