Spark 1.6.1 uses this version of jackson:
<fasterxml.jackson.version>2.4.4</fasterxml.jackson.version>
Looks like Tranquility uses different version of jackson.
How do you build your jar ?
Consider using maven-shade-plugin to resolve the conflict if you use maven.
Cheers
On Thu, Mar 31, 2016 at 9:50 AM, Marcelo Oikawa <marcelo.oik...@webradar.com
> wrote:
> Hi, list.
>
> We are working on a spark application that sends messages to Druid. For
> that, we're using Tranquility core. In my local test, I'm using the
> "spark-1.6.1-bin-hadoop2.6" distribution and the following dependencies in
> my app:
>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_2.10</artifactId>
> <version>1.6.1</version>
> <scope>provided</scope>
> </dependency>
> <dependency>
> <groupId>io.druid</groupId>
> <artifactId>tranquility-core_2.10</artifactId>
> <version>0.7.4</version>
> </dependency>
>
> But i getting the error down below when Tranquility tries to create
> Tranquilizer object:
>
> tranquilizer =
> DruidBeams.fromConfig(dataSourceConfig).buildTranquilizer(tranquilizerBuider);
>
> The stacktrace is down below:
>
> java.lang.IllegalAccessError: tried to access method
> com.fasterxml.jackson.databind.introspect.AnnotatedMember.getAllAnnotations()Lcom/fasterxml/jackson/databind/introspect/AnnotationMap;
> from class
> com.fasterxml.jackson.databind.introspect.GuiceAnnotationIntrospector
> at
> com.fasterxml.jackson.databind.introspect.GuiceAnnotationIntrospector.findInjectableValueId(GuiceAnnotationIntrospector.java:39)
> at
> com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.findInjectableValueId(AnnotationIntrospectorPair.java:269)
> at
> com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addDeserializerConstructors(BasicDeserializerFactory.java:433)
> at
> com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:325)
> at
> com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:266)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:266)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:168)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:399)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:348)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:261)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:241)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
> at
> com.fasterxml.jackson.databind.DeserializationContext.findContextualValueDeserializer(DeserializationContext.java:380)
> at
> com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.construct(PropertyBasedCreator.java:96)
> at
> com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:413)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:292)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:241)
> at
> com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:142)
> at
> com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:394)
> at
> com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3169)
> at
> com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:2767)
> at
> com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:2700)
> at
> com.metamx.tranquility.druid.DruidBeams$.fromConfigInternal(DruidBeams.scala:192)
> at
> com.metamx.tranquility.druid.DruidBeams$.fromConfig(DruidBeams.scala:119)
> at com.metamx.tranquility.druid.DruidBeams.fromConfig(DruidBeams.scala)
>
> Does someone faced that problem too?
>
> I know that it's related to jackson lib conflict but could anyone please
> shed some light? I created a jar with dependencies and when I submit a job
> for spark, does it run with just with the libraries inside the jar, right?
> Where is the conflict between jacksons libraries?
>
