RE: Spark 2.0.0-preview ... problem with jackson core version

[Paolo Patierno]

Yes ! We got it  :-)

Btw it's not available on Maven yet. :-(

Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

> From: so...@cloudera.com
> Date: Sat, 2 Jul 2016 15:12:11 +0100
> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
> To: ppatie...@live.com
> CC: charles.al...@metamarkets.com; user@spark.apache.org
> 
> Ah, it looks like it was 2.5.3 as of 2.0.0-preview:
> 
> https://github.com/apache/spark/blob/2.0.0-preview/pom.xml#L164
> 
> but was updated to 2.6.5 soon after that, since it was 2.6.5 in 2.0.0-RC1:
> 
> https://github.com/apache/spark/blob/v2.0.0-rc1/pom.xml#L163
> 
> On Sat, Jul 2, 2016 at 3:04 PM, Paolo Patierno <ppatie...@live.com> wrote:
> > This sounds strange to me because here :
> >
> > https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.0-preview
> >
> > I see :
> >
> > com.fasterxml.jackson.module » jackson-module-scala_2.112.5.3
> >
> > So it seems that 2.0.0-preview is bringing jackson module scala 2.5.3 that
> > is what I see.
> >
> >
> >
> >> From: so...@cloudera.com
> >> Date: Sat, 2 Jul 2016 14:32:58 +0100
> >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
> >> To: ppatie...@live.com
> >> CC: charles.al...@metamarkets.com; user@spark.apache.org
> >
> >>
> >> This is something to do with your app. The version is 2.6.5 in master
> >> and branch-2.0, and jackson-module-scala is managed to this version
> >> along with all the other jackson artifacts.
> >>
> >> On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno <ppatie...@live.com> wrote:
> >> > What I see is the following ...
> >> >
> >> > - Working configuration
> >> >
> >> > Spark Version : "2.0.0-SNAPSHOT"
> >> >
> >> > The Vert.x library brings ...
> >> > jackson-annotations:2.6.0
> >> > jackson-core:2.6.1
> >> > jackson-databind:2.6.1
> >> >
> >> > Spark brings
> >> > jackson-annotations:2.6.5
> >> > jackson-core:2.6.5
> >> > jackson-databind:2.6.5
> >> > jackson-module-scala_2.11: 2.6.5
> >> >
> >> > The runtime will use the latest version 2.6.5 and all works fine.
> >> >
> >> > - NOT Working configuration
> >> >
> >> > Spark Version : "2.0.0-preview"
> >> >
> >> > The Vert.x library brings ... (same dependencies as before)
> >> > jackson-annotations:2.6.0
> >> > jackson-core:2.6.1
> >> > jackson-databind:2.6.1
> >> >
> >> > Spark brings
> >> > jackson-module-scala_2.11: 2.5.3
> >> >
> >> > The module scale related to jackson is 2.5.3 ... seems not work with
> >> > above
> >> > 2.6.0/2.6.1 version ... so the exception.
> >> >
> >> >
> >> >> From: so...@cloudera.com
> >> >> Date: Sat, 2 Jul 2016 08:34:45 +0100
> >> >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
> >> >> To: charles.al...@metamarkets.com
> >> >> CC: ppatie...@live.com; user@spark.apache.org
> >> >
> >> >>
> >> >> mvn dependency:tree?
> >> >>
> >> >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen
> >> >> <charles.al...@metamarkets.com> wrote:
> >> >> > I'm having the same difficulty porting
> >> >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to
> >> >> > spark2.x,
> >> >> > where I have to go track down who is pulling in bad jackson versions.
> >> >> >
> >> >>
> >> >> -
> >> >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> >> >>
> >>
> >> -
> >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> >>
> 
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> 
  

Re: Spark 2.0.0-preview ... problem with jackson core version

[Sean Owen]

Ah, it looks like it was 2.5.3 as of 2.0.0-preview:

https://github.com/apache/spark/blob/2.0.0-preview/pom.xml#L164

but was updated to 2.6.5 soon after that, since it was 2.6.5 in 2.0.0-RC1:

https://github.com/apache/spark/blob/v2.0.0-rc1/pom.xml#L163

On Sat, Jul 2, 2016 at 3:04 PM, Paolo Patierno <ppatie...@live.com> wrote:
> This sounds strange to me because here :
>
> https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.0-preview
>
> I see :
>
> com.fasterxml.jackson.module » jackson-module-scala_2.112.5.3
>
> So it seems that 2.0.0-preview is bringing jackson module scala 2.5.3 that
> is what I see.
>
>
>
>> From: so...@cloudera.com
>> Date: Sat, 2 Jul 2016 14:32:58 +0100
>> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
>> To: ppatie...@live.com
>> CC: charles.al...@metamarkets.com; user@spark.apache.org
>
>>
>> This is something to do with your app. The version is 2.6.5 in master
>> and branch-2.0, and jackson-module-scala is managed to this version
>> along with all the other jackson artifacts.
>>
>> On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno <ppatie...@live.com> wrote:
>> > What I see is the following ...
>> >
>> > - Working configuration
>> >
>> > Spark Version : "2.0.0-SNAPSHOT"
>> >
>> > The Vert.x library brings ...
>> > jackson-annotations:2.6.0
>> > jackson-core:2.6.1
>> > jackson-databind:2.6.1
>> >
>> > Spark brings
>> > jackson-annotations:2.6.5
>> > jackson-core:2.6.5
>> > jackson-databind:2.6.5
>> > jackson-module-scala_2.11: 2.6.5
>> >
>> > The runtime will use the latest version 2.6.5 and all works fine.
>> >
>> > - NOT Working configuration
>> >
>> > Spark Version : "2.0.0-preview"
>> >
>> > The Vert.x library brings ... (same dependencies as before)
>> > jackson-annotations:2.6.0
>> > jackson-core:2.6.1
>> > jackson-databind:2.6.1
>> >
>> > Spark brings
>> > jackson-module-scala_2.11: 2.5.3
>> >
>> > The module scale related to jackson is 2.5.3 ... seems not work with
>> > above
>> > 2.6.0/2.6.1 version ... so the exception.
>> >
>> >
>> >> From: so...@cloudera.com
>> >> Date: Sat, 2 Jul 2016 08:34:45 +0100
>> >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
>> >> To: charles.al...@metamarkets.com
>> >> CC: ppatie...@live.com; user@spark.apache.org
>> >
>> >>
>> >> mvn dependency:tree?
>> >>
>> >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen
>> >> <charles.al...@metamarkets.com> wrote:
>> >> > I'm having the same difficulty porting
>> >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to
>> >> > spark2.x,
>> >> > where I have to go track down who is pulling in bad jackson versions.
>> >> >
>> >>
>> >> -
>> >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>> >>
>>
>> -
>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>>

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

RE: Spark 2.0.0-preview ... problem with jackson core version

[Paolo Patierno]

This sounds strange to me because here :

https://mvnrepository.com/artifact/org.apache.spark/spark-core_2.11/2.0.0-preview

I see :

com.fasterxml.jackson.module
» jackson-module-scala_2.112.5.3

So it seems that 2.0.0-preview is bringing jackson module scala 2.5.3 that is 
what I see.



> From: so...@cloudera.com
> Date: Sat, 2 Jul 2016 14:32:58 +0100
> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
> To: ppatie...@live.com
> CC: charles.al...@metamarkets.com; user@spark.apache.org
> 
> This is something to do with your app. The version is 2.6.5 in master
> and branch-2.0, and jackson-module-scala is managed to this version
> along with all the other jackson artifacts.
> 
> On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno <ppatie...@live.com> wrote:
> > What I see is the following ...
> >
> > - Working configuration
> >
> > Spark Version : "2.0.0-SNAPSHOT"
> >
> > The Vert.x library brings ...
> > jackson-annotations:2.6.0
> > jackson-core:2.6.1
> > jackson-databind:2.6.1
> >
> > Spark brings
> > jackson-annotations:2.6.5
> > jackson-core:2.6.5
> > jackson-databind:2.6.5
> > jackson-module-scala_2.11: 2.6.5
> >
> > The runtime will use the latest version 2.6.5 and all works fine.
> >
> > - NOT Working configuration
> >
> > Spark Version : "2.0.0-preview"
> >
> > The Vert.x library brings ... (same dependencies as before)
> > jackson-annotations:2.6.0
> > jackson-core:2.6.1
> > jackson-databind:2.6.1
> >
> > Spark brings
> > jackson-module-scala_2.11: 2.5.3
> >
> > The module scale related to jackson is 2.5.3 ... seems not work with above
> > 2.6.0/2.6.1 version ... so the exception.
> >
> >
> >> From: so...@cloudera.com
> >> Date: Sat, 2 Jul 2016 08:34:45 +0100
> >> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
> >> To: charles.al...@metamarkets.com
> >> CC: ppatie...@live.com; user@spark.apache.org
> >
> >>
> >> mvn dependency:tree?
> >>
> >> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen
> >> <charles.al...@metamarkets.com> wrote:
> >> > I'm having the same difficulty porting
> >> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to
> >> > spark2.x,
> >> > where I have to go track down who is pulling in bad jackson versions.
> >> >
> >>
> >> -
> >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> >>
> 
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> 
  

Re: Spark 2.0.0-preview ... problem with jackson core version

[Sean Owen]

This is something to do with your app. The version is 2.6.5 in master
and branch-2.0, and jackson-module-scala is managed to this version
along with all the other jackson artifacts.

On Sat, Jul 2, 2016 at 1:35 PM, Paolo Patierno <ppatie...@live.com> wrote:
> What I see is the following ...
>
> - Working configuration
>
> Spark Version : "2.0.0-SNAPSHOT"
>
> The Vert.x library brings ...
> jackson-annotations:2.6.0
> jackson-core:2.6.1
> jackson-databind:2.6.1
>
> Spark brings
> jackson-annotations:2.6.5
> jackson-core:2.6.5
> jackson-databind:2.6.5
> jackson-module-scala_2.11: 2.6.5
>
> The runtime will use the latest version 2.6.5 and all works fine.
>
> - NOT Working configuration
>
> Spark Version : "2.0.0-preview"
>
> The Vert.x library brings ... (same dependencies as before)
> jackson-annotations:2.6.0
> jackson-core:2.6.1
> jackson-databind:2.6.1
>
> Spark brings
> jackson-module-scala_2.11: 2.5.3
>
> The module scale related to jackson is 2.5.3 ... seems not work with above
> 2.6.0/2.6.1 version ... so the exception.
>
>
>> From: so...@cloudera.com
>> Date: Sat, 2 Jul 2016 08:34:45 +0100
>> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
>> To: charles.al...@metamarkets.com
>> CC: ppatie...@live.com; user@spark.apache.org
>
>>
>> mvn dependency:tree?
>>
>> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen
>> <charles.al...@metamarkets.com> wrote:
>> > I'm having the same difficulty porting
>> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to
>> > spark2.x,
>> > where I have to go track down who is pulling in bad jackson versions.
>> >
>>
>> -
>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>>

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

RE: Spark 2.0.0-preview ... problem with jackson core version

[Paolo Patierno]

What I see is the following ...

- Working configuration

Spark Version : "2.0.0-SNAPSHOT"

The Vert.x library brings ...
jackson-annotations:2.6.0
jackson-core:2.6.1
jackson-databind:2.6.1

Spark brings
jackson-annotations:2.6.5
jackson-core:2.6.5
jackson-databind:2.6.5
jackson-module-scala_2.11: 2.6.5

The runtime will use the latest version 2.6.5 and all works fine.

- NOT Working configuration

Spark Version : "2.0.0-preview"

The Vert.x library brings ... (same dependencies as before)
jackson-annotations:2.6.0
jackson-core:2.6.1
jackson-databind:2.6.1

Spark brings
jackson-module-scala_2.11: 2.5.3

The module scale related to jackson is 2.5.3 ... seems not work with above 
2.6.0/2.6.1 version ... so the exception.


> From: so...@cloudera.com
> Date: Sat, 2 Jul 2016 08:34:45 +0100
> Subject: Re: Spark 2.0.0-preview ... problem with jackson core version
> To: charles.al...@metamarkets.com
> CC: ppatie...@live.com; user@spark.apache.org
> 
> mvn dependency:tree?
> 
> On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen
> <charles.al...@metamarkets.com> wrote:
> > I'm having the same difficulty porting
> > https://github.com/metamx/druid-spark-batch/tree/spark2 over to spark2.x,
> > where I have to go track down who is pulling in bad jackson versions.
> >
> 
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
> 
  

Re: Spark 2.0.0-preview ... problem with jackson core version

[Sean Owen]

mvn dependency:tree?

On Sat, Jul 2, 2016 at 12:46 AM, Charles Allen
 wrote:
> I'm having the same difficulty porting
> https://github.com/metamx/druid-spark-batch/tree/spark2 over to spark2.x,
> where I have to go track down who is pulling in bad jackson versions.
>

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Spark 2.0.0-preview ... problem with jackson core version

[Charles Allen]

I'm having the same difficulty porting
https://github.com/metamx/druid-spark-batch/tree/spark2 over to spark2.x,
where I have to go track down who is pulling in bad jackson versions.



On Fri, Jul 1, 2016 at 11:59 AM Sean Owen  wrote:

> Are you just asking why you can't use 2.5.3 in your app? because
> Jackson isn't shaded, which is sort of the bad news. But just use
> 2.6.5 too, ideally. I don't know where 2.6.1 is coming from, but Spark
> doesn't use it.
>
> On Fri, Jul 1, 2016 at 5:48 PM,   wrote:
> > In my project I found the library which brings Jackson core 2.6.5 and it
> is
> > used in conjunction with the requested Jackson scala module 2.5.3 wanted
> by
> > spark 2.0.0 preview. At runtime it's the cause of exception.
> >
> > Now I have excluded 2.6.5 using sbt but it could be dangerous for the
> other
> > library.
> >
> > Why this regression to Jackson 2.5.3 switching from 2.0.0 snapshot to
> > preview ?
> >
> > Thanks
> > Paolo
> >
> > Get Outlook for Android
> >
> >
> >
> >
> > On Fri, Jul 1, 2016 at 4:24 PM +0200, "Paolo Patierno" <
> ppatie...@live.com>
> > wrote:
> >
> > Hi,
> >
> > developing a custom receiver up today I used spark version
> "2.0.0-SNAPSHOT"
> > and scala version 2.11.7.
> > With these version all tests work fine.
> >
> > I have just switching to "2.0.0-preview" as spark version but not I have
> > following error :
> >
> > An exception or error caused a run to abort: class
> > com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides
> > final method
> >
> withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
> > java.lang.VerifyError: class
> > com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides
> > final method
> >
> withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
> > at java.lang.ClassLoader.defineClass1(Native Method)
> >
> > I see that using 2.0.0-SNAPSHOT the jackson core version is 2.6.5 ... now
> > 2.6.1 with module scala 2.5.3.
> >
> > Thanks,
> > Paolo.
> >
> > Paolo Patierno
> > Senior Software Engineer (IoT) @ Red Hat
> > Microsoft MVP on Windows Embedded & IoT
> > Microsoft Azure Advisor
> >
> > Twitter : @ppatierno
> > Linkedin : paolopatierno
> > Blog : DevExperience
>
> -
> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>
>

Re: Spark 2.0.0-preview ... problem with jackson core version

[Sean Owen]

Are you just asking why you can't use 2.5.3 in your app? because
Jackson isn't shaded, which is sort of the bad news. But just use
2.6.5 too, ideally. I don't know where 2.6.1 is coming from, but Spark
doesn't use it.

On Fri, Jul 1, 2016 at 5:48 PM,   wrote:
> In my project I found the library which brings Jackson core 2.6.5 and it is
> used in conjunction with the requested Jackson scala module 2.5.3 wanted by
> spark 2.0.0 preview. At runtime it's the cause of exception.
>
> Now I have excluded 2.6.5 using sbt but it could be dangerous for the other
> library.
>
> Why this regression to Jackson 2.5.3 switching from 2.0.0 snapshot to
> preview ?
>
> Thanks
> Paolo
>
> Get Outlook for Android
>
>
>
>
> On Fri, Jul 1, 2016 at 4:24 PM +0200, "Paolo Patierno" 
> wrote:
>
> Hi,
>
> developing a custom receiver up today I used spark version "2.0.0-SNAPSHOT"
> and scala version 2.11.7.
> With these version all tests work fine.
>
> I have just switching to "2.0.0-preview" as spark version but not I have
> following error :
>
> An exception or error caused a run to abort: class
> com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides
> final method
> withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
> java.lang.VerifyError: class
> com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides
> final method
> withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
> at java.lang.ClassLoader.defineClass1(Native Method)
>
> I see that using 2.0.0-SNAPSHOT the jackson core version is 2.6.5 ... now
> 2.6.1 with module scala 2.5.3.
>
> Thanks,
> Paolo.
>
> Paolo Patierno
> Senior Software Engineer (IoT) @ Red Hat
> Microsoft MVP on Windows Embedded & IoT
> Microsoft Azure Advisor
>
> Twitter : @ppatierno
> Linkedin : paolopatierno
> Blog : DevExperience

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Spark 2.0.0-preview ... problem with jackson core version

[ppatierno]

In my project I found the library which brings Jackson core 2.6.5 and it is 
used in conjunction with the requested Jackson scala module 2.5.3 wanted by 
spark 2.0.0 preview. At runtime it's the cause of exception.


Now I have excluded 2.6.5 using sbt but it could be dangerous for the other 
library.


Why this regression to Jackson 2.5.3 switching from 2.0.0 snapshot to preview ?


Thanks

Paolo


Get Outlook for Android






On Fri, Jul 1, 2016 at 4:24 PM +0200, "Paolo Patierno"  
wrote:





Hi,

developing a custom receiver up today I used spark version "2.0.0-SNAPSHOT" and 
scala version 2.11.7.
With these version all tests work fine.

I have just switching to "2.0.0-preview" as spark version but not I have 
following error :

An exception or error caused a run to abort: class 
com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final 
method 
withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
java.lang.VerifyError: class 
com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final 
method 
withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;
at java.lang.ClassLoader.defineClass1(Native Method)

I see that using 2.0.0-SNAPSHOT the jackson core version is 2.6.5 ... now 2.6.1 
with module scala 2.5.3.

Thanks,
Paolo.

Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience