[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350837#comment-15350837
]
Ian Hellstrom commented on SPARK-8332:
--
Is there a solution for this?
I'm running 1.4.1 on a Hortonworks cluster. One has Kerberos the other doesn't.
On the non-Kerberos cluster the same job runs fine and on the one with Kerberos
I get the same problem. AFAIK, both clusters are identical except for Kerberos.
At least the /usr/hdp/ directories have the same library versions in them.
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15006828#comment-15006828
]
Pedro Vilaça commented on SPARK-8332:
-
We're facing the same problem with a spark streaming job and I noticed that
this issue was closed.
Don't you have a plan to upgrade the jackson version that is being used?
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965484#comment-14965484
]
Jonathan Kelly commented on SPARK-8332:
---
Oh, you might be right. I was thinking that I had tried compiling Spark with a
newer version of Jackson (2.5.3) and still ran into problems, but now I'm
second guessing myself. It might have only been that my classpath had both
versions.
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965459#comment-14965459
]
Jonathan Kelly commented on SPARK-8332:
---
[~srowen], isn't this still a problem? It seems odd that Spark cannot be
upgraded beyond Jackson 2.4.4. It would at least be nice if Jackson were
shadowed inside the Spark assembly so that users may use newer versions of
Jackson in application code.
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965477#comment-14965477
]
Sean Owen commented on SPARK-8332:
--
I think Spark can use a later Jackson, though that's a different question. The
question here is version mismatch.
Shading might be possible too. I have this distant memory that it causes some
problem, but it could be gone now. A first step may be to just see if shading
works (i.e. tests pass). If so, this is likely the best way forward yes.
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965703#comment-14965703
]
Jonathan Kelly commented on SPARK-8332:
---
Good news! I tried compiling Spark with Jackson 2.5.3 instead of 2.4.4, and it
seems to work just fine (as long as this is the only version of Jackson 2.x in
the classpath; the issue described above does indeed only occur when multiple
versions are in the classpath). Would it be possible to upgrade to Jackson
2.5.3 in Spark 1.6.0? If so, should I cut an issue for this?
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965829#comment-14965829
]
Sean Owen commented on SPARK-8332:
--
How about 2.6, just out of curiosity?
The other issue is colliding with Hadoop's Jackson, which I'm pretty sure is
also not shaded. Of course, shading is an answer there, for Spark, also worth a
try. It may be that it just pushes the problem around, and now your app still
has to shade its Jackson to dodge Hadoop's copy, until Hadoop 3 at least.
What a mess eh?
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14965900#comment-14965900
]
Jonathan Kelly commented on SPARK-8332:
---
Sorry, yes, I should have checked to see what the latest version of Jackson 2.x
is now. Looks like it's Jackson 2.6.2, so we'd probably want to try to upgrade
to that if anything.
And yes, Hadoop's Jackson can also collide, so we'd either need to upgrade it
or shade it there as well. Classpath hell is hell. :(
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964562#comment-14964562
]
Earthson Lu commented on SPARK-8332:
SparkUI not works when upgrade fasterxml.jackson to 2.5.3
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14964563#comment-14964563
]
Earthson Lu commented on SPARK-8332:
SparkUI not works when upgrade fasterxml.jackson to 2.5.3
> NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> --
>
> Key: SPARK-8332
> URL: https://issues.apache.org/jira/browse/SPARK-8332
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
>Affects Versions: 1.4.0
> Environment: spark 1.4 & hadoop 2.3.0-cdh5.0.0
>Reporter: Tao Li
>Priority: Critical
> Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
>
> I complied new spark 1.4.0 version.
> But when I run a simple WordCount demo, it throws NoSuchMethodError
> {code}
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
> {code}
> I found out that the default "fasterxml.jackson.version" is 2.4.4.
> Is there any wrong or conflict with the jackson version?
> Or is there possibly some project maven dependency containing the wrong
> version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636919#comment-14636919
]
Earthson Lu commented on SPARK-8332:
I recompiled spark with fasterxml.jackson 2.5.3, it works with play-2.4.x
I want to know: is this ok to use 2.5.3 instead of 2.4.4?
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621022#comment-14621022
]
Sean Owen commented on SPARK-8332:
--
This has always happened to me when using Jackson 2.5. Just use Jackson 2.4.x
if you can. It's not shaded in Spark so you can't necessarily mix a different
version in your app.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14620975#comment-14620975
]
Kevin Tham commented on SPARK-8332:
---
I had no issue having play-json_2.11:2.4.0 (which depends on jackson 2.5.3) in
my SBT build with Spark 1.3, and then upgrading to Spark 1.4 leads me to not be
able to do an sc.parallelize(someCollection) where the collection is a
Seq[(TreeMap[String, Double], Double)] and am able to reproduce the error
Jonathan Kelly saw with (Exception in thread main
com.fasterxml.jackson.databind.JsonMappingException: Could not find creator
property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope))
I would help when I have time to figure out why, but I'm interested to see what
commit caused this and the actual cause of this error. This is causing us to
stick with Spark 1.3 for now.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621067#comment-14621067
]
Jonathan Kelly commented on SPARK-8332:
---
Thanks, Sean. Is there any reason why Jackson can't be shaded in the Spark
assembly? It's such a common package to use that it would really help for it to
be shaded so that Spark apps aren't locked in to a particular Jackson version.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14604046#comment-14604046
]
Olivier Girardot commented on SPARK-8332:
-
Great [~jonathak] can you please post the configuration parameters you've been
using (including the classpath) ?
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602468#comment-14602468
]
Tao Li commented on SPARK-8332:
---
I found that BigDecimalDeserializer will extend StdDeserializer in
jackson-databind project. Above version jackson-databind-2.3, StdDeserializer
have method handledType(). But under version jackson-databind-2.2,
StdDeserializer don't have method handledType().
In my environment, hadoop 2.3.0-cdh5.0.0, the jackson-databind is
/usr/lib/hadoop-mapreduce/.//jackson-databind-2.2.3.jar. So it don't have the
method handledType() and throws NoSuchMethodError.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602477#comment-14602477
]
Olivier Girardot commented on SPARK-8332:
-
Ok can you print the command line you're using to submit your job.
My conclusion was that it's difficult to create classpath compatible between
hive/hadoop from CDH5.x and Spark if you want to use hive tables...
I'd like to see your classpath in order to better understand what is going on.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603881#comment-14603881
]
Jonathan Kelly commented on SPARK-8332:
---
Actually, it seems like it may have been the fact that I was using Jackson
2.5.3 instead of Jackson 2.4.4. I downgraded and was able to run the SparkPi
example successfully.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14603751#comment-14603751
]
Jonathan Kelly commented on SPARK-8332:
---
I was running into the same issue, so I made sure to get rid of all Jackson 2.2
jars from my Hadoop classpath, but now I've run into a different issue:
Exception in thread main com.fasterxml.jackson.databind.JsonMappingException:
Could not find creator property with name 'id' (in class
org.apache.spark.rdd.RDDOperationScope)
at [Source: {id:0,name:parallelize}; line: 1, column: 1]
at
com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at
com.fasterxml.jackson.databind.DeserializationContext.mappingException(DeserializationContext.java:843)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.addBeanProps(BeanDeserializerFactory.java:533)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:220)
at
com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:358)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:265)
at
com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:245)
at
com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:143)
at
com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:439)
at
com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666)
at
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558)
at
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2578)
at
org.apache.spark.rdd.RDDOperationScope$.fromJson(RDDOperationScope.scala:82)
at org.apache.spark.rdd.RDD$$anonfun$34.apply(RDD.scala:1486)
at org.apache.spark.rdd.RDD$$anonfun$34.apply(RDD.scala:1486)
at scala.Option.map(Option.scala:145)
at org.apache.spark.rdd.RDD.init(RDD.scala:1486)
at
org.apache.spark.rdd.ParallelCollectionRDD.init(ParallelCollectionRDD.scala:85)
at
org.apache.spark.SparkContext$$anonfun$parallelize$1.apply(SparkContext.scala:697)
at
org.apache.spark.SparkContext$$anonfun$parallelize$1.apply(SparkContext.scala:695)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:148)
at
org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:109)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:681)
at org.apache.spark.SparkContext.parallelize(SparkContext.scala:695)
at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:31)
at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
I'm just running: MASTER=yarn-client /usr/lib/spark/bin/run-example SparkPi 10
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602698#comment-14602698
]
Olivier Girardot commented on SPARK-8332:
-
Great Tao !
can you please send your SPARK_CLASSPATH ?
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14602693#comment-14602693
]
Tao Li commented on SPARK-8332:
---
Hi Olivier Girardot, you can see all the jars in your classpath at the web UI
Environment tab. For this case, you can check if you have old version jackson
related jars (under version 2.2).
If you want to custom your own classpath, you can edit conf/spark-env.sh file
and change SPARK_CLASSPATH variable.
For me, I have add 'hadoop classpath' in my SPARK_CLASSPATH. But hadoop
classpath contains 2.2 version jackson jars. So I have to exclude the jars in
the SPARK_CLASSPATH. It help me to overcome this NoSuchMethodError.
So I suggest you to check your SPARK_CLASSPATH carefully.
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[
https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593898#comment-14593898
]
Olivier Girardot commented on SPARK-8332:
-
You're right sorry
NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
--
Key: SPARK-8332
URL: https://issues.apache.org/jira/browse/SPARK-8332
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.4.0
Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0
Reporter: Tao Li
Priority: Critical
Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson
I complied new spark 1.4.0 version.
But when I run a simple WordCount demo, it throws NoSuchMethodError
{code}
java.lang.NoSuchMethodError:
com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
{code}
I found out that the default fasterxml.jackson.version is 2.4.4.
Is there any wrong or conflict with the jackson version?
Or is there possibly some project maven dependency containing the wrong
version of jackson?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8332) NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer
[ https://issues.apache.org/jira/browse/SPARK-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589903#comment-14589903 ] Olivier Girardot commented on SPARK-8332: - I have the same issue on a CDH5 cluster NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer -- Key: SPARK-8332 URL: https://issues.apache.org/jira/browse/SPARK-8332 Project: Spark Issue Type: Bug Components: Spark Core Affects Versions: 1.4.0 Environment: spark 1.4 hadoop 2.3.0-cdh5.0.0 Reporter: Tao Li Priority: Critical Labels: 1.4.0, NoSuchMethodError, com.fasterxml.jackson I complied new spark 1.4.0 versio. But when I run a simple WordCount demo, it throws NoSuchMethodError java.lang.NoSuchMethodError: com.fasterxml.jackson.module.scala.deser.BigDecimalDeserializer. I found the default fasterxml.jackson.version is 2.4.4. It's there any wrong or conflict with the jackson version? Or is there possible some project maven dependency contains wrong version jackson? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
