[
https://issues.apache.org/jira/browse/SPARK-40911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
PJ Fanning updated SPARK-40911:
-------------------------------
Description:
This 2.14.0 release is still a few weeks a way. There is an rc2 release but
there will probably be an rc3 before a full release.
The reason I marked the Jira component as 'Java API' is that this issue will
affect Java users more than Scala users.
I raised this separately to SPARK-40666 because I have a different reason for
this issue. SPARK-40666 can probably already be closed as the CVE is fixed in
jackson-databind 2.13.4.2.
There are performance issues in jackson-module-scala 2.13.x that may affect
some Spark users. Specifically, the per issue is
[https://github.com/FasterXML/jackson-module-scala/issues/576]
Scala3 support added in jackson-module-scala 2.13.0 means that if you use Scala
2.13, you should be able to use Scala3 compiled classes with
jackson-module-scala. Scala3 compiled classes are harder to recognise using
runtime reflection (and Jackson is built around runtime reflection). Scala2
compiled classes have specific annotations. With Scala3 compiled classes, we
need to look for .tasty files. This lookup can be slow if you have a lot of
jars (or big jars). Issue
[576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an
issue where this .tasty lookup is done every time you try to
serialize/deserialize a Java class with an ObjectMapper that has the
DefaultScalaModule registered. I will also disable the .tasty file lookups for
Scala 2.11/2.12 as they are not useful for those users.
For Spark usage, it may be worth turning off this .tasty file support
altogether. This is another enhancement in jackson-module-scala (but not in the
RC2 release).
I will follow up and update this issue when the v2.14.0 release is ready. This
change will require updating all Jackson jars to v2.14.0 (as Jackson does not
support using version mismatches - except at patch version level).
was:
This 2.14.0 release is still a few weeks a way. There is an rc2 release but
there will probably be an rc3 before a full release.
The reason I marked the Jira component as 'Java API' is that this issue will
affect Java users more than Scala users.
I raised this separately to SPARK-40666 because I have a different reason for
this issue. SPARK-40666 can probably already be closed as the CVE is fixed in
jackson-databind 2.13.4.2.
There are performance issues in jackson-module-scala 2.13.x that may affect
some Spark users. Specifically, the jackson issue is
[https://github.com/FasterXML/jackson-module-scala/issues/576]
Scala3 support added in jackson-module-scala 2.13.0 means that if you use Scala
2.13, you should be able to use Scala3 compiled classes with
jackson-module-scala. Scala3 compiled classes are harder to recognise using
runtime reflection (and Jackson is built around runtime reflection). Scala2
compiled classes have specific annotations. With Scala3 compiled classes, we
need to look for .tasty files. This lookup can be slow if you have a lot of
jars (or big jars). Issue
[576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an
issue where this .tasty lookup is done every time you try to
serialize/deserialize a Java class with an ObjectMapper that has the
DefaultScalaModule registered.
For Scala usage, it may be worth turning off this .tasty file support
altogether. This is another enhancement in jackson-module-scala (but not in the
RC2 release).
I will follow up and update this issue when the v2.14.0 release is ready. This
change will require updating all Jackson jars to v2.14.0 (as Jackson does not
support using version mismatches - except at patch version level).
> Upgrade jackson-module-scala to 2.14.0
> --------------------------------------
>
> Key: SPARK-40911
> URL: https://issues.apache.org/jira/browse/SPARK-40911
> Project: Spark
> Issue Type: Improvement
> Components: Java API
> Affects Versions: 3.3.0
> Reporter: PJ Fanning
> Priority: Major
>
> This 2.14.0 release is still a few weeks a way. There is an rc2 release but
> there will probably be an rc3 before a full release.
> The reason I marked the Jira component as 'Java API' is that this issue will
> affect Java users more than Scala users.
> I raised this separately to SPARK-40666 because I have a different reason for
> this issue. SPARK-40666 can probably already be closed as the CVE is fixed in
> jackson-databind 2.13.4.2.
> There are performance issues in jackson-module-scala 2.13.x that may affect
> some Spark users. Specifically, the per issue is
> [https://github.com/FasterXML/jackson-module-scala/issues/576]
> Scala3 support added in jackson-module-scala 2.13.0 means that if you use
> Scala 2.13, you should be able to use Scala3 compiled classes with
> jackson-module-scala. Scala3 compiled classes are harder to recognise using
> runtime reflection (and Jackson is built around runtime reflection). Scala2
> compiled classes have specific annotations. With Scala3 compiled classes, we
> need to look for .tasty files. This lookup can be slow if you have a lot of
> jars (or big jars). Issue
> [576|https://github.com/FasterXML/jackson-module-scala/issues/576] fixes an
> issue where this .tasty lookup is done every time you try to
> serialize/deserialize a Java class with an ObjectMapper that has the
> DefaultScalaModule registered. I will also disable the .tasty file lookups
> for Scala 2.11/2.12 as they are not useful for those users.
> For Spark usage, it may be worth turning off this .tasty file support
> altogether. This is another enhancement in jackson-module-scala (but not in
> the RC2 release).
> I will follow up and update this issue when the v2.14.0 release is ready.
> This change will require updating all Jackson jars to v2.14.0 (as Jackson
> does not support using version mismatches - except at patch version level).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
