[
https://issues.apache.org/jira/browse/SPARK-34023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259999#comment-17259999
]
Nick Nezis commented on SPARK-34023:
------------------------------------
In the same Google Groups
[post|https://groups.google.com/g/kryo-users/c/sBZ10dwrwFQ] mentioned above,
the author gave a good write-up on some of the risks and why a project might
not want to upgrade.
{quote}
I'm sure everyone's first question will be about what happens to data
serialized in an older Kryo version. Supporting that is noble for a minor
version increment, but the changes above are too extensive for that to work. It
would be dirty to have settings to disable free optimizations or enable bugs
we've fixed, solely to attempt loading previously serialized bytes. I don't
want the difficulties of data migration to prevent the library from evolving
and I feel an improvement and maintenance pass like this is long overdue.
It may not make sense to upgrade your projects to a new major version. Kryo has
been stable a long time. While it's nice to have as many projects on the latest
as possible, you should not feel obligated to upgrade if the benefits do not
make it worthwhile, given the pain involved. New projects of course benefit
from using the latest version.
Updating Kryo versions when you care about previously serialized bytes has
always been hard for all but the most trivial updates. Probably the safest way
to do this is to load data with an old version, then write it with the new
version. This can be done by using a class loader for the old Kryo classes.
Maybe we could provide an example for this. For this to work, obviously the
class files must not have not changed, only the Kryo version.
Please share your thoughts! Now is the time for us to make any big changes and
improvements to both the API and serialized data that wouldn't make sense in a
minor version bump. Any places in the API you don't like, are awkward, or
confusing? Have any ideas for improving the API? Have any ideas for more
efficient serialization?
{quote}
> Updated to Kryo 5.0.3
> ---------------------
>
> Key: SPARK-34023
> URL: https://issues.apache.org/jira/browse/SPARK-34023
> Project: Spark
> Issue Type: Improvement
> Components: Build, Spark Core
> Affects Versions: 3.2.0
> Reporter: Nick Nezis
> Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
