[ https://issues.apache.org/jira/browse/SPARK-34023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17259999#comment-17259999 ]
Nick Nezis commented on SPARK-34023: ------------------------------------ In the same Google Groups [post|https://groups.google.com/g/kryo-users/c/sBZ10dwrwFQ] mentioned above, the author gave a good write-up on some of the risks and why a project might not want to upgrade. {quote} I'm sure everyone's first question will be about what happens to data serialized in an older Kryo version. Supporting that is noble for a minor version increment, but the changes above are too extensive for that to work. It would be dirty to have settings to disable free optimizations or enable bugs we've fixed, solely to attempt loading previously serialized bytes. I don't want the difficulties of data migration to prevent the library from evolving and I feel an improvement and maintenance pass like this is long overdue. It may not make sense to upgrade your projects to a new major version. Kryo has been stable a long time. While it's nice to have as many projects on the latest as possible, you should not feel obligated to upgrade if the benefits do not make it worthwhile, given the pain involved. New projects of course benefit from using the latest version. Updating Kryo versions when you care about previously serialized bytes has always been hard for all but the most trivial updates. Probably the safest way to do this is to load data with an old version, then write it with the new version. This can be done by using a class loader for the old Kryo classes. Maybe we could provide an example for this. For this to work, obviously the class files must not have not changed, only the Kryo version. Please share your thoughts! Now is the time for us to make any big changes and improvements to both the API and serialized data that wouldn't make sense in a minor version bump. Any places in the API you don't like, are awkward, or confusing? Have any ideas for improving the API? Have any ideas for more efficient serialization? {quote} > Updated to Kryo 5.0.3 > --------------------- > > Key: SPARK-34023 > URL: https://issues.apache.org/jira/browse/SPARK-34023 > Project: Spark > Issue Type: Improvement > Components: Build, Spark Core > Affects Versions: 3.2.0 > Reporter: Nick Nezis > Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org