Hi Sergey, I checked the CI [1] which was executed with Java21, and noticed that the StatefulJobSnapshotMigrationITCase-related tests have passed, which proves what I guessed before, most checkpoints/savepoints should be restored successfully.
I think we shall introduce such snapshot migration tests, which restore snapshots containing scala code. I also create a ticket focused on Java17 [2] [1] https://dev.azure.com/snuyanzin/flink/_build/results?buildId=2620&view=logs&j=0a15d512-44ac-5ba5-97ab-13a5d066c22c&t=9a028d19-6c4b-5a4e-d378-03fca149d0b1 [2] https://issues.apache.org/jira/browse/FLINK-33707 Best Yun Tang ________________________________ From: Sergey Nuyanzin <snuyan...@gmail.com> Sent: Thursday, November 30, 2023 14:41 To: dev@flink.apache.org <dev@flink.apache.org> Subject: Re: [ANNOUNCE] Experimental Java 21 support now available on master Thanks Yun Tang One question to clarify: since the scala version was also bumped for java 17, shouldn't there be a similar task for java 17? On Thu, Nov 30, 2023 at 3:43 AM Yun Tang <myas...@live.com> wrote: > Hi Sergey, > > You can leverage all tests extending SnapshotMigrationTestBase[1] to > verify the logic. I believe all binary _metadata existing in the resources > folder[2] were built by JDK8. > > I also create a ticket FLINK-33699[3] to track this. > > [1] > https://github.com/apache/flink/blob/master/flink-tests/src/test/java/org/apache/flink/test/checkpointing/utils/SnapshotMigrationTestBase.java > [2] > https://github.com/apache/flink/tree/master/flink-tests/src/test/resources > [3] https://issues.apache.org/jira/browse/FLINK-33699 > > Best > Yun Tang > ________________________________ > From: Sergey Nuyanzin <snuyan...@gmail.com> > Sent: Wednesday, November 29, 2023 22:56 > To: dev@flink.apache.org <dev@flink.apache.org> > Subject: Re: [ANNOUNCE] Experimental Java 21 support now available on > master > > thanks for the response > > > >I feel doubt about the conclusion that "don't try to load a savepoint from > a Java 8/11/17 build due to bumping to scala-2.12.18", since the > snapshotted state (operator/keyed state-backend), and most key/value > serializer snapshots are generated by pure-java code. > >The only left part is that the developer uses scala UDF or scala types for > key/value types. However, since all user-facing scala APIs have been > deprecated, I don't think we have so many cases. Maybe we can give > descriptions without such strong suggestions. > > That is the area where I feel I lack the knowledge to answer this > precisely. > My assumption was that statement about Java 21 regarding this should be > similar to Java 17 which is almost same [1] > Sorry for the inaccuracy > Based on your statements I agree that the conclusion could be more relaxed. > > I'm curious whether there are some tests or anything which could clarify > this? > > [1] https://lists.apache.org/thread/mz0m6wqjmqy8htob3w4469pjbg9305do > > On Wed, Nov 29, 2023 at 12:25 PM Yun Tang <myas...@live.com> wrote: > > > Thanks Sergey for the great work. > > > > I feel doubt about the conclusion that "don't try to load a savepoint > from > > a Java 8/11/17 build due to bummping to scala-2.12.18", since the > > snapshotted state (operator/keyed state-backend), and most key/value > > serializer snapshots are generated by pure-java code. The only left part > is > > that the developer uses scala UDF or scala types for key/value types. > > However, since all user-facing scala APIs have been deprecated [1], I > don't > > think we have so many cases. Maybe we can give descriptions without such > > strong suggestions. > > > > Please correct me if I am wrong. > > > > > > [1] https://issues.apache.org/jira/browse/FLINK-29740 > > > > Best > > Yun Tang > > > > ________________________________ > > From: Rui Fan <1996fan...@gmail.com> > > Sent: Wednesday, November 29, 2023 16:43 > > To: dev@flink.apache.org <dev@flink.apache.org> > > Subject: Re: [ANNOUNCE] Experimental Java 21 support now available on > > master > > > > Thanks Sergey for the great work! > > > > Best, > > Rui > > > > On Wed, Nov 29, 2023 at 4:42 PM Leonard Xu <xbjt...@gmail.com> wrote: > > > > > Cool ! > > > > > > Thanks Sergey for the great effort and all involved. > > > > > > > > > Best, > > > Leonard > > > > > > > 2023年11月29日 下午4:31，Swapnal Varma <swapnal...@gmail.com> 写道： > > > > > > > > Congratulations Sergey, and everyone involved! > > > > > > > > Excited to work with and on this! > > > > > > > > Best, > > > > Swapnal > > > > > > > > > > > > On Wed, 29 Nov 2023, 13:58 Sergey Nuyanzin, <snuyan...@gmail.com> > > wrote: > > > > > > > >> The master branch now builds and runs with Java 21 out-of-the-box. > > > >> > > > >> Notes: > > > >> - a nightly cron build was set up. > > > >> - In Java 21 builds, Scala is being bumped to 2.12.18 > > > >> which causes incompatibilities within Flink; > > > >> i.e. don't try to load a savepoint from a Java 8/11/17 build > > > >> - All the tests that are being skipped on Java 11/17 > > > >> are also skipped on Java 21. > > > >> > > > >> Huge shout-out to everyone participating > > > >> in review of my Java 21 related PRs > > > >> > > > >> If you run into any issues, please report it in FLINK-33163 > > > >> <https://issues.apache.org/jira/browse/FLINK-33163> . > > > >> > > > >> -- > > > >> Best regards, > > > >> Sergey > > > >> > > > > > > > > > > > -- > Best regards, > Sergey > -- Best regards, Sergey
