Hi Günter, David, Let me reply to you both in one email. First of all, thank you for engaging.
Günter: - I fully agree that losing Scala API as officially supported in Flink would be very unfortunate. Future of Scala is interesting and will bring more benefits to Flink users. Just to remind everyone, Flink Scala users can't only use Java API, they require additional hop to serialize Scala objects. This is one of the reasons why Flink still has Scala API (2.11) and why a few more 3-rd party wrappers appeared to support newer versions of Scala when it became possible. David: Let me address your concerns. 1. It is indeed not a very active project. This is exactly the reason, I want to save https://github.com/findify/flink-scala-api from dying, because it is quite a good library to work with. Our Idea is to hit two targets: get a newer/official Scala API for Flink and do not let the 3rd-party (currently) library to sink. I use this library for daily work. 2. It works for Flink 1.15, support of Flink 1.16. requires just publishing a new version. I guess it is a one line change in the build.sbt file. Will see if more changes would be needed. I think the nature of changes will be similar like in StateFun, i.e. adopt to breaking changes of public methods and/or switch from deprecated methods to newer alternatives. Migrating further should not be a problem. Again, Scala API is supposed to be a thin wrapper on top of Java API, so that it is not labour-intensive 3. That single person left Findify (Roman) and they did not pay much attention to it. Actually, there is no other better alternative for Scala wrapper currently. This single committer is now with us and ready to maintain it in open source. The best situation to be :-) 4/5. Yes, same as #1. You can see some PRs in the queue from a Scala bot, but Findify does not merge them. The library is so small and covers most of the needs that additional changes are not yet identified/needed. I agree this could be a sign that few people are using it. I have no idea which companies or users use this library. Is it really important to know? I just want to provide proper substitution to guarantee Scala is used further with Flink. I know that the official Scala API was used or still used by world known enterprises. Thank you for your suggestion. I have included dev ML in the original email. Let me try to come up with a more detailed plan. Among maintainers you will get me, Roman (main dev of https://github.com/findify/flink-scala-api) and maybe Günter. What is the downside or loss if we import this library into the Flink and in a few years nobody will use it? I guess we'll just depreciate it? I just propose my free time to maintain that. As per Roman, required work to maintain the library is very simple. Best regards, Alexey On Sun, Apr 16, 2023 at 11:46 AM David Morávek <d...@apache.org> wrote: > cc dev@f.a.o > > On Sun, Apr 16, 2023 at 11:42 AM David Morávek <d...@apache.org> wrote: > >> Hi Alexey, >> >> I'm a bit skeptical because, looking at the project, I see a couple of >> red flags: >> >> - The project is inactive. The last release and commit are both from the >> last May. >> - The project has not been adapted for the last two Flink versions, which >> signals a lack of users. >> - All commits are by a single person, which could mean that there is no >> community around the project. >> - There was no external contribution (except the Scala bot). >> - There is no fork of the project (except the Scala bot). >> >> > As I know, FIndify does not want or cannot maintain this library. >> >> Who are the users of the library? I'd assume Findify no longer uses it if >> they're abandoning it. >> >> > which would be similar to the StateFun >> >> We're currently dealing with a lack of maintainers for StateFun, so we >> should have a solid building ground around the project to avoid the same >> issue. >> >> >> I think there is value in having a modern Scala API, but we should have a >> bigger plan to address the future of Flink Scala APIs than importing an >> unmaintained library and calling it a day. I suggest starting a thread on >> the dev ML and concluding the overall plan first. >> >> Best, >> D. >> >> On Sun, Apr 16, 2023 at 10:48 AM guenterh.lists < >> guenterh.li...@bluewin.ch> wrote: >> >>> Hello Alexey >>> >>> Thank you for your initiative and your suggestion! >>> >>> I can only fully support the following statements in your email: >>> >>> >Taking into account my Scala experience for the last 8 years, I >>> predict these wrappers will eventually be abandoned, unless such a Scala >>> library is a part of some bigger community like ASF. >>> >Also, non-official Scala API will lead people to play safe and choose >>> Java API only, even if they didn't want that at the beginning. >>> >>> Second sentence is my current state. >>> >>> From my point of view it would be very unfortunate if the Flink project >>> would lose the Scala API and thus the integration of concise, flexible >>> and future-oriented language constructs of the Scala language (and >>> further development of version 3). >>> >>> Documentation of the API is essential. I would be interested to support >>> this efforts. >>> >>> Best wishes >>> >>> Günter >>> >>> >>> On 13.04.23 15:39, Alexey Novakov via user wrote: >>> > Hello Flink PMCs and Flink Scala Users, >>> > >>> > I would like to propose an idea to take the 3rd party Scala API >>> > findify/flink-scala-api <https://github.com/findify/flink-scala-api> >>> > project into the Apache Flink organization. >>> > >>> > *Motivation * >>> > >>> > The Scala-free Flink idea was finally implemented by the 1.15 release >>> and >>> > allowed Flink users to bring their own Scala version and use it via the >>> > Flink Java API. See blog-post here: Scala Free in One Fifteen >>> > <https://flink.apache.org/2022/02/22/scala-free-in-one-fifteen/>. >>> Also, >>> > existing Flink Scala API will be deprecated, because it is too hard to >>> > upgrade it to Scala 2.13 or 3. >>> > >>> > Taking into account my Scala experience for the last 8 years, I predict >>> > these wrappers will eventually be abandoned, unless such a Scala >>> library is >>> > a part of some bigger community like ASF. >>> > Also, non-official Scala API will lead people to play safe and choose >>> Java >>> > API only, even if they did want that at the beginning. >>> > >>> > https://github.com/findify/flink-scala-api has already advanced and >>> > implemented Scala support for 2.13 and 3 versions on top of Flink Java >>> API. >>> > As I know, FIndify does not want or does not have a capacity to >>> maintain >>> > this library. I propose to fork this great library and create a new >>> Flink >>> > project with its own version and build process (SBT, not Maven), which >>> > would be similar to the StateFun or FlinkML projects. >>> > >>> > *Proposal * >>> > >>> > 1. Create a fork of findify/flink-scala-api and host in Apache Flink >>> Git >>> > space (PMCs please advise). >>> > 2. I and Roman >>> > < >>> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=rgrebennikov >>> > >>> > would >>> > be willing to maintain this library in future for the next several >>> years. >>> > Further, we believe it will live on its own. >>> > 3. Flink Docs: PMCs, we need your guidelines here. One way I see is to >>> > create new documentation in a similar way as StateFun docs. >>> Alternatively, >>> > we could just fix existing Flink Scala code examples to make sure they >>> work >>> > with the new wrapper. In any case, I see docs will be upgraded/fixed >>> > gradually. >>> > >>> > I hope you will find this idea interesting and worth going forward. >>> > >>> > P.S. The irony here is that findify/flink-scala-api was also a fork of >>> > Flink Scala-API some time ago, so we have a chance to close the loop >>> :-) >>> > >>> > Best regards. >>> > Alexey >>> > >>> -- >>> Günter Hipler >>> https://openbiblio.social/@vog61 >>> https://twitter.com/vog61 >>> >>>
