Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Márton Balassi]

Hi Martjin,

After some more careful consideration I am in favor of dropping the Scala
API support in with Flink 2.0 given that we add Java 17 support earlier or
latest at the same time.

Best,
Marton

On Thu, Oct 13, 2022 at 12:01 PM Chesnay Schepler 
wrote:

> Support for records has not been investigated yet. We're still at the
> stage of getting things to run at all on Java 17.
>
> It _may_ be possible, it _may_ not be.
>
> On 13/10/2022 07:39, Salva Alcántara wrote:
>
> Hi Martijn,
>
> Maybe a bit of an off-topic, but regarding Java 17 support, will it be
> possible to replace POJOs with Java records in existing applications?
>
> In a project I maintain we use Lombok a lot, but with Java records we
> would probably stop using it (or significantly reduce its usage).
>
> Will there be a way to promote existing POJOs (either written "manually"
> or using Lombok) to Java records without breaking serialization? (assuming
> that those POJOs are used as immutable values, e.g., setters are never
> used).
>
> Regards,
>
> Salva
>
> On Wed, Oct 12, 2022 at 9:11 PM Martijn Visser 
> wrote:
>
>> Hi everyone,
>>
>> Thanks again for all your feedback. It's very much appreciated.
>>
>> My overall feeling is that people are not opposed to the FLIP. There is
>> demand for adding Java 17 support before dropping the Scala APIs. Given
>> that the proposal for actually dropping the Scala APIs would only happen
>> with a Flink 2.0 and Java 17 support would either happen in a new minor
>> version or a new major version (I haven't seen a FLIP or discussion being
>> opened adding Java 17 support, only on deprecating Java 8), Java 17 support
>> would either be there earlier (in a new minor version) or at the same time
>> (with Flink 2.0) when the Scala APIs would be dropped.
>>
>> If there are no more discussion topics, I would move this FLIP to a vote
>> at the beginning of next week.
>>
>> Best regards,
>>
>> Martijn
>>
>> On Sun, Oct 9, 2022 at 10:36 AM guenterh.lists 
>> wrote:
>>
>>> Hi Martijn
>>>
>>> I do not maintain a large production application based on Flink, so it
>>> would not be a problem for me to convert existing implementations to
>>> whatever API.
>>>
>>> I am working in the area of cultural heritage, which is mainly about the
>>> processing of structured (meta)-data (scientific libraries, archives and
>>> museums)
>>> My impression: People without much background/experience with Java
>>> implementations find it easier to get into the functional mindset as
>>> supported in Scala. That's why I think it would be very unfortunate if the
>>> use of Scala in Flink becomes more and more limited or neglected.
>>>
>>> I think using the Java API in Scala is a possible way also in my
>>> environment.
>>>
>>> In the last weeks I tried to port the examples from the "Flink Course"
>>> of Daniel Ciorcilan (https://rockthejvm.com/p/flink - he mainly offers
>>> Scala courses), which are exclusively based on the native Scala API, to the
>>> Java API. This has worked without any problems as far as I can see. So far
>>> I haven't tried any examples based on the Table API or streaming workflows
>>> in batch mode (which would be important for our environment).
>>>
>>> My main trouble: So far I don't know enough about the limitations of
>>> using the Java API in a Scala implementation and what that means. My
>>> current understanding: the limitation is mainly in deriving the type
>>> information in generic APIs with Scala types. For me it would be very
>>> significant and helpful if there would be more information, descriptions
>>> and examples about this topic.
>>>
>>> So far unfortunately I had too little time to deal with a wrapper like
>>> flink-scala-api (https://github.com/findify/flink-scala-api ) and the
>>> current alternative is probably going to be deprecated in the future (
>>> https://github.com/ariskk/flink4s/issues/17#issuecomment-1125806808 )
>>>
>>> Günter
>>>
>>>
>>> On 04.10.22 13:58, Martijn Visser wrote:
>>>
>>> Hi Marton,
>>>
>>> You're making a good point, I originally wanted to include already the
>>> User mailing list to get their feedback but forgot to do so. I'll do some
>>> more outreach via other channels as well.
>>>
>>> @Users of Flink, I've made a proposal to deprecate and remove Scala API
>>> support in a future version of Flink. Your feedback on this topic is very
>>> much appreciated.
>>>
>>> Regarding the large Scala codebase for Flink, a potential alternative
>>> could be to have a wrapper for all Java APIs that makes them available as
>>> Scala APIs. However, this still requires Scala maintainers and I don't
>>> think that we currently have those in our community. The easiest solution
>>> for them would be to use the Java APIs directly. Yes it would involve work,
>>> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
>>> there's still time for that :)
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
>>> wrote:
>>>
 Hi 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Chesnay Schepler]

Support for records has not been investigated yet. We're still at the 
stage of getting things to run at all on Java 17.


It _may_ be possible, it _may_ not be.

On 13/10/2022 07:39, Salva Alcántara wrote:

Hi Martijn,

Maybe a bit of an off-topic, but regarding Java 17 support, will it be 
possible to replace POJOs with Java records in existing applications?


In a project I maintain we use Lombok a lot, but with Java records we 
would probably stop using it (or significantly reduce its usage).


Will there be a way to promote existing POJOs (either written 
"manually" or using Lombok) to Java records without breaking 
serialization? (assuming that those POJOs are used as immutable 
values, e.g., setters are never used).


Regards,

Salva

On Wed, Oct 12, 2022 at 9:11 PM Martijn Visser 
 wrote:


Hi everyone,

Thanks again for all your feedback. It's very much appreciated.

My overall feeling is that people are not opposed to the FLIP.
There is demand for adding Java 17 support before dropping the
Scala APIs. Given that the proposal for actually dropping the
Scala APIs would only happen with a Flink 2.0 and Java 17 support
would either happen in a new minor version or a new major version
(I haven't seen a FLIP or discussion being opened adding Java 17
support, only on deprecating Java 8), Java 17 support would either
be there earlier (in a new minor version) or at the same time
(with Flink 2.0) when the Scala APIs would be dropped.

If there are no more discussion topics, I would move this FLIP to
a vote at the beginning of next week.

Best regards,

Martijn

On Sun, Oct 9, 2022 at 10:36 AM guenterh.lists
 wrote:

Hi Martijn

I do not maintain a large production application based on
Flink, so it would not be a problem for me to convert existing
implementations to whatever API.

I am working in the area of cultural heritage, which is mainly
about the processing of structured (meta)-data (scientific
libraries, archives and museums)
My impression: People without much background/experience with
Java implementations find it easier to get into the functional
mindset as supported in Scala. That's why I think it would be
very unfortunate if the use of Scala in Flink becomes more and
more limited or neglected.

I think using the Java API in Scala is a possible way also in
my environment.

In the last weeks I tried to port the examples from the "Flink
Course" of Daniel Ciorcilan (https://rockthejvm.com/p/flink -
he mainly offers Scala courses), which are exclusively based
on the native Scala API, to the Java API. This has worked
without any problems as far as I can see. So far I haven't
tried any examples based on the Table API or streaming
workflows in batch mode (which would be important for our
environment).

My main trouble: So far I don't know enough about the
limitations of using the Java API in a Scala implementation
and what that means. My current understanding: the limitation
is mainly in deriving the type information in generic APIs
with Scala types. For me it would be very significant and
helpful if there would be more information, descriptions and
examples about this topic.

So far unfortunately I had too little time to deal with a
wrapper like flink-scala-api
(https://github.com/findify/flink-scala-api ) and the current
alternative is probably going to be deprecated in the future
(https://github.com/ariskk/flink4s/issues/17#issuecomment-1125806808
)

Günter


On 04.10.22 13:58, Martijn Visser wrote:

Hi Marton,

You're making a good point, I originally wanted to include
already the User mailing list to get their feedback but
forgot to do so. I'll do some more outreach via other
channels as well.

@Users of Flink, I've made a proposal to deprecate and remove
Scala API support in a future version of Flink. Your feedback
on this topic is very much appreciated.

Regarding the large Scala codebase for Flink, a potential
alternative could be to have a wrapper for all Java APIs that
makes them available as Scala APIs. However, this still
requires Scala maintainers and I don't think that we
currently have those in our community. The easiest solution
for them would be to use the Java APIs directly. Yes it would
involve work, but we won't actually be able to remove the
Scala APIs until Flink 2.0 so there's still time for that :)

Best regards,

Martijn

On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi
 wrote:

Hi Martjin,

Thanks for compiling the FLIP. I agree with the 

RE: Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Salva Alcántara]

Yep, I agree with that, but I guess losing the first-class citizen status
within Flink will make many companies currently in doubt finally adopt
Java. For non-FP shops or companies without a strong command of Scala,
using Java will simplify things in general and avoid some unnecessary pains
(hiring & training). Although possible, IMO using Scala without the
required serialization support for its specific types nor a nice API
wrapper feels a bit artificial and awkward, especially once Java 17 is
supported, and I don't think it is worth it for most cases.

Salva

On 2022/10/05 19:26:49 David Anderson wrote:
> I want to clarify one point here, which is that modifying jobs written in
> Scala to use Flink's Java API does not require porting them to Java. I can
> readily understand why folks using Scala might rather use Java 17 than
Java
> 11, but sticking to Scala will remain an option even if Flink's Scala API
> goes away.
>
> For more on this, see [1] and some of the examples it points to, such as
> those in [2].
>
> [1] https://flink.apache.org/2022/02/22/scala-free.html
> [2] https://github.com/sjwiesman/flink-scala-3
>
> On Tue, Oct 4, 2022 at 6:16 PM Clayton Wohl  wrote:
>
> > +1
> >
> > At my employer, we maintain several Flink jobs in Scala. We've been
> > writing newer jobs in Java, and we'd be fine with porting our Scala jobs
> > over to the Java API.
> >
> > I'd like to request Java 17 support. Specifically, Java records is a
> > feature our Flink code would use a lot of and make the Java syntax much
> > nicer.
> >
>

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Salva Alcántara]

Hi Martijn,

Maybe a bit of an off-topic, but regarding Java 17 support, will it be
possible to replace POJOs with Java records in existing applications?

In a project I maintain we use Lombok a lot, but with Java records we would
probably stop using it (or significantly reduce its usage).

Will there be a way to promote existing POJOs (either written "manually" or
using Lombok) to Java records without breaking serialization? (assuming
that those POJOs are used as immutable values, e.g., setters are never
used).

Regards,

Salva

On Wed, Oct 12, 2022 at 9:11 PM Martijn Visser 
wrote:

> Hi everyone,
>
> Thanks again for all your feedback. It's very much appreciated.
>
> My overall feeling is that people are not opposed to the FLIP. There is
> demand for adding Java 17 support before dropping the Scala APIs. Given
> that the proposal for actually dropping the Scala APIs would only happen
> with a Flink 2.0 and Java 17 support would either happen in a new minor
> version or a new major version (I haven't seen a FLIP or discussion being
> opened adding Java 17 support, only on deprecating Java 8), Java 17 support
> would either be there earlier (in a new minor version) or at the same time
> (with Flink 2.0) when the Scala APIs would be dropped.
>
> If there are no more discussion topics, I would move this FLIP to a vote
> at the beginning of next week.
>
> Best regards,
>
> Martijn
>
> On Sun, Oct 9, 2022 at 10:36 AM guenterh.lists 
> wrote:
>
>> Hi Martijn
>>
>> I do not maintain a large production application based on Flink, so it
>> would not be a problem for me to convert existing implementations to
>> whatever API.
>>
>> I am working in the area of cultural heritage, which is mainly about the
>> processing of structured (meta)-data (scientific libraries, archives and
>> museums)
>> My impression: People without much background/experience with Java
>> implementations find it easier to get into the functional mindset as
>> supported in Scala. That's why I think it would be very unfortunate if the
>> use of Scala in Flink becomes more and more limited or neglected.
>>
>> I think using the Java API in Scala is a possible way also in my
>> environment.
>>
>> In the last weeks I tried to port the examples from the "Flink Course" of
>> Daniel Ciorcilan (https://rockthejvm.com/p/flink - he mainly offers
>> Scala courses), which are exclusively based on the native Scala API, to the
>> Java API. This has worked without any problems as far as I can see. So far
>> I haven't tried any examples based on the Table API or streaming workflows
>> in batch mode (which would be important for our environment).
>>
>> My main trouble: So far I don't know enough about the limitations of
>> using the Java API in a Scala implementation and what that means. My
>> current understanding: the limitation is mainly in deriving the type
>> information in generic APIs with Scala types. For me it would be very
>> significant and helpful if there would be more information, descriptions
>> and examples about this topic.
>>
>> So far unfortunately I had too little time to deal with a wrapper like
>> flink-scala-api (https://github.com/findify/flink-scala-api ) and the
>> current alternative is probably going to be deprecated in the future (
>> https://github.com/ariskk/flink4s/issues/17#issuecomment-1125806808 )
>>
>> Günter
>>
>>
>> On 04.10.22 13:58, Martijn Visser wrote:
>>
>> Hi Marton,
>>
>> You're making a good point, I originally wanted to include already the
>> User mailing list to get their feedback but forgot to do so. I'll do some
>> more outreach via other channels as well.
>>
>> @Users of Flink, I've made a proposal to deprecate and remove Scala API
>> support in a future version of Flink. Your feedback on this topic is very
>> much appreciated.
>>
>> Regarding the large Scala codebase for Flink, a potential alternative
>> could be to have a wrapper for all Java APIs that makes them available as
>> Scala APIs. However, this still requires Scala maintainers and I don't
>> think that we currently have those in our community. The easiest solution
>> for them would be to use the Java APIs directly. Yes it would involve work,
>> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
>> there's still time for that :)
>>
>> Best regards,
>>
>> Martijn
>>
>> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
>> wrote:
>>
>>> Hi Martjin,
>>>
>>> Thanks for compiling the FLIP. I agree with the sentiment that Scala
>>> poses
>>> considerable maintenance overhead and key improvements (like 2.13 or
>>> 2.12.8
>>> supports) are hanging stale. With that said before we make this move we
>>> should attempt to understand the userbase affected.
>>> A quick Slack and user mailing list search does return quite a bit of
>>> results for scala (admittedly a cursory look at them suggest that many of
>>> them have to do with missing features in Scala that exist in Java or
>>> Scala
>>> versions). I would love to see some polls on this 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Martijn Visser]

Hi everyone,

Thanks again for all your feedback. It's very much appreciated.

My overall feeling is that people are not opposed to the FLIP. There is
demand for adding Java 17 support before dropping the Scala APIs. Given
that the proposal for actually dropping the Scala APIs would only happen
with a Flink 2.0 and Java 17 support would either happen in a new minor
version or a new major version (I haven't seen a FLIP or discussion being
opened adding Java 17 support, only on deprecating Java 8), Java 17 support
would either be there earlier (in a new minor version) or at the same time
(with Flink 2.0) when the Scala APIs would be dropped.

If there are no more discussion topics, I would move this FLIP to a vote at
the beginning of next week.

Best regards,

Martijn

On Sun, Oct 9, 2022 at 10:36 AM guenterh.lists 
wrote:

> Hi Martijn
>
> I do not maintain a large production application based on Flink, so it
> would not be a problem for me to convert existing implementations to
> whatever API.
>
> I am working in the area of cultural heritage, which is mainly about the
> processing of structured (meta)-data (scientific libraries, archives and
> museums)
> My impression: People without much background/experience with Java
> implementations find it easier to get into the functional mindset as
> supported in Scala. That's why I think it would be very unfortunate if the
> use of Scala in Flink becomes more and more limited or neglected.
>
> I think using the Java API in Scala is a possible way also in my
> environment.
>
> In the last weeks I tried to port the examples from the "Flink Course" of
> Daniel Ciorcilan (https://rockthejvm.com/p/flink - he mainly offers Scala
> courses), which are exclusively based on the native Scala API, to the Java
> API. This has worked without any problems as far as I can see. So far I
> haven't tried any examples based on the Table API or streaming workflows in
> batch mode (which would be important for our environment).
>
> My main trouble: So far I don't know enough about the limitations of using
> the Java API in a Scala implementation and what that means. My current
> understanding: the limitation is mainly in deriving the type information in
> generic APIs with Scala types. For me it would be very significant and
> helpful if there would be more information, descriptions and examples about
> this topic.
>
> So far unfortunately I had too little time to deal with a wrapper like
> flink-scala-api (https://github.com/findify/flink-scala-api ) and the
> current alternative is probably going to be deprecated in the future (
> https://github.com/ariskk/flink4s/issues/17#issuecomment-1125806808 )
>
> Günter
>
>
> On 04.10.22 13:58, Martijn Visser wrote:
>
> Hi Marton,
>
> You're making a good point, I originally wanted to include already the
> User mailing list to get their feedback but forgot to do so. I'll do some
> more outreach via other channels as well.
>
> @Users of Flink, I've made a proposal to deprecate and remove Scala API
> support in a future version of Flink. Your feedback on this topic is very
> much appreciated.
>
> Regarding the large Scala codebase for Flink, a potential alternative
> could be to have a wrapper for all Java APIs that makes them available as
> Scala APIs. However, this still requires Scala maintainers and I don't
> think that we currently have those in our community. The easiest solution
> for them would be to use the Java APIs directly. Yes it would involve work,
> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
> there's still time for that :)
>
> Best regards,
>
> Martijn
>
> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
> wrote:
>
>> Hi Martjin,
>>
>> Thanks for compiling the FLIP. I agree with the sentiment that Scala poses
>> considerable maintenance overhead and key improvements (like 2.13 or
>> 2.12.8
>> supports) are hanging stale. With that said before we make this move we
>> should attempt to understand the userbase affected.
>> A quick Slack and user mailing list search does return quite a bit of
>> results for scala (admittedly a cursory look at them suggest that many of
>> them have to do with missing features in Scala that exist in Java or Scala
>> versions). I would love to see some polls on this topic, we could also use
>> the Flink twitter handle to ask the community about this.
>>
>> I am aware of users having large existing Scala codebases for Flink. This
>> move would pose a very large effort on them, as they would need to rewrite
>> much of their existing code. What are the alternatives in your opinion,
>> Martjin?
>>
>> On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser 
>> wrote:
>>
>> > Hi everyone,
>> >
>> > I would like to open a discussion thread on FLIP-265 Deprecate and
>> remove
>> > Scala API support. Please take a look at
>> >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
>> > and provide your feedback.
>> >
>> > Best regards,
>> >

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[guenterh.lists]

Hi Martijn

I do not maintain a large production application based on Flink, so it 
would not be a problem for me to convert existing implementations to 
whatever API.


I am working in the area of cultural heritage, which is mainly about the 
processing of structured (meta)-data (scientific libraries, archives and 
museums)
My impression: People without much background/experience with Java 
implementations find it easier to get into the functional mindset as 
supported in Scala. That's why I think it would be very unfortunate if 
the use of Scala in Flink becomes more and more limited or neglected.


I think using the Java API in Scala is a possible way also in my 
environment.


In the last weeks I tried to port the examples from the "Flink Course" 
of Daniel Ciorcilan (https://rockthejvm.com/p/flink - he mainly offers 
Scala courses), which are exclusively based on the native Scala API, to 
the Java API. This has worked without any problems as far as I can see. 
So far I haven't tried any examples based on the Table API or streaming 
workflows in batch mode (which would be important for our environment).


My main trouble: So far I don't know enough about the limitations of 
using the Java API in a Scala implementation and what that means. My 
current understanding: the limitation is mainly in deriving the type 
information in generic APIs with Scala types. For me it would be very 
significant and helpful if there would be more information, descriptions 
and examples about this topic.


So far unfortunately I had too little time to deal with a wrapper like 
flink-scala-api (https://github.com/findify/flink-scala-api ) and the 
current alternative is probably going to be deprecated in the future 
(https://github.com/ariskk/flink4s/issues/17#issuecomment-1125806808 )


Günter


On 04.10.22 13:58, Martijn Visser wrote:

Hi Marton,

You're making a good point, I originally wanted to include already the 
User mailing list to get their feedback but forgot to do so. I'll do 
some more outreach via other channels as well.


@Users of Flink, I've made a proposal to deprecate and remove Scala 
API support in a future version of Flink. Your feedback on this topic 
is very much appreciated.


Regarding the large Scala codebase for Flink, a potential alternative 
could be to have a wrapper for all Java APIs that makes them available 
as Scala APIs. However, this still requires Scala maintainers and I 
don't think that we currently have those in our community. The easiest 
solution for them would be to use the Java APIs directly. Yes it would 
involve work, but we won't actually be able to remove the Scala APIs 
until Flink 2.0 so there's still time for that :)


Best regards,

Martijn

On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
 wrote:


Hi Martjin,

Thanks for compiling the FLIP. I agree with the sentiment that
Scala poses
considerable maintenance overhead and key improvements (like 2.13
or 2.12.8
supports) are hanging stale. With that said before we make this
move we
should attempt to understand the userbase affected.
A quick Slack and user mailing list search does return quite a bit of
results for scala (admittedly a cursory look at them suggest that
many of
them have to do with missing features in Scala that exist in Java
or Scala
versions). I would love to see some polls on this topic, we could
also use
the Flink twitter handle to ask the community about this.

I am aware of users having large existing Scala codebases for
Flink. This
move would pose a very large effort on them, as they would need to
rewrite
much of their existing code. What are the alternatives in your
opinion,
Martjin?

On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser

wrote:

> Hi everyone,
>
> I would like to open a discussion thread on FLIP-265 Deprecate
and remove
> Scala API support. Please take a look at
>
>

https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
> and provide your feedback.
>
> Best regards,
>
> Martijn
> https://twitter.com/MartijnVisser82
> https://github.com/MartijnVisser
>


--
Günter Hipler
University library Leipzig

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Martijn Visser]

I also noticed that we two replies in a separate thread on the User mailing
list, which can be found at
https://lists.apache.org/thread/m5ntl3cj81wg7frbfqg9v75c7hqnxtls.

I've included Clayton and David in this email, to at least centralize the
conversation once more :)



On Wed, Oct 5, 2022 at 11:36 AM Martijn Visser 
wrote:

> @Maciek
>
> I saw that I missed replying to your question:
>
> > Could you please remind what was the conclusion of discussion on
> upgrading Scala to 2.12.15/16?
> > https://lists.apache.org/thread/hwksnsqyg7n3djymo7m1s7loymxxbc3t - I
> couldn't find any follow-up vote?
>
> There is a vote thread, but that never got enough votes. See
> https://lists.apache.org/thread/l93l5qqr5n2oty3r2bjsz3ks3tjf1655
>
> > If it's acceptable to break binary compatibility by such an upgrade,
> then upgrading to JDK17 before 2.0 will be doable?
>
> I'm not sure, because I don't think a discussion and vote has been made
> yet if upgrading JDK17 can/will be done in a Flink 1.0 release or if it
> requires a 2.0 release. It was mentioned in the original discussion thread
> on dropping Java 8 support within 2/3 releases, but if I recall correctly
> there was no discussion yet on when Java 17 support would be added [1].
>
> Best regards,
>
> Martijn
>
> [1] https://lists.apache.org/thread/0fwo7nwzy51gck4vxhyfnbnttd4jycpx
>
> On Wed, Oct 5, 2022 at 6:58 AM Gaël Renoux 
> wrote:
>
>> > I'm curious what target Scala versions people are currently interested
>> in.
>> > I would've expected that everyone wants to migrate to Scala 3, for
>> which several wrapper projects around Flink already exist
>>
>> The Scala 3 tooling is still subpar (we're using IntelliJ), so I'm not
>> sure I would move my team to Scala 3 right now (I'm currently toying with
>> it on a personal project). In addition, moving to Scala 3 is not completely
>> free - you have to do some rewrites, and developers will need some
>> adaptation time. Scala 2.13 is another thing entirely, we've wanted to
>> migrate for a long while.
>>
>> On Wed, Oct 5, 2022 at 12:53 PM Chesnay Schepler 
>> wrote:
>>
>>> > It's possible that for the sake of the Scala API, we would
>>> occasionally require some changes in the Java API. As long as those changes
>>> are not detrimental to Java users, they should be considered.
>>>
>>> That's exactly the model we're trying to get to. Don't fix
>>> scala-specific issues with scala code, but rather on the Java side as much
>>> as possible which could also benefit other JVM languages (e.g., Kotlin).
>>>
>>> > A question regarding the Flink wrapper: would it be possible to keep
>>> it under the Flink project's umbrella? Or does it need to be a completely
>>> separate structure? I'm not aware of the full organizational implications
>>> of this, I'm afraid.
>>>
>>> Technically it can be under the Flink umbrella, but then Flink would
>>> still be (at least for a while) be the bottleneck because we'd have to
>>> review any changes coming in.
>>> That would only improve once several new committers were added to take
>>> care of this project.
>>> (In the end you'd just split Flink and the Scala API _codebases_, but
>>> achieve little else)
>>>
>>> > And if that is what it takes to move beyond Scala 2.12.7… This has
>>> been a big pain point for us.
>>>
>>> I'm curious what target Scala versions people are currently interested
>>> in.
>>> I would've expected that everyone wants to migrate to Scala 3, for which
>>> several wrapper projects around Flink already exist.
>>>
>>> On 05/10/2022 12:35, Gaël Renoux wrote:
>>>
>>> Hello everyone,
>>>
>>> I've already answered a bit on Twitter, I'll develop my thoughts a bit
>>> here. For context, my company (DataDome) has a significant codebase on
>>> Scala Flink (around 110K LOC), having been using it since 2017. I myself am
>>> an enthusiastic Scala developer (I don't think I'd like moving back to
>>> Java)
>>>
>>> Given that, I think separating the Scala support from Flink is actually
>>> a good move long term. We would then have a full-Java Flink, and a separate
>>> Scala wrapper. It would help a lot in solving the skills issue: Flink
>>> maintainers would no longer need to be fluent in Scala, and maintainers of
>>> the Scala wrapper would not need a deep knowledge of Flink's inner
>>> workings, just the API would be sufficient. And if that is what it takes to
>>> move beyond Scala 2.12.7… This has been a big pain point for us.
>>>
>>> I'm not too worried about finding contributors for the Scala wrapper.
>>> Within my company, we have developed additional wrappers and extension
>>> methods (for parts where we felt the Flink Scala API was insufficient), and
>>> we've been looking at ways we could contribute back. What held us back was
>>> our lack of knowledge of the full Flink environment (we're only using the
>>> Scala Datastream API). I don't think we're the only ones in that situation.
>>> One major point, though, is that Flink developers would not be completely
>>> rid of us ;-) 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[David Anderson]

I want to clarify one point here, which is that modifying jobs written in
Scala to use Flink's Java API does not require porting them to Java. I can
readily understand why folks using Scala might rather use Java 17 than Java
11, but sticking to Scala will remain an option even if Flink's Scala API
goes away.

For more on this, see [1] and some of the examples it points to, such as
those in [2].

[1] https://flink.apache.org/2022/02/22/scala-free.html
[2] https://github.com/sjwiesman/flink-scala-3

On Tue, Oct 4, 2022 at 6:16 PM Clayton Wohl  wrote:

> +1
>
> At my employer, we maintain several Flink jobs in Scala. We've been
> writing newer jobs in Java, and we'd be fine with porting our Scala jobs
> over to the Java API.
>
> I'd like to request Java 17 support. Specifically, Java records is a
> feature our Flink code would use a lot of and make the Java syntax much
> nicer.
>

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Martijn Visser]

@Maciek

I saw that I missed replying to your question:

> Could you please remind what was the conclusion of discussion on
upgrading Scala to 2.12.15/16?
> https://lists.apache.org/thread/hwksnsqyg7n3djymo7m1s7loymxxbc3t - I
couldn't find any follow-up vote?

There is a vote thread, but that never got enough votes. See
https://lists.apache.org/thread/l93l5qqr5n2oty3r2bjsz3ks3tjf1655

> If it's acceptable to break binary compatibility by such an upgrade, then
upgrading to JDK17 before 2.0 will be doable?

I'm not sure, because I don't think a discussion and vote has been made yet
if upgrading JDK17 can/will be done in a Flink 1.0 release or if it
requires a 2.0 release. It was mentioned in the original discussion thread
on dropping Java 8 support within 2/3 releases, but if I recall correctly
there was no discussion yet on when Java 17 support would be added [1].

Best regards,

Martijn

[1] https://lists.apache.org/thread/0fwo7nwzy51gck4vxhyfnbnttd4jycpx

On Wed, Oct 5, 2022 at 6:58 AM Gaël Renoux  wrote:

> > I'm curious what target Scala versions people are currently interested
> in.
> > I would've expected that everyone wants to migrate to Scala 3, for which
> several wrapper projects around Flink already exist
>
> The Scala 3 tooling is still subpar (we're using IntelliJ), so I'm not
> sure I would move my team to Scala 3 right now (I'm currently toying with
> it on a personal project). In addition, moving to Scala 3 is not completely
> free - you have to do some rewrites, and developers will need some
> adaptation time. Scala 2.13 is another thing entirely, we've wanted to
> migrate for a long while.
>
> On Wed, Oct 5, 2022 at 12:53 PM Chesnay Schepler 
> wrote:
>
>> > It's possible that for the sake of the Scala API, we would occasionally
>> require some changes in the Java API. As long as those changes are not
>> detrimental to Java users, they should be considered.
>>
>> That's exactly the model we're trying to get to. Don't fix scala-specific
>> issues with scala code, but rather on the Java side as much as possible
>> which could also benefit other JVM languages (e.g., Kotlin).
>>
>> > A question regarding the Flink wrapper: would it be possible to keep it
>> under the Flink project's umbrella? Or does it need to be a completely
>> separate structure? I'm not aware of the full organizational implications
>> of this, I'm afraid.
>>
>> Technically it can be under the Flink umbrella, but then Flink would
>> still be (at least for a while) be the bottleneck because we'd have to
>> review any changes coming in.
>> That would only improve once several new committers were added to take
>> care of this project.
>> (In the end you'd just split Flink and the Scala API _codebases_, but
>> achieve little else)
>>
>> > And if that is what it takes to move beyond Scala 2.12.7… This has been
>> a big pain point for us.
>>
>> I'm curious what target Scala versions people are currently interested in.
>> I would've expected that everyone wants to migrate to Scala 3, for which
>> several wrapper projects around Flink already exist.
>>
>> On 05/10/2022 12:35, Gaël Renoux wrote:
>>
>> Hello everyone,
>>
>> I've already answered a bit on Twitter, I'll develop my thoughts a bit
>> here. For context, my company (DataDome) has a significant codebase on
>> Scala Flink (around 110K LOC), having been using it since 2017. I myself am
>> an enthusiastic Scala developer (I don't think I'd like moving back to
>> Java)
>>
>> Given that, I think separating the Scala support from Flink is actually a
>> good move long term. We would then have a full-Java Flink, and a separate
>> Scala wrapper. It would help a lot in solving the skills issue: Flink
>> maintainers would no longer need to be fluent in Scala, and maintainers of
>> the Scala wrapper would not need a deep knowledge of Flink's inner
>> workings, just the API would be sufficient. And if that is what it takes to
>> move beyond Scala 2.12.7… This has been a big pain point for us.
>>
>> I'm not too worried about finding contributors for the Scala wrapper.
>> Within my company, we have developed additional wrappers and extension
>> methods (for parts where we felt the Flink Scala API was insufficient), and
>> we've been looking at ways we could contribute back. What held us back was
>> our lack of knowledge of the full Flink environment (we're only using the
>> Scala Datastream API). I don't think we're the only ones in that situation.
>> One major point, though, is that Flink developers would not be completely
>> rid of us ;-) It's possible that for the sake of the Scala API, we would
>> occasionally require some changes in the Java API. As long as those changes
>> are not detrimental to Java users, they should be considered.
>>
>> A question regarding the Flink wrapper: would it be possible to keep it
>> under the Flink project's umbrella? Or does it need to be a completely
>> separate structure? I'm not aware of the full organizational implications
>> of this, I'm afraid.

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Gaël Renoux]

> I'm curious what target Scala versions people are currently interested
in.
> I would've expected that everyone wants to migrate to Scala 3, for which
several wrapper projects around Flink already exist

The Scala 3 tooling is still subpar (we're using IntelliJ), so I'm not sure
I would move my team to Scala 3 right now (I'm currently toying with it on
a personal project). In addition, moving to Scala 3 is not completely free
- you have to do some rewrites, and developers will need some adaptation
time. Scala 2.13 is another thing entirely, we've wanted to migrate for a
long while.

On Wed, Oct 5, 2022 at 12:53 PM Chesnay Schepler  wrote:

> > It's possible that for the sake of the Scala API, we would occasionally
> require some changes in the Java API. As long as those changes are not
> detrimental to Java users, they should be considered.
>
> That's exactly the model we're trying to get to. Don't fix scala-specific
> issues with scala code, but rather on the Java side as much as possible
> which could also benefit other JVM languages (e.g., Kotlin).
>
> > A question regarding the Flink wrapper: would it be possible to keep it
> under the Flink project's umbrella? Or does it need to be a completely
> separate structure? I'm not aware of the full organizational implications
> of this, I'm afraid.
>
> Technically it can be under the Flink umbrella, but then Flink would still
> be (at least for a while) be the bottleneck because we'd have to review any
> changes coming in.
> That would only improve once several new committers were added to take
> care of this project.
> (In the end you'd just split Flink and the Scala API _codebases_, but
> achieve little else)
>
> > And if that is what it takes to move beyond Scala 2.12.7… This has been
> a big pain point for us.
>
> I'm curious what target Scala versions people are currently interested in.
> I would've expected that everyone wants to migrate to Scala 3, for which
> several wrapper projects around Flink already exist.
>
> On 05/10/2022 12:35, Gaël Renoux wrote:
>
> Hello everyone,
>
> I've already answered a bit on Twitter, I'll develop my thoughts a bit
> here. For context, my company (DataDome) has a significant codebase on
> Scala Flink (around 110K LOC), having been using it since 2017. I myself am
> an enthusiastic Scala developer (I don't think I'd like moving back to
> Java)
>
> Given that, I think separating the Scala support from Flink is actually a
> good move long term. We would then have a full-Java Flink, and a separate
> Scala wrapper. It would help a lot in solving the skills issue: Flink
> maintainers would no longer need to be fluent in Scala, and maintainers of
> the Scala wrapper would not need a deep knowledge of Flink's inner
> workings, just the API would be sufficient. And if that is what it takes to
> move beyond Scala 2.12.7… This has been a big pain point for us.
>
> I'm not too worried about finding contributors for the Scala wrapper.
> Within my company, we have developed additional wrappers and extension
> methods (for parts where we felt the Flink Scala API was insufficient), and
> we've been looking at ways we could contribute back. What held us back was
> our lack of knowledge of the full Flink environment (we're only using the
> Scala Datastream API). I don't think we're the only ones in that situation.
> One major point, though, is that Flink developers would not be completely
> rid of us ;-) It's possible that for the sake of the Scala API, we would
> occasionally require some changes in the Java API. As long as those changes
> are not detrimental to Java users, they should be considered.
>
> A question regarding the Flink wrapper: would it be possible to keep it
> under the Flink project's umbrella? Or does it need to be a completely
> separate structure? I'm not aware of the full organizational implications
> of this, I'm afraid.
>
> Finally, the hard part would be the migration to the new version. My dream
> solution would be to have the existing Scala API be entirely converted into
> a Scala wrapper over the Java API. That way, migration would be pretty
> minimal: add a dependency, change the imports for the Scala API, and we're
> done. However, even starting from the existing flink4s project, that's
> still quite a lot of work. So, more realistically, I'd settle for at least
> a partial implementation. We would have some broken code that we could fix,
> but at the very least I'd like the basic DataStream functions (process,
> uid, name…) to be available.
>
> Thanks for all the work that went into making Flink what it is!
>
>
> Gaël Renoux - Lead R Engineer
> E - gael.ren...@datadome.co
> W - www.datadome.co
>
>
>
> On Wed, Oct 5, 2022 at 9:30 AM Maciek Próchniak  wrote:
>
>> Hi Martin,
>>
>> Could you please remind what was the conclusion of discussion on
>> upgrading Scala to 2.12.15/16?
>> https://lists.apache.org/thread/hwksnsqyg7n3djymo7m1s7loymxxbc3t - I
>> couldn't find any follow-up vote?
>>
>> If it's 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Chesnay Schepler]

> It's possible that for the sake of the Scala API, we would 
occasionally require some changes in the Java API. As long as those 
changes are not detrimental to Java users, they should be considered.


That's exactly the model we're trying to get to. Don't fix 
scala-specific issues with scala code, but rather on the Java side as 
much as possible which could also benefit other JVM languages (e.g., 
Kotlin).


> A question regarding the Flink wrapper: would it be possible to keep 
it under the Flink project's umbrella? Or does it need to be a 
completely separate structure? I'm not aware of the full organizational 
implications of this, I'm afraid.


Technically it can be under the Flink umbrella, but then Flink would 
still be (at least for a while) be the bottleneck because we'd have to 
review any changes coming in.
That would only improve once several new committers were added to take 
care of this project.
(In the end you'd just split Flink and the Scala API _codebases_, but 
achieve little else)


> And if that is what it takes to move beyond Scala 2.12.7… This has 
been a big pain point for us.


I'm curious what target Scala versions people are currently interested in.
I would've expected that everyone wants to migrate to Scala 3, for which 
several wrapper projects around Flink already exist.


On 05/10/2022 12:35, Gaël Renoux wrote:

Hello everyone,

I've already answered a bit on Twitter, I'll develop my thoughts a bit 
here. For context, my company (DataDome) has a significant codebase on 
Scala Flink (around 110K LOC), having been using it since 2017. I 
myself am an enthusiastic Scala developer (I don't think I'd like 
moving back to Java)


Given that, I think separating the Scala support from Flink is 
actually a good move long term. We would then have a full-Java Flink, 
and a separate Scala wrapper. It would help a lot in solving the 
skills issue: Flink maintainers would no longer need to be fluent in 
Scala, and maintainers of the Scala wrapper would not need a deep 
knowledge of Flink's inner workings, just the API would be sufficient. 
And if that is what it takes to move beyond Scala 2.12.7… This has 
been a big pain point for us.


I'm not too worried about finding contributors for the Scala wrapper. 
Within my company, we have developed additional wrappers and extension 
methods (for parts where we felt the Flink Scala API was 
insufficient), and we've been looking at ways we could contribute 
back. What held us back was our lack of knowledge of the full Flink 
environment (we're only using the Scala Datastream API). I don't think 
we're the only ones in that situation. One major point, though, is 
that Flink developers would not be completely rid of us ;-) It's 
possible that for the sake of the Scala API, we would occasionally 
require some changes in the Java API. As long as those changes are not 
detrimental to Java users, they should be considered.


A question regarding the Flink wrapper: would it be possible to keep 
it under the Flink project's umbrella? Or does it need to be a 
completely separate structure? I'm not aware of the full 
organizational implications of this, I'm afraid.


Finally, the hard part would be the migration to the new version. My 
dream solution would be to have the existing Scala API be entirely 
converted into a Scala wrapper over the Java API. That way, migration 
would be pretty minimal: add a dependency, change the imports for the 
Scala API, and we're done. However, even starting from the existing 
flink4s project, that's still quite a lot of work. So, more 
realistically, I'd settle for at least a partial implementation. We 
would have some broken code that we could fix, but at the very least 
I'd like the basic DataStream functions (process, uid, name…) to be 
available.


Thanks for all the work that went into making Flink what it is!


Gaël Renoux - Lead R Engineer
E - gael.ren...@datadome.co
W - www.datadome.co 



On Wed, Oct 5, 2022 at 9:30 AM Maciek Próchniak  wrote:

Hi Martin,

Could you please remind what was the conclusion of discussion on
upgrading Scala to 2.12.15/16?
https://lists.apache.org/thread/hwksnsqyg7n3djymo7m1s7loymxxbc3t -
I couldn't find any follow-up vote?

If it's acceptable to break binary compatibility by such an
upgrade, then upgrading to JDK17 before 2.0 will be doable?


thanks,

maciek


On 04.10.2022 18:21, Martijn Visser wrote:

Hi Yaroslav,

Thanks for the feedback, that's much appreciated! Regarding Java
17 as a prerequisite, we would have to break compatibility
already since Scala 2.12.7 doesn't compile on Java 17 [1].

Given that we can only remove Scala APIs with the next major
Flink (2.0) version, would that still impact you a lot? I do
imagine that if we get to a Flink 2.0 version there would be more
breaking involved anyway. The biggest consequence of deprecating
support for Scala in Flink 1.x would 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Gaël Renoux]

Hello everyone,

I've already answered a bit on Twitter, I'll develop my thoughts a bit
here. For context, my company (DataDome) has a significant codebase on
Scala Flink (around 110K LOC), having been using it since 2017. I myself am
an enthusiastic Scala developer (I don't think I'd like moving back to
Java)

Given that, I think separating the Scala support from Flink is actually a
good move long term. We would then have a full-Java Flink, and a separate
Scala wrapper. It would help a lot in solving the skills issue: Flink
maintainers would no longer need to be fluent in Scala, and maintainers of
the Scala wrapper would not need a deep knowledge of Flink's inner
workings, just the API would be sufficient. And if that is what it takes to
move beyond Scala 2.12.7… This has been a big pain point for us.

I'm not too worried about finding contributors for the Scala wrapper.
Within my company, we have developed additional wrappers and extension
methods (for parts where we felt the Flink Scala API was insufficient), and
we've been looking at ways we could contribute back. What held us back was
our lack of knowledge of the full Flink environment (we're only using the
Scala Datastream API). I don't think we're the only ones in that situation.
One major point, though, is that Flink developers would not be completely
rid of us ;-) It's possible that for the sake of the Scala API, we would
occasionally require some changes in the Java API. As long as those changes
are not detrimental to Java users, they should be considered.

A question regarding the Flink wrapper: would it be possible to keep it
under the Flink project's umbrella? Or does it need to be a completely
separate structure? I'm not aware of the full organizational implications
of this, I'm afraid.

Finally, the hard part would be the migration to the new version. My dream
solution would be to have the existing Scala API be entirely converted into
a Scala wrapper over the Java API. That way, migration would be pretty
minimal: add a dependency, change the imports for the Scala API, and we're
done. However, even starting from the existing flink4s project, that's
still quite a lot of work. So, more realistically, I'd settle for at least
a partial implementation. We would have some broken code that we could fix,
but at the very least I'd like the basic DataStream functions (process,
uid, name…) to be available.

Thanks for all the work that went into making Flink what it is!


Gaël Renoux - Lead R Engineer
E - gael.ren...@datadome.co
W - www.datadome.co



On Wed, Oct 5, 2022 at 9:30 AM Maciek Próchniak  wrote:

> Hi Martin,
>
> Could you please remind what was the conclusion of discussion on upgrading
> Scala to 2.12.15/16?
> https://lists.apache.org/thread/hwksnsqyg7n3djymo7m1s7loymxxbc3t - I
> couldn't find any follow-up vote?
>
> If it's acceptable to break binary compatibility by such an upgrade, then
> upgrading to JDK17 before 2.0 will be doable?
>
>
> thanks,
>
> maciek
>
>
> On 04.10.2022 18:21, Martijn Visser wrote:
>
> Hi Yaroslav,
>
> Thanks for the feedback, that's much appreciated! Regarding Java 17 as a
> prerequisite, we would have to break compatibility already since Scala
> 2.12.7 doesn't compile on Java 17 [1].
>
> Given that we can only remove Scala APIs with the next major Flink (2.0)
> version, would that still impact you a lot? I do imagine that if we get to
> a Flink 2.0 version there would be more breaking involved anyway. The
> biggest consequence of deprecating support for Scala in Flink 1.x would be
> that new APIs would only be available in Java, but since these don't exist
> yet there would be no refactoring involved. I can imagine that we might
> change something in an existing API, but that would have certain
> compatibility guarantees already (depending if it's
> Public/PublicEvolving/Experimental). If a change would happen there, I
> think it would be smaller refactoring.
>
> Best regards,
>
> Martijn
>
> [1] https://issues.apache.org/jira/browse/FLINK-25000
>
> On Tue, Oct 4, 2022 at 10:58 AM Yaroslav Tkachenko 
> wrote:
>
>> Hi Martijn,
>>
>> As a Scala user, this change would affect me a lot and I'm not looking
>> forward to rewriting my codebase, and it's not even a very large one :)
>>
>> I'd like to suggest supporting Java 17 as a prerequisite (
>> https://issues.apache.org/jira/browse/FLINK-15736). Things like switch
>> expressions and records could simplify the migration quite a bit. Would you
>> consider adding it to the FLIP?
>>
>> On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:
>>
>>> Hi Martijn,
>>>
>>> Thanks for bringing this up. It is generally a great idea, so +1.
>>>
>>> Since both scala extension projects mentioned in the FLIP are still very
>>> young and I don't think they will attract more scala developers as Flink
>>> could just because they are external projects. It will be a big issue for
>>> users who have to rewrite their large codebases. Those users should be
>>> aware of the effort from now on and 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Maciek Próchniak]

Hi Martin,

Could you please remind what was the conclusion of discussion on 
upgrading Scala to 2.12.15/16? 
https://lists.apache.org/thread/hwksnsqyg7n3djymo7m1s7loymxxbc3t - I 
couldn't find any follow-up vote?


If it's acceptable to break binary compatibility by such an upgrade, 
then upgrading to JDK17 before 2.0 will be doable?



thanks,

maciek


On 04.10.2022 18:21, Martijn Visser wrote:

Hi Yaroslav,

Thanks for the feedback, that's much appreciated! Regarding Java 17 as 
a prerequisite, we would have to break compatibility already since 
Scala 2.12.7 doesn't compile on Java 17 [1].


Given that we can only remove Scala APIs with the next major Flink 
(2.0) version, would that still impact you a lot? I do imagine that if 
we get to a Flink 2.0 version there would be more breaking involved 
anyway. The biggest consequence of deprecating support for Scala in 
Flink 1.x would be that new APIs would only be available in Java, but 
since these don't exist yet there would be no refactoring involved. I 
can imagine that we might change something in an existing API, but 
that would have certain compatibility guarantees already (depending if 
it's Public/PublicEvolving/Experimental). If a change would happen 
there, I think it would be smaller refactoring.


Best regards,

Martijn

[1] https://issues.apache.org/jira/browse/FLINK-25000

On Tue, Oct 4, 2022 at 10:58 AM Yaroslav Tkachenko 
 wrote:


Hi Martijn,

As a Scala user, this change would affect me a lot and I'm not
looking forward to rewriting my codebase, and it's not even a very
large one :)

I'd like to suggest supporting Java 17 as a prerequisite
(https://issues.apache.org/jira/browse/FLINK-15736). Things like
switch expressions and records could simplify the migration
quite a bit. Would you consider adding it to the FLIP?

On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:

Hi Martijn,

Thanks for bringing this up. It is generally a great idea, so +1.

Since both scala extension projects mentioned in the FLIP are
still very young and I don't think they will attract more
scala developers as Flink could just because they are external
projects. It will be a big issue for users who have to rewrite
their large codebases. Those users should be aware of the
effort from now on and would better not count on those scala
extension projects and prepare their migration plan
before Flink 2.0.

Best regards,
Jing


On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser
 wrote:

Hi Marton,

You're making a good point, I originally wanted to include
already the User mailing list to get their feedback but
forgot to do so. I'll do some more outreach via other
channels as well.

@Users of Flink, I've made a proposal to deprecate and
remove Scala API support in a future version of Flink.
Your feedback on this topic is very much appreciated.

Regarding the large Scala codebase for Flink, a potential
alternative could be to have a wrapper for all Java APIs
that makes them available as Scala APIs. However, this
still requires Scala maintainers and I don't think that we
currently have those in our community. The easiest
solution for them would be to use the Java APIs directly.
Yes it would involve work, but we won't actually be able
to remove the Scala APIs until Flink 2.0 so there's still
time for that :)

Best regards,

Martijn

On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi
 wrote:

Hi Martjin,

Thanks for compiling the FLIP. I agree with the
sentiment that Scala poses
considerable maintenance overhead and key improvements
(like 2.13 or 2.12.8
supports) are hanging stale. With that said before we
make this move we
should attempt to understand the userbase affected.
A quick Slack and user mailing list search does return
quite a bit of
results for scala (admittedly a cursory look at them
suggest that many of
them have to do with missing features in Scala that
exist in Java or Scala
versions). I would love to see some polls on this
topic, we could also use
the Flink twitter handle to ask the community about this.

I am aware of users having large existing Scala
codebases for Flink. This
move would pose a very large effort on them, as they
would need to rewrite
much of their existing code. What are the alternatives
in your 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Yaroslav Tkachenko]

Hey Martijn,

Well, as a user I think that Scala API still adds a tremendous value, with
all its issues. But I'm not a committer and I don't know what effort it
takes to keep maintaining it... so I prepare for the worst :)

Regarding the proposed timeline, I don't know all the specifics around
breaking compatibility changes, so it's hard for me to say, but I'd love to
see Java 17 before Scala is fully removed from the project.




On Tue, Oct 4, 2022 at 3:07 PM Martijn Visser 
wrote:

> Hi Yaroslav,
>
> If I could summarize your suggestion, would it mean that you would only be
> in favour of dropping Scala API support if we introduce Java 17 support
> exactly at the same time (say Flink 2.0). I was first thinking that an
> alternative would be to have a Flink 2.0 which supports Java 17 while
> keeping the Scala APIs (and postpone dropping those in Flink 3.0), but that
> wouldn't make much sense since we would have to break savepoint
> compatibility for Scala users then anyway.
>
> However, since we're already talking about when/how the APIs could be
> dropped, does that mean that you agree with the idea to deprecate and
> remove the Scala APIs?
>
> Best regards,
>
> Martijn
>
>
> On Tue, Oct 4, 2022 at 2:41 PM Yaroslav Tkachenko 
> wrote:
>
>> Hi Martijn,
>>
>> The 2.0 argument makes sense (I agree it's easier to introduce more
>> breaking changes in one major release), but I think my comment about Java
>> 17 also makes sense in this case: 1) easier to introduce because breaking
>> changes are possible 2) you'd need to give some syntax sugar as an
>> alternative after removing Scala.
>>
>> On Tue, Oct 4, 2022 at 11:21 AM Martijn Visser 
>> wrote:
>>
>>> Hi Yaroslav,
>>>
>>> Thanks for the feedback, that's much appreciated! Regarding Java 17 as a
>>> prerequisite, we would have to break compatibility already since Scala
>>> 2.12.7 doesn't compile on Java 17 [1].
>>>
>>> Given that we can only remove Scala APIs with the next major Flink (2.0)
>>> version, would that still impact you a lot? I do imagine that if we get to
>>> a Flink 2.0 version there would be more breaking involved anyway. The
>>> biggest consequence of deprecating support for Scala in Flink 1.x would be
>>> that new APIs would only be available in Java, but since these don't exist
>>> yet there would be no refactoring involved. I can imagine that we might
>>> change something in an existing API, but that would have certain
>>> compatibility guarantees already (depending if it's
>>> Public/PublicEvolving/Experimental). If a change would happen there, I
>>> think it would be smaller refactoring.
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> [1] https://issues.apache.org/jira/browse/FLINK-25000
>>>
>>> On Tue, Oct 4, 2022 at 10:58 AM Yaroslav Tkachenko 
>>> wrote:
>>>
 Hi Martijn,

 As a Scala user, this change would affect me a lot and I'm not looking
 forward to rewriting my codebase, and it's not even a very large one :)

 I'd like to suggest supporting Java 17 as a prerequisite (
 https://issues.apache.org/jira/browse/FLINK-15736). Things like switch
 expressions and records could simplify the migration quite a bit. Would you
 consider adding it to the FLIP?

 On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:

> Hi Martijn,
>
> Thanks for bringing this up. It is generally a great idea, so +1.
>
> Since both scala extension projects mentioned in the FLIP are still
> very young and I don't think they will attract more scala developers as
> Flink could just because they are external projects. It will be a big 
> issue
> for users who have to rewrite their large codebases. Those users should be
> aware of the effort from now on and would better not count on those scala
> extension projects and prepare their migration plan before Flink 2.0.
>
> Best regards,
> Jing
>
>
> On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser <
> martijnvis...@apache.org> wrote:
>
>> Hi Marton,
>>
>> You're making a good point, I originally wanted to include already
>> the User mailing list to get their feedback but forgot to do so. I'll do
>> some more outreach via other channels as well.
>>
>> @Users of Flink, I've made a proposal to deprecate and remove Scala
>> API support in a future version of Flink. Your feedback on this topic is
>> very much appreciated.
>>
>> Regarding the large Scala codebase for Flink, a potential alternative
>> could be to have a wrapper for all Java APIs that makes them available as
>> Scala APIs. However, this still requires Scala maintainers and I don't
>> think that we currently have those in our community. The easiest solution
>> for them would be to use the Java APIs directly. Yes it would involve 
>> work,
>> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
>> there's still time for that :)
>>
>> Best regards,

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Martijn Visser]

Hi Yaroslav,

If I could summarize your suggestion, would it mean that you would only be
in favour of dropping Scala API support if we introduce Java 17 support
exactly at the same time (say Flink 2.0). I was first thinking that an
alternative would be to have a Flink 2.0 which supports Java 17 while
keeping the Scala APIs (and postpone dropping those in Flink 3.0), but that
wouldn't make much sense since we would have to break savepoint
compatibility for Scala users then anyway.

However, since we're already talking about when/how the APIs could be
dropped, does that mean that you agree with the idea to deprecate and
remove the Scala APIs?

Best regards,

Martijn


On Tue, Oct 4, 2022 at 2:41 PM Yaroslav Tkachenko 
wrote:

> Hi Martijn,
>
> The 2.0 argument makes sense (I agree it's easier to introduce more
> breaking changes in one major release), but I think my comment about Java
> 17 also makes sense in this case: 1) easier to introduce because breaking
> changes are possible 2) you'd need to give some syntax sugar as an
> alternative after removing Scala.
>
> On Tue, Oct 4, 2022 at 11:21 AM Martijn Visser 
> wrote:
>
>> Hi Yaroslav,
>>
>> Thanks for the feedback, that's much appreciated! Regarding Java 17 as a
>> prerequisite, we would have to break compatibility already since Scala
>> 2.12.7 doesn't compile on Java 17 [1].
>>
>> Given that we can only remove Scala APIs with the next major Flink (2.0)
>> version, would that still impact you a lot? I do imagine that if we get to
>> a Flink 2.0 version there would be more breaking involved anyway. The
>> biggest consequence of deprecating support for Scala in Flink 1.x would be
>> that new APIs would only be available in Java, but since these don't exist
>> yet there would be no refactoring involved. I can imagine that we might
>> change something in an existing API, but that would have certain
>> compatibility guarantees already (depending if it's
>> Public/PublicEvolving/Experimental). If a change would happen there, I
>> think it would be smaller refactoring.
>>
>> Best regards,
>>
>> Martijn
>>
>> [1] https://issues.apache.org/jira/browse/FLINK-25000
>>
>> On Tue, Oct 4, 2022 at 10:58 AM Yaroslav Tkachenko 
>> wrote:
>>
>>> Hi Martijn,
>>>
>>> As a Scala user, this change would affect me a lot and I'm not looking
>>> forward to rewriting my codebase, and it's not even a very large one :)
>>>
>>> I'd like to suggest supporting Java 17 as a prerequisite (
>>> https://issues.apache.org/jira/browse/FLINK-15736). Things like switch
>>> expressions and records could simplify the migration quite a bit. Would you
>>> consider adding it to the FLIP?
>>>
>>> On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:
>>>
 Hi Martijn,

 Thanks for bringing this up. It is generally a great idea, so +1.

 Since both scala extension projects mentioned in the FLIP are still
 very young and I don't think they will attract more scala developers as
 Flink could just because they are external projects. It will be a big issue
 for users who have to rewrite their large codebases. Those users should be
 aware of the effort from now on and would better not count on those scala
 extension projects and prepare their migration plan before Flink 2.0.

 Best regards,
 Jing


 On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser 
 wrote:

> Hi Marton,
>
> You're making a good point, I originally wanted to include already the
> User mailing list to get their feedback but forgot to do so. I'll do some
> more outreach via other channels as well.
>
> @Users of Flink, I've made a proposal to deprecate and remove Scala
> API support in a future version of Flink. Your feedback on this topic is
> very much appreciated.
>
> Regarding the large Scala codebase for Flink, a potential alternative
> could be to have a wrapper for all Java APIs that makes them available as
> Scala APIs. However, this still requires Scala maintainers and I don't
> think that we currently have those in our community. The easiest solution
> for them would be to use the Java APIs directly. Yes it would involve 
> work,
> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
> there's still time for that :)
>
> Best regards,
>
> Martijn
>
> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi <
> balassi.mar...@gmail.com> wrote:
>
>> Hi Martjin,
>>
>> Thanks for compiling the FLIP. I agree with the sentiment that Scala
>> poses
>> considerable maintenance overhead and key improvements (like 2.13 or
>> 2.12.8
>> supports) are hanging stale. With that said before we make this move
>> we
>> should attempt to understand the userbase affected.
>> A quick Slack and user mailing list search does return quite a bit of
>> results for scala (admittedly a cursory look at them suggest that
>> many of
>> 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Yaroslav Tkachenko]

Hi Martijn,

The 2.0 argument makes sense (I agree it's easier to introduce more
breaking changes in one major release), but I think my comment about Java
17 also makes sense in this case: 1) easier to introduce because breaking
changes are possible 2) you'd need to give some syntax sugar as an
alternative after removing Scala.

On Tue, Oct 4, 2022 at 11:21 AM Martijn Visser 
wrote:

> Hi Yaroslav,
>
> Thanks for the feedback, that's much appreciated! Regarding Java 17 as a
> prerequisite, we would have to break compatibility already since Scala
> 2.12.7 doesn't compile on Java 17 [1].
>
> Given that we can only remove Scala APIs with the next major Flink (2.0)
> version, would that still impact you a lot? I do imagine that if we get to
> a Flink 2.0 version there would be more breaking involved anyway. The
> biggest consequence of deprecating support for Scala in Flink 1.x would be
> that new APIs would only be available in Java, but since these don't exist
> yet there would be no refactoring involved. I can imagine that we might
> change something in an existing API, but that would have certain
> compatibility guarantees already (depending if it's
> Public/PublicEvolving/Experimental). If a change would happen there, I
> think it would be smaller refactoring.
>
> Best regards,
>
> Martijn
>
> [1] https://issues.apache.org/jira/browse/FLINK-25000
>
> On Tue, Oct 4, 2022 at 10:58 AM Yaroslav Tkachenko 
> wrote:
>
>> Hi Martijn,
>>
>> As a Scala user, this change would affect me a lot and I'm not looking
>> forward to rewriting my codebase, and it's not even a very large one :)
>>
>> I'd like to suggest supporting Java 17 as a prerequisite (
>> https://issues.apache.org/jira/browse/FLINK-15736). Things like switch
>> expressions and records could simplify the migration quite a bit. Would you
>> consider adding it to the FLIP?
>>
>> On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:
>>
>>> Hi Martijn,
>>>
>>> Thanks for bringing this up. It is generally a great idea, so +1.
>>>
>>> Since both scala extension projects mentioned in the FLIP are still very
>>> young and I don't think they will attract more scala developers as Flink
>>> could just because they are external projects. It will be a big issue for
>>> users who have to rewrite their large codebases. Those users should be
>>> aware of the effort from now on and would better not count on those scala
>>> extension projects and prepare their migration plan before Flink 2.0.
>>>
>>> Best regards,
>>> Jing
>>>
>>>
>>> On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser 
>>> wrote:
>>>
 Hi Marton,

 You're making a good point, I originally wanted to include already the
 User mailing list to get their feedback but forgot to do so. I'll do some
 more outreach via other channels as well.

 @Users of Flink, I've made a proposal to deprecate and remove Scala API
 support in a future version of Flink. Your feedback on this topic is very
 much appreciated.

 Regarding the large Scala codebase for Flink, a potential alternative
 could be to have a wrapper for all Java APIs that makes them available as
 Scala APIs. However, this still requires Scala maintainers and I don't
 think that we currently have those in our community. The easiest solution
 for them would be to use the Java APIs directly. Yes it would involve work,
 but we won't actually be able to remove the Scala APIs until Flink 2.0 so
 there's still time for that :)

 Best regards,

 Martijn

 On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
 wrote:

> Hi Martjin,
>
> Thanks for compiling the FLIP. I agree with the sentiment that Scala
> poses
> considerable maintenance overhead and key improvements (like 2.13 or
> 2.12.8
> supports) are hanging stale. With that said before we make this move we
> should attempt to understand the userbase affected.
> A quick Slack and user mailing list search does return quite a bit of
> results for scala (admittedly a cursory look at them suggest that many
> of
> them have to do with missing features in Scala that exist in Java or
> Scala
> versions). I would love to see some polls on this topic, we could also
> use
> the Flink twitter handle to ask the community about this.
>
> I am aware of users having large existing Scala codebases for Flink.
> This
> move would pose a very large effort on them, as they would need to
> rewrite
> much of their existing code. What are the alternatives in your opinion,
> Martjin?
>
> On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser <
> martijnvis...@apache.org>
> wrote:
>
> > Hi everyone,
> >
> > I would like to open a discussion thread on FLIP-265 Deprecate and
> remove
> > Scala API support. Please take a look at
> >
> >
> 

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Martijn Visser]

Hi Yaroslav,

Thanks for the feedback, that's much appreciated! Regarding Java 17 as a
prerequisite, we would have to break compatibility already since Scala
2.12.7 doesn't compile on Java 17 [1].

Given that we can only remove Scala APIs with the next major Flink (2.0)
version, would that still impact you a lot? I do imagine that if we get to
a Flink 2.0 version there would be more breaking involved anyway. The
biggest consequence of deprecating support for Scala in Flink 1.x would be
that new APIs would only be available in Java, but since these don't exist
yet there would be no refactoring involved. I can imagine that we might
change something in an existing API, but that would have certain
compatibility guarantees already (depending if it's
Public/PublicEvolving/Experimental). If a change would happen there, I
think it would be smaller refactoring.

Best regards,

Martijn

[1] https://issues.apache.org/jira/browse/FLINK-25000

On Tue, Oct 4, 2022 at 10:58 AM Yaroslav Tkachenko 
wrote:

> Hi Martijn,
>
> As a Scala user, this change would affect me a lot and I'm not looking
> forward to rewriting my codebase, and it's not even a very large one :)
>
> I'd like to suggest supporting Java 17 as a prerequisite (
> https://issues.apache.org/jira/browse/FLINK-15736). Things like switch
> expressions and records could simplify the migration quite a bit. Would you
> consider adding it to the FLIP?
>
> On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:
>
>> Hi Martijn,
>>
>> Thanks for bringing this up. It is generally a great idea, so +1.
>>
>> Since both scala extension projects mentioned in the FLIP are still very
>> young and I don't think they will attract more scala developers as Flink
>> could just because they are external projects. It will be a big issue for
>> users who have to rewrite their large codebases. Those users should be
>> aware of the effort from now on and would better not count on those scala
>> extension projects and prepare their migration plan before Flink 2.0.
>>
>> Best regards,
>> Jing
>>
>>
>> On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser 
>> wrote:
>>
>>> Hi Marton,
>>>
>>> You're making a good point, I originally wanted to include already the
>>> User mailing list to get their feedback but forgot to do so. I'll do some
>>> more outreach via other channels as well.
>>>
>>> @Users of Flink, I've made a proposal to deprecate and remove Scala API
>>> support in a future version of Flink. Your feedback on this topic is very
>>> much appreciated.
>>>
>>> Regarding the large Scala codebase for Flink, a potential alternative
>>> could be to have a wrapper for all Java APIs that makes them available as
>>> Scala APIs. However, this still requires Scala maintainers and I don't
>>> think that we currently have those in our community. The easiest solution
>>> for them would be to use the Java APIs directly. Yes it would involve work,
>>> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
>>> there's still time for that :)
>>>
>>> Best regards,
>>>
>>> Martijn
>>>
>>> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
>>> wrote:
>>>
 Hi Martjin,

 Thanks for compiling the FLIP. I agree with the sentiment that Scala
 poses
 considerable maintenance overhead and key improvements (like 2.13 or
 2.12.8
 supports) are hanging stale. With that said before we make this move we
 should attempt to understand the userbase affected.
 A quick Slack and user mailing list search does return quite a bit of
 results for scala (admittedly a cursory look at them suggest that many
 of
 them have to do with missing features in Scala that exist in Java or
 Scala
 versions). I would love to see some polls on this topic, we could also
 use
 the Flink twitter handle to ask the community about this.

 I am aware of users having large existing Scala codebases for Flink.
 This
 move would pose a very large effort on them, as they would need to
 rewrite
 much of their existing code. What are the alternatives in your opinion,
 Martjin?

 On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser >>> >
 wrote:

 > Hi everyone,
 >
 > I would like to open a discussion thread on FLIP-265 Deprecate and
 remove
 > Scala API support. Please take a look at
 >
 >
 https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
 > and provide your feedback.
 >
 > Best regards,
 >
 > Martijn
 > https://twitter.com/MartijnVisser82
 > https://github.com/MartijnVisser
 >

>>>

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Clayton Wohl]

+1

At my employer, we maintain several Flink jobs in Scala. We've been writing
newer jobs in Java, and we'd be fine with porting our Scala jobs over to
the Java API.

I'd like to request Java 17 support. Specifically, Java records is a
feature our Flink code would use a lot of and make the Java syntax much
nicer.

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Yaroslav Tkachenko]

Hi Martijn,

As a Scala user, this change would affect me a lot and I'm not looking
forward to rewriting my codebase, and it's not even a very large one :)

I'd like to suggest supporting Java 17 as a prerequisite (
https://issues.apache.org/jira/browse/FLINK-15736). Things like switch
expressions and records could simplify the migration quite a bit. Would you
consider adding it to the FLIP?

On Tue, Oct 4, 2022 at 10:50 AM Jing Ge  wrote:

> Hi Martijn,
>
> Thanks for bringing this up. It is generally a great idea, so +1.
>
> Since both scala extension projects mentioned in the FLIP are still very
> young and I don't think they will attract more scala developers as Flink
> could just because they are external projects. It will be a big issue for
> users who have to rewrite their large codebases. Those users should be
> aware of the effort from now on and would better not count on those scala
> extension projects and prepare their migration plan before Flink 2.0.
>
> Best regards,
> Jing
>
>
> On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser 
> wrote:
>
>> Hi Marton,
>>
>> You're making a good point, I originally wanted to include already the
>> User mailing list to get their feedback but forgot to do so. I'll do some
>> more outreach via other channels as well.
>>
>> @Users of Flink, I've made a proposal to deprecate and remove Scala API
>> support in a future version of Flink. Your feedback on this topic is very
>> much appreciated.
>>
>> Regarding the large Scala codebase for Flink, a potential alternative
>> could be to have a wrapper for all Java APIs that makes them available as
>> Scala APIs. However, this still requires Scala maintainers and I don't
>> think that we currently have those in our community. The easiest solution
>> for them would be to use the Java APIs directly. Yes it would involve work,
>> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
>> there's still time for that :)
>>
>> Best regards,
>>
>> Martijn
>>
>> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
>> wrote:
>>
>>> Hi Martjin,
>>>
>>> Thanks for compiling the FLIP. I agree with the sentiment that Scala
>>> poses
>>> considerable maintenance overhead and key improvements (like 2.13 or
>>> 2.12.8
>>> supports) are hanging stale. With that said before we make this move we
>>> should attempt to understand the userbase affected.
>>> A quick Slack and user mailing list search does return quite a bit of
>>> results for scala (admittedly a cursory look at them suggest that many of
>>> them have to do with missing features in Scala that exist in Java or
>>> Scala
>>> versions). I would love to see some polls on this topic, we could also
>>> use
>>> the Flink twitter handle to ask the community about this.
>>>
>>> I am aware of users having large existing Scala codebases for Flink. This
>>> move would pose a very large effort on them, as they would need to
>>> rewrite
>>> much of their existing code. What are the alternatives in your opinion,
>>> Martjin?
>>>
>>> On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser 
>>> wrote:
>>>
>>> > Hi everyone,
>>> >
>>> > I would like to open a discussion thread on FLIP-265 Deprecate and
>>> remove
>>> > Scala API support. Please take a look at
>>> >
>>> >
>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
>>> > and provide your feedback.
>>> >
>>> > Best regards,
>>> >
>>> > Martijn
>>> > https://twitter.com/MartijnVisser82
>>> > https://github.com/MartijnVisser
>>> >
>>>
>>

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Jing Ge]

Hi Martijn,

Thanks for bringing this up. It is generally a great idea, so +1.

Since both scala extension projects mentioned in the FLIP are still very
young and I don't think they will attract more scala developers as Flink
could just because they are external projects. It will be a big issue for
users who have to rewrite their large codebases. Those users should be
aware of the effort from now on and would better not count on those scala
extension projects and prepare their migration plan before Flink 2.0.

Best regards,
Jing


On Tue, Oct 4, 2022 at 1:59 PM Martijn Visser 
wrote:

> Hi Marton,
>
> You're making a good point, I originally wanted to include already the
> User mailing list to get their feedback but forgot to do so. I'll do some
> more outreach via other channels as well.
>
> @Users of Flink, I've made a proposal to deprecate and remove Scala API
> support in a future version of Flink. Your feedback on this topic is very
> much appreciated.
>
> Regarding the large Scala codebase for Flink, a potential alternative
> could be to have a wrapper for all Java APIs that makes them available as
> Scala APIs. However, this still requires Scala maintainers and I don't
> think that we currently have those in our community. The easiest solution
> for them would be to use the Java APIs directly. Yes it would involve work,
> but we won't actually be able to remove the Scala APIs until Flink 2.0 so
> there's still time for that :)
>
> Best regards,
>
> Martijn
>
> On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
> wrote:
>
>> Hi Martjin,
>>
>> Thanks for compiling the FLIP. I agree with the sentiment that Scala poses
>> considerable maintenance overhead and key improvements (like 2.13 or
>> 2.12.8
>> supports) are hanging stale. With that said before we make this move we
>> should attempt to understand the userbase affected.
>> A quick Slack and user mailing list search does return quite a bit of
>> results for scala (admittedly a cursory look at them suggest that many of
>> them have to do with missing features in Scala that exist in Java or Scala
>> versions). I would love to see some polls on this topic, we could also use
>> the Flink twitter handle to ask the community about this.
>>
>> I am aware of users having large existing Scala codebases for Flink. This
>> move would pose a very large effort on them, as they would need to rewrite
>> much of their existing code. What are the alternatives in your opinion,
>> Martjin?
>>
>> On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser 
>> wrote:
>>
>> > Hi everyone,
>> >
>> > I would like to open a discussion thread on FLIP-265 Deprecate and
>> remove
>> > Scala API support. Please take a look at
>> >
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
>> > and provide your feedback.
>> >
>> > Best regards,
>> >
>> > Martijn
>> > https://twitter.com/MartijnVisser82
>> > https://github.com/MartijnVisser
>> >
>>
>

Re: [DISCUSS] FLIP-265 Deprecate and remove Scala API support

[Martijn Visser]

Hi Marton,

You're making a good point, I originally wanted to include already the User
mailing list to get their feedback but forgot to do so. I'll do some more
outreach via other channels as well.

@Users of Flink, I've made a proposal to deprecate and remove Scala API
support in a future version of Flink. Your feedback on this topic is very
much appreciated.

Regarding the large Scala codebase for Flink, a potential alternative could
be to have a wrapper for all Java APIs that makes them available as Scala
APIs. However, this still requires Scala maintainers and I don't think that
we currently have those in our community. The easiest solution for them
would be to use the Java APIs directly. Yes it would involve work, but we
won't actually be able to remove the Scala APIs until Flink 2.0 so there's
still time for that :)

Best regards,

Martijn

On Tue, Oct 4, 2022 at 1:26 AM Márton Balassi 
wrote:

> Hi Martjin,
>
> Thanks for compiling the FLIP. I agree with the sentiment that Scala poses
> considerable maintenance overhead and key improvements (like 2.13 or 2.12.8
> supports) are hanging stale. With that said before we make this move we
> should attempt to understand the userbase affected.
> A quick Slack and user mailing list search does return quite a bit of
> results for scala (admittedly a cursory look at them suggest that many of
> them have to do with missing features in Scala that exist in Java or Scala
> versions). I would love to see some polls on this topic, we could also use
> the Flink twitter handle to ask the community about this.
>
> I am aware of users having large existing Scala codebases for Flink. This
> move would pose a very large effort on them, as they would need to rewrite
> much of their existing code. What are the alternatives in your opinion,
> Martjin?
>
> On Tue, Oct 4, 2022 at 6:22 AM Martijn Visser 
> wrote:
>
> > Hi everyone,
> >
> > I would like to open a discussion thread on FLIP-265 Deprecate and remove
> > Scala API support. Please take a look at
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-265+Deprecate+and+remove+Scala+API+support
> > and provide your feedback.
> >
> > Best regards,
> >
> > Martijn
> > https://twitter.com/MartijnVisser82
> > https://github.com/MartijnVisser
> >
>