Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Matthias Berndt]

Htey there,

I'd like to raise this topic again because I haven't yet gotten anything
actionable here and I want to make a Scala 3 release of Kafka-Streams-Scala
happen sooner rather than later.

All the best,
Matthias

Am Di., 13. Feb. 2024 um 12:48 Uhr schrieb Matthias Berndt <
matthias.ber...@ttmzero.com>:

> Hey Matthew,
>
> Kafka-Streams-Scala is an entirely separate codebase from Kafka Core, and
> the fact that almost three years after the release of Scala 3 there is
> still no release of Kafka-Streams-Scala for Scala 3 – even though the
> required changes to the codebase are trivial – tells me that these things
> should not be coupled in the way that they currently are. I think having
> Kafka-Streams-Scala users wait for Kafka 4 would be doing them a
> disservice. Are we really going to delay this because of some CI scripts?
>
> All the best
> Matthias
>
> Am Di., 13. Feb. 2024 um 11:20 Uhr schrieb Matthew de Detrich
> :
>
>> As added info on top of what Josep said, in the Scala space most OS
>> software supports
>> Scala 2.12/Scala 2.13 and Scala 3 but with Scala 2.12 specifically the
>> Scala OS
>> community itself wants to eventually have people to stop supporting it
>> (it's in maintenance mode) so it makes sense to tie this into Kafka 4.0.x
>> which will
>> drop Scala 2.12 support.
>>
>> We should also take care and make sure the Scala 3 version is pinned to
>> 3.3.x since
>> it's part of the LTS[1] series, in other words we should be careful that
>> people won't bump
>> the Scala version to 3.4.x.
>>
>> 1:
>>
>> https://www.scala-lang.org/blog/2022/08/17/long-term-compatibility-plans.html
>>
>> On Sat, Feb 10, 2024 at 11:54 AM Matthias Berndt <
>> matthias.ber...@ttmzero.com> wrote:
>>
>> > Hey there,
>> >
>> > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala
>> library.
>> > As you might have seen already, I have recently created a ticket
>> > https://issues.apache.org/jira/browse/KAFKA-16237
>> > and a PR
>> > https://github.com/apache/kafka/pull/15338
>> > to move this forward. The changes required to make Kafka-Streams-Scala
>> > compile with Scala 3 are trivial; the trickier part is the build system
>> and
>> > the release process
>> > I have made some changes to the build system (feel free to comment on
>> the
>> > above PR about that) that make it possible to test Kafka-Streams-Scala
>> and
>> > build the jar. What remains to be done is the CI and release process.
>> There
>> > is a `release.py` file in the Kafka repository's root directory, which
>> > assumes that all artifacts are available for all supported Scala
>> versions.
>> > This is no longer the case with my changes because while porting
>> > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is
>> less
>> > so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
>> > would appreciate some guidance as to what the release process should
>> look
>> > like in the future.
>> >
>> > Oh and I've made a PR to remove a syntax error from release.py.
>> > https://github.com/apache/kafka/pull/15350
>> >
>> > All the best,
>> > Matthias
>> >
>>
>>
>> --
>>
>> Matthew de Detrich
>>
>> *Aiven Deutschland GmbH*
>>
>> Immanuelkirchstraße 26, 10405 Berlin
>>
>> Alexanderufer 3-7, 10117 Berlin
>>
>> Amtsgericht Charlottenburg, HRB 209739 B
>>
>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>>
>> *m:* +491603708037
>>
>> *w:* aiven.io *e:* matthew.dedetr...@aiven.io
>>
>

Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Matthias Berndt]

Hey Matthew,

Kafka-Streams-Scala is an entirely separate codebase from Kafka Core, and
the fact that almost three years after the release of Scala 3 there is
still no release of Kafka-Streams-Scala for Scala 3 – even though the
required changes to the codebase are trivial – tells me that these things
should not be coupled in the way that they currently are. I think having
Kafka-Streams-Scala users wait for Kafka 4 would be doing them a
disservice. Are we really going to delay this because of some CI scripts?

All the best
Matthias

Am Di., 13. Feb. 2024 um 11:20 Uhr schrieb Matthew de Detrich
:

> As added info on top of what Josep said, in the Scala space most OS
> software supports
> Scala 2.12/Scala 2.13 and Scala 3 but with Scala 2.12 specifically the
> Scala OS
> community itself wants to eventually have people to stop supporting it
> (it's in maintenance mode) so it makes sense to tie this into Kafka 4.0.x
> which will
> drop Scala 2.12 support.
>
> We should also take care and make sure the Scala 3 version is pinned to
> 3.3.x since
> it's part of the LTS[1] series, in other words we should be careful that
> people won't bump
> the Scala version to 3.4.x.
>
> 1:
>
> https://www.scala-lang.org/blog/2022/08/17/long-term-compatibility-plans.html
>
> On Sat, Feb 10, 2024 at 11:54 AM Matthias Berndt <
> matthias.ber...@ttmzero.com> wrote:
>
> > Hey there,
> >
> > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala library.
> > As you might have seen already, I have recently created a ticket
> > https://issues.apache.org/jira/browse/KAFKA-16237
> > and a PR
> > https://github.com/apache/kafka/pull/15338
> > to move this forward. The changes required to make Kafka-Streams-Scala
> > compile with Scala 3 are trivial; the trickier part is the build system
> and
> > the release process
> > I have made some changes to the build system (feel free to comment on the
> > above PR about that) that make it possible to test Kafka-Streams-Scala
> and
> > build the jar. What remains to be done is the CI and release process.
> There
> > is a `release.py` file in the Kafka repository's root directory, which
> > assumes that all artifacts are available for all supported Scala
> versions.
> > This is no longer the case with my changes because while porting
> > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is
> less
> > so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
> > would appreciate some guidance as to what the release process should look
> > like in the future.
> >
> > Oh and I've made a PR to remove a syntax error from release.py.
> > https://github.com/apache/kafka/pull/15350
> >
> > All the best,
> > Matthias
> >
>
>
> --
>
> Matthew de Detrich
>
> *Aiven Deutschland GmbH*
>
> Immanuelkirchstraße 26, 10405 Berlin
>
> Alexanderufer 3-7, 10117 Berlin
>
> Amtsgericht Charlottenburg, HRB 209739 B
>
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
>
> *m:* +491603708037
>
> *w:* aiven.io *e:* matthew.dedetr...@aiven.io
>

Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Matthew de Detrich]

As added info on top of what Josep said, in the Scala space most OS
software supports
Scala 2.12/Scala 2.13 and Scala 3 but with Scala 2.12 specifically the
Scala OS
community itself wants to eventually have people to stop supporting it
(it's in maintenance mode) so it makes sense to tie this into Kafka 4.0.x
which will
drop Scala 2.12 support.

We should also take care and make sure the Scala 3 version is pinned to
3.3.x since
it's part of the LTS[1] series, in other words we should be careful that
people won't bump
the Scala version to 3.4.x.

1:
https://www.scala-lang.org/blog/2022/08/17/long-term-compatibility-plans.html

On Sat, Feb 10, 2024 at 11:54 AM Matthias Berndt <
matthias.ber...@ttmzero.com> wrote:

> Hey there,
>
> I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala library.
> As you might have seen already, I have recently created a ticket
> https://issues.apache.org/jira/browse/KAFKA-16237
> and a PR
> https://github.com/apache/kafka/pull/15338
> to move this forward. The changes required to make Kafka-Streams-Scala
> compile with Scala 3 are trivial; the trickier part is the build system and
> the release process
> I have made some changes to the build system (feel free to comment on the
> above PR about that) that make it possible to test Kafka-Streams-Scala and
> build the jar. What remains to be done is the CI and release process. There
> is a `release.py` file in the Kafka repository's root directory, which
> assumes that all artifacts are available for all supported Scala versions.
> This is no longer the case with my changes because while porting
> Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is less
> so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
> would appreciate some guidance as to what the release process should look
> like in the future.
>
> Oh and I've made a PR to remove a syntax error from release.py.
> https://github.com/apache/kafka/pull/15350
>
> All the best,
> Matthias
>


-- 

Matthew de Detrich

*Aiven Deutschland GmbH*

Immanuelkirchstraße 26, 10405 Berlin

Alexanderufer 3-7, 10117 Berlin

Amtsgericht Charlottenburg, HRB 209739 B

Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen

*m:* +491603708037

*w:* aiven.io *e:* matthew.dedetr...@aiven.io

Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Matthias Berndt]

The additional load on the ASF's CI infrastructure may or may not be a
problem, but either way I don't think it is to be addressed within the
scope of this particular change.

Best,
Matthias

Am Mo., 12. Feb. 2024 um 11:37 Uhr schrieb Josep Prat
:

> Hi Matthias,
> If I understand it right (I'm not an expert in CI infra), Kafka uses the
> ASF shared Jenkins, meaning this would potentially increase the
> time-to-built for the rest of the projects.
>
> Best,
>
> On Mon, Feb 12, 2024 at 11:30 AM Matthias Berndt <
> matthias.ber...@ttmzero.com> wrote:
>
> > Hey Josep,
> >
> > wouldn't it be possible to run this in a separate CI job that runs in
> > parallel? Then it shouldn't take any longer than today – assuming enough
> CI
> > resources are available.
> >
> > Best,
> > Matthias
> >
> > Am Mo., 12. Feb. 2024 um 09:14 Uhr schrieb Josep Prat
> > :
> >
> > > Hi Matthias,
> > >
> > > One of the problems of adding support for Scala 3 for the scala-streams
> > > submodule is that we would need to have another build run on CI (which
> is
> > > already extremely long). I guess we could restrict this only when
> having
> > > changes on the Stream module(s).
> > > If I remember it correctly, when I tried to port Kafka to Scala 3 (as a
> > > whole) the sentiment was that only 2 versions of Scala should be
> > supported
> > > at a time. Kafka 4.0.0 will remove support for Scala 2.12. I'll wait
> for
> > > others to chime in. Maybe Ismael has some thoughts about it.
> > >
> > > Best,
> > >
> > > On Sat, Feb 10, 2024 at 1:55 AM Matthias Berndt <
> > > matthias.ber...@ttmzero.com>
> > > wrote:
> > >
> > > > Hey there,
> > > >
> > > > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala
> > library.
> > > > As you might have seen already, I have recently created a ticket
> > > > https://issues.apache.org/jira/browse/KAFKA-16237
> > > > and a PR
> > > > https://github.com/apache/kafka/pull/15338
> > > > to move this forward. The changes required to make
> Kafka-Streams-Scala
> > > > compile with Scala 3 are trivial; the trickier part is the build
> system
> > > and
> > > > the release process
> > > > I have made some changes to the build system (feel free to comment on
> > the
> > > > above PR about that) that make it possible to test
> Kafka-Streams-Scala
> > > and
> > > > build the jar. What remains to be done is the CI and release process.
> > > There
> > > > is a `release.py` file in the Kafka repository's root directory,
> which
> > > > assumes that all artifacts are available for all supported Scala
> > > versions.
> > > > This is no longer the case with my changes because while porting
> > > > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3
> is
> > > less
> > > > so, and shouldn't hold back a Scala 3 release of
> Kafka-Streams-Scala. I
> > > > would appreciate some guidance as to what the release process should
> > look
> > > > like in the future.
> > > >
> > > > Oh and I've made a PR to remove a syntax error from release.py.
> > > > https://github.com/apache/kafka/pull/15350
> > > >
> > > > All the best,
> > > > Matthias
> > > >
> > >
> > >
> > > --
> > > [image: Aiven] 
> > >
> > > *Josep Prat*
> > > Open Source Engineering Director, *Aiven*
> > > josep.p...@aiven.io   |   +491715557497
> > > aiven.io    |   <
> > https://www.facebook.com/aivencloud
> > > >
> > >      <
> > > https://twitter.com/aiven_io>
> > > *Aiven Deutschland GmbH*
> > > Alexanderufer 3-7, 10117 Berlin
> > > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > > Amtsgericht Charlottenburg, HRB 209739 B
> > >
> >
>
>
> --
> [image: Aiven] 
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.p...@aiven.io   |   +491715557497
> aiven.io    |    >
>      <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Josep Prat]

Hi Matthias,
If I understand it right (I'm not an expert in CI infra), Kafka uses the
ASF shared Jenkins, meaning this would potentially increase the
time-to-built for the rest of the projects.

Best,

On Mon, Feb 12, 2024 at 11:30 AM Matthias Berndt <
matthias.ber...@ttmzero.com> wrote:

> Hey Josep,
>
> wouldn't it be possible to run this in a separate CI job that runs in
> parallel? Then it shouldn't take any longer than today – assuming enough CI
> resources are available.
>
> Best,
> Matthias
>
> Am Mo., 12. Feb. 2024 um 09:14 Uhr schrieb Josep Prat
> :
>
> > Hi Matthias,
> >
> > One of the problems of adding support for Scala 3 for the scala-streams
> > submodule is that we would need to have another build run on CI (which is
> > already extremely long). I guess we could restrict this only when having
> > changes on the Stream module(s).
> > If I remember it correctly, when I tried to port Kafka to Scala 3 (as a
> > whole) the sentiment was that only 2 versions of Scala should be
> supported
> > at a time. Kafka 4.0.0 will remove support for Scala 2.12. I'll wait for
> > others to chime in. Maybe Ismael has some thoughts about it.
> >
> > Best,
> >
> > On Sat, Feb 10, 2024 at 1:55 AM Matthias Berndt <
> > matthias.ber...@ttmzero.com>
> > wrote:
> >
> > > Hey there,
> > >
> > > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala
> library.
> > > As you might have seen already, I have recently created a ticket
> > > https://issues.apache.org/jira/browse/KAFKA-16237
> > > and a PR
> > > https://github.com/apache/kafka/pull/15338
> > > to move this forward. The changes required to make Kafka-Streams-Scala
> > > compile with Scala 3 are trivial; the trickier part is the build system
> > and
> > > the release process
> > > I have made some changes to the build system (feel free to comment on
> the
> > > above PR about that) that make it possible to test Kafka-Streams-Scala
> > and
> > > build the jar. What remains to be done is the CI and release process.
> > There
> > > is a `release.py` file in the Kafka repository's root directory, which
> > > assumes that all artifacts are available for all supported Scala
> > versions.
> > > This is no longer the case with my changes because while porting
> > > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is
> > less
> > > so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
> > > would appreciate some guidance as to what the release process should
> look
> > > like in the future.
> > >
> > > Oh and I've made a PR to remove a syntax error from release.py.
> > > https://github.com/apache/kafka/pull/15350
> > >
> > > All the best,
> > > Matthias
> > >
> >
> >
> > --
> > [image: Aiven] 
> >
> > *Josep Prat*
> > Open Source Engineering Director, *Aiven*
> > josep.p...@aiven.io   |   +491715557497
> > aiven.io    |   <
> https://www.facebook.com/aivencloud
> > >
> >      <
> > https://twitter.com/aiven_io>
> > *Aiven Deutschland GmbH*
> > Alexanderufer 3-7, 10117 Berlin
> > Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> > Amtsgericht Charlottenburg, HRB 209739 B
> >
>


-- 
[image: Aiven] 

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.p...@aiven.io   |   +491715557497
aiven.io    |   
     
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B

Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Matthias Berndt]

Hey Josep,

wouldn't it be possible to run this in a separate CI job that runs in
parallel? Then it shouldn't take any longer than today – assuming enough CI
resources are available.

Best,
Matthias

Am Mo., 12. Feb. 2024 um 09:14 Uhr schrieb Josep Prat
:

> Hi Matthias,
>
> One of the problems of adding support for Scala 3 for the scala-streams
> submodule is that we would need to have another build run on CI (which is
> already extremely long). I guess we could restrict this only when having
> changes on the Stream module(s).
> If I remember it correctly, when I tried to port Kafka to Scala 3 (as a
> whole) the sentiment was that only 2 versions of Scala should be supported
> at a time. Kafka 4.0.0 will remove support for Scala 2.12. I'll wait for
> others to chime in. Maybe Ismael has some thoughts about it.
>
> Best,
>
> On Sat, Feb 10, 2024 at 1:55 AM Matthias Berndt <
> matthias.ber...@ttmzero.com>
> wrote:
>
> > Hey there,
> >
> > I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala library.
> > As you might have seen already, I have recently created a ticket
> > https://issues.apache.org/jira/browse/KAFKA-16237
> > and a PR
> > https://github.com/apache/kafka/pull/15338
> > to move this forward. The changes required to make Kafka-Streams-Scala
> > compile with Scala 3 are trivial; the trickier part is the build system
> and
> > the release process
> > I have made some changes to the build system (feel free to comment on the
> > above PR about that) that make it possible to test Kafka-Streams-Scala
> and
> > build the jar. What remains to be done is the CI and release process.
> There
> > is a `release.py` file in the Kafka repository's root directory, which
> > assumes that all artifacts are available for all supported Scala
> versions.
> > This is no longer the case with my changes because while porting
> > Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is
> less
> > so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
> > would appreciate some guidance as to what the release process should look
> > like in the future.
> >
> > Oh and I've made a PR to remove a syntax error from release.py.
> > https://github.com/apache/kafka/pull/15350
> >
> > All the best,
> > Matthias
> >
>
>
> --
> [image: Aiven] 
>
> *Josep Prat*
> Open Source Engineering Director, *Aiven*
> josep.p...@aiven.io   |   +491715557497
> aiven.io    |    >
>      <
> https://twitter.com/aiven_io>
> *Aiven Deutschland GmbH*
> Alexanderufer 3-7, 10117 Berlin
> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
> Amtsgericht Charlottenburg, HRB 209739 B
>

Re: [DISCUSS] Kafka-Streams-Scala for Scala 3

[Josep Prat]

Hi Matthias,

One of the problems of adding support for Scala 3 for the scala-streams
submodule is that we would need to have another build run on CI (which is
already extremely long). I guess we could restrict this only when having
changes on the Stream module(s).
If I remember it correctly, when I tried to port Kafka to Scala 3 (as a
whole) the sentiment was that only 2 versions of Scala should be supported
at a time. Kafka 4.0.0 will remove support for Scala 2.12. I'll wait for
others to chime in. Maybe Ismael has some thoughts about it.

Best,

On Sat, Feb 10, 2024 at 1:55 AM Matthias Berndt 
wrote:

> Hey there,
>
> I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala library.
> As you might have seen already, I have recently created a ticket
> https://issues.apache.org/jira/browse/KAFKA-16237
> and a PR
> https://github.com/apache/kafka/pull/15338
> to move this forward. The changes required to make Kafka-Streams-Scala
> compile with Scala 3 are trivial; the trickier part is the build system and
> the release process
> I have made some changes to the build system (feel free to comment on the
> above PR about that) that make it possible to test Kafka-Streams-Scala and
> build the jar. What remains to be done is the CI and release process. There
> is a `release.py` file in the Kafka repository's root directory, which
> assumes that all artifacts are available for all supported Scala versions.
> This is no longer the case with my changes because while porting
> Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is less
> so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
> would appreciate some guidance as to what the release process should look
> like in the future.
>
> Oh and I've made a PR to remove a syntax error from release.py.
> https://github.com/apache/kafka/pull/15350
>
> All the best,
> Matthias
>


-- 
[image: Aiven] 

*Josep Prat*
Open Source Engineering Director, *Aiven*
josep.p...@aiven.io   |   +491715557497
aiven.io    |   
     
*Aiven Deutschland GmbH*
Alexanderufer 3-7, 10117 Berlin
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
Amtsgericht Charlottenburg, HRB 209739 B

[DISCUSS] Kafka-Streams-Scala for Scala 3

[Matthias Berndt]

Hey there,

I'd like to discuss a Scala 3 release of the Kafka-Streams-Scala library.
As you might have seen already, I have recently created a ticket
https://issues.apache.org/jira/browse/KAFKA-16237
and a PR
https://github.com/apache/kafka/pull/15338
to move this forward. The changes required to make Kafka-Streams-Scala
compile with Scala 3 are trivial; the trickier part is the build system and
the release process
I have made some changes to the build system (feel free to comment on the
above PR about that) that make it possible to test Kafka-Streams-Scala and
build the jar. What remains to be done is the CI and release process. There
is a `release.py` file in the Kafka repository's root directory, which
assumes that all artifacts are available for all supported Scala versions.
This is no longer the case with my changes because while porting
Kafka-Streams-Scala to Scala 3 is trivial, porting Kafka to Scala 3 is less
so, and shouldn't hold back a Scala 3 release of Kafka-Streams-Scala. I
would appreciate some guidance as to what the release process should look
like in the future.

Oh and I've made a PR to remove a syntax error from release.py.
https://github.com/apache/kafka/pull/15350

All the best,
Matthias