Hey there, I'd like to discuss a Scala 3 port of the kafka-streams-scala library. Currently, the build system is set up such that kafka-streams-scala and core (i. e. kafka itself) are compiled with the same Scala compiler versions. This is not an optimal situation because it means that a Scala 3 release of kafka-streams-scala cannot happen independently of kafka itself. I think this should be changed
The production codebase of scala-streams-kafka actually compiles just fine on Scala 3.3.1 with two lines of trivial syntax changes. The problem is with the tests. These use the `EmbeddedKafkaCluster` class, which means that kafka is pulled into the classpath, potentially leading to binary compatibility issues. I can see several approaches to fixing this: 1. Run the kafka-streams-scala tests using the compatible version of :core if one is available. Currently, this means that everything can be tested (test kafka-streams-scala_2.12 using core_2.12, kafka-streams-scala_2.13 using core_2.13 and kafka-streams-scala_3 using core_2.13, as these should be compatible), but when a new scala-library version is released that is no longer compatible with 2.13, we won't be able to test that. 2. Rewrite the tests to run without EmbeddedKafkaCluster, instead running the test cluster in a separate JVM or perhaps even a container. I'd be willing to get my hands dirty working on this, but before I start I'd like to get some feedback from the Kafka team regarding the approaches outlined above. All the best Matthias Berndt
