Re: Should Spark 4.x use Java modules (those you define with module-info.java sources)?

2023-12-04 Thread Sean Owen
It already does. I think that's not the same idea? On Mon, Dec 4, 2023, 8:12 PM Almog Tavor wrote: > I think Spark should start shading it’s problematic deps similar to how > it’s done in Flink > > On Mon, 4 Dec 2023 at 2:57 Sean Owen wrote: > >> I am not sure we can control that - the Scala

Re: Should Spark 4.x use Java modules (those you define with module-info.java sources)?

2023-12-04 Thread Almog Tavor
I think Spark should start shading it’s problematic deps similar to how it’s done in Flink On Mon, 4 Dec 2023 at 2:57 Sean Owen wrote: > I am not sure we can control that - the Scala _x.y suffix has particular > meaning in the Scala ecosystem for artifacts and thus the naming of .jar > files.

Re: Should Spark 4.x use Java modules (those you define with module-info.java sources)?

2023-12-03 Thread Sean Owen
I am not sure we can control that - the Scala _x.y suffix has particular meaning in the Scala ecosystem for artifacts and thus the naming of .jar files. And we need to work with the Scala ecosystem. What can't handle these files, Spring Boot? does it somehow assume the .jar file name relates to

Should Spark 4.x use Java modules (those you define with module-info.java sources)?

2023-12-03 Thread Marc Le Bihan
Hello,     Last month, I've attempted the experience of upgrading my Spring-Boot 2 Java project, that relies heavily on Spark 3.4.2, to Spring-Boot 3. It didn't succeed yet, but was informative.     Spring-Boot 2 → 3 means especially javax.* becoming jakarka.* : javax.activation,