Hello,
Last month, I've attempted the experience of upgrading my
Spring-Boot 2 Java project, that relies heavily on Spark 3.4.2, to
Spring-Boot 3. It didn't succeed yet, but was informative.
Spring-Boot 2 → 3 means especially javax.* becoming jakarka.* :
javax.activation, javax.ws.rs, javax.persistence, javax.validation,
javax.servlet... all of these have to change their packages and
dependencies.
Apart of that, they were some trouble with ANTLR 4 against ANTLR 3,
and few things with SFL4 and Log4J.
It was not easy, and I guessed that going into modules could be a
key. But when I'm near the Spark submodules of my project, it fail with
messages such as:
package org.apache.spark.sql.types is declared in the unnamed
module, but module fr.ecoemploi.outbound.spark.core does not read it
But I can't handle the spark dependencies easily, because they have
an "invalid name" for Java. It's a matter that it doesn't want the "_"
that is in the "_2.13" suffix of the jars.
[WARNING] Can't extract module name from
breeze-macros_2.13-2.1.0.jar: breeze.macros.2.13: Invalid module name:
'2' is not a Java identifier
[WARNING] Can't extract module name from
spark-tags_2.13-3.4.2.jar: spark.tags.2.13: Invalid module name: '2' is
not a Java identifier
[WARNING] Can't extract module name from
spark-unsafe_2.13-3.4.2.jar: spark.unsafe.2.13: Invalid module name: '2'
is not a Java identifier
[WARNING] Can't extract module name from
spark-mllib_2.13-3.4.2.jar: spark.mllib.2.13: Invalid module name: '2'
is not a Java identifier
[... around 30 ...]
I think that changing the naming pattern of the Spark jars for the
4.x could be a good idea,
but beyond that, what about attempting to integrate Spark into
modules, it's submodules defining module-info.java?
Is it something that you think that [must | should | might | should
not | must not] be done?
Regards,
Marc Le Bihan
- Should Spark 4.x use Java modules (those you define with mod... Marc Le Bihan
-
