Re: JPMS whitebox testing issues
Le 2024-10-14 à 10 h 45, Stephen Colebourne a écrit : So you've defined a GAV for Java modules like java.desktop? Good point. I didn't, but in the proposed approach this is something that would need to be done (with version number omitted I guess). Alternatively, it can also be considered another case where module-info duplication is needed, or just rely on the classical way of adding command-line options. Happy to test - I remain unconvinced that an approach based on command line flags derived from existing Maven info is sufficient. Maybe not all case. But I think that it can cover a lot of cases, and makes other cases easier by reducing the number of remaining options that developers need to provide. I have some urgent work to finish in the next couple of weeks, but hope to be back on the compiler plugin after that. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: JPMS whitebox testing issues
Hello Stephen Le 2024-10-13 à 22 h 57, Stephen Colebourne a écrit : Problem 1:Services Already a known problem, as there is no command line flag for services. This cannot be automatically derived as described above Already answered in my previous email: this is one of the few cases where duplicated module-info.java are necessary, and it can be supported with hacks. Problem 2: Extra dependencies One of my projects depends on java.desktop in testing, but not in prod. Since this cannot be expressed in , it cannot be automatically derived as described above As mentioned in my previous email, it is expressed by a dependency with test. The new compiler plugin already supports that. This is handled in the same way as any other test dependencies such as JUnit. If we mean a dependency needed only at test runtime (not test compile time), we may need to introduce a new type of dependency. Problem 3: Extra packages One of my projects has a package in the tests that is not present in the main code. Classes in that package need to be exported (to another module) in order for the tests to pass. This cannot be automatically derived as described above The old way to specify additional compiler options is still available. So it is possible to add explicit --add-exports flags there. If developers need to add only special cases such as this one instead of all --add-exports flags, it is still a better situation than the current one. Problem 4: Export to all Problem 3 involved exporting an additional package to a specific module, which is possible via command line flags. However, there are no doubt cases where the additional package needs to be exported to all module, something which cannot be done via a command line flag. For testing purpose only? If you mean using a different module-info for tests than the module-info for the main code when testing from a different Maven module, it seems to not be possible at all, unless we put the project upside-down (test become main code and main code become patch), or unless we temporarily replace de module-info.class file. This is doable if there is really a need for that. Problem 5: Optional dependencies Various of my projects have optional module dependencies (requires static). It is desirable to test both with and without the optional dependency (multiple runs of surefire). If the optional dependency is always added, that could be problematic. Yes, we will need a mechanism for controlling that. But the problem is the same with the current plugins. Problem 6: Opening for reflection As mentioned in your previous email, some packages may need to be opened for testing. Yes, as said before this is one of the points where the mechanism is yet to be determined. In summary, in this list of problems, I think that 2 or them should already works. 2 other require hacks, even with the current plugins, so the proposed work would hopefully be at least an improvements. The last 2 are known open questions, but not blocking as their solution would be some new configuration options somewhere. The most productive way to resolve those questions may be to have a working prototype and checks which issues are still valid on that prototype. However some issues such as problem 6 are not compiler issues, but Surefire issues. Do we need to have the full stack tested before validating the approach? Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: JPMS whitebox testing issues
Le 2024-10-12 à 00 h 06, Gary Gregory a écrit : That's quite the false equivalence: Lambas and records don't force you to hack JVM start scripts with --add-exports, --add-opens, --add-modules, --add-reads, and --patch-module... We are not supposed to add any such option to a start script. They should be used during tests only, or as temporary workaround for patching a bug. Maybe your configuration is a mix of modular and non-modular JARs, with the non-modular ones missing in the module-info? (because some consider that relying on automatic module naming is a bad practice). Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: JPMS whitebox testing issues
Le 2024-10-11 à 20 h 21, Gary Gregory a écrit : I do not need JPMS and I've never had a request for features around it at work. Our customers care about our products' features and couldn't care less about JPMS. We could also said that "customers couldn't care less about lambdas", or "about records", etc. JPMS is for developers. The benefit for users is only indirect. It would be nice if the JVM had a --stop-complaining-about-jpms Migrating from a non-JPMS to a JPMS environment is a little bit equivalent to migrating from a Unix environment on which we were root, to a Unix environment on which we are an ordinary user with more limited authorizations. Asking for a --stop-complaining-about-jpms option is equivalent to asking "I want to be root again". The world is becoming more concerned about security, it will require some adjustment on our side. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: JPMS whitebox testing issues
Le 2024-10-11 à 20 h 02, Gary Gregory a écrit : JPMS feels like something to workaround, not with :( This is the perception that I hope to change. I happily work with JPMS everyday, but had to move from Maven to Gradle and tweak Gradle a lot of that purpose. Let revisit that evaluation after Maven compiler, surefire, jar and exec have been updated. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: JPMS whitebox testing issues
Le 2024-10-11 à 10 h 21, Stephen Colebourne a écrit : (…snip…, was about module-info duplicated in main and tests) This issue remains hugely painful not just in Maven, but across Java. It is a key part of why JPMS adoption is so painful. I don't think that adoption is painful because of JPMS, since I think that there is good security reasons for the way that it works. I believe adoption is painful because Maven and Gradle are not helping. I hope that the new compiler plugin will demonstrate that (I didn't had the time to work on it during the last month, sorry for that). The key principle here is that tests for a Maven module should be located *within* the same Maven module of the main source code. Forcing developers to write a separate Maven module just for tests is not an acceptable solution. Yes, we agree on that. But it doesn't change with JPMS, there is no need to put the tests in a separated module. Overwriting a module-info in the tests should be a deprecated practice, as there is better alternatives (again, with Maven's help for making that easy). If nevertheless a developer really wants to overwrite the module-info, it is possible with better hacks than what the current plugin does (e.g. we could temporarily move the main module-info). 1) Classpath testing with one (main) module-info.java file. IIUC, this is what you are planning for maven-compiler-plugin and maven-surefire-plugin to support. It tests with the main code on the modulepath and the test code on the classpath. No, this is not what I plan to do. My plan is to test everything on the module path (for modular projects), with a single module-info specified in the main code. The test-only dependencies are handled by --add-modules and --add-reads options added automatically by Maven based on what is declared in POM section (so developers have control). The test classes in the same packages as the main classes are handled by --path-modules options added automatically by Maven. JUnit execution needs --add-opens options to be added by automatically by Maven too (the way to keep that part under developer's control has not been determined yet). So there is usually no need to overwrite the module-info in the test code. There are a few cases where overwriting module-info in test code may be necessary. For example, I'm not aware of any Java option for adding or removing services (the "provides" clause in module-info). So this is a case where overwriting module-info may be necessary, but it should be a minority of cases, to use in last resort only. And as said above, for those cases, hacks are still possible. 2) Blackbox testing with two module-info.java files. This is supported by IntelliJ and Gradle IIUC. Maven must support this IMO. If the two module-info files declare a different module name and there is no package name collision, indeed this would be a good think to add. I will put that on my "TODO" list for a future version. I believe Gradle recommends src/integrationTest for blackbox tests, and I think that convention makes a lot of sense. It offers up the potential for a single Maven module to contain both whitebox and blackbox tests, which is incredibly powerful. Indeed, I agree we should support something like that out-of-the-box. 3) Whitebox testing with two module-info.java files. This has yet to achieve an agreed approach in the community, despite JPMS being out for many years. If the two module-info files declares the same module name with the intend that the test one overwrite the main one, we can do with a hack different than the current one (without putting the project upside-down). But it should be a deprecated practice, to use in last resort only. For the majority of cases, we don't need two module-info for whitebox testing with everything on the module path (see point 1 above). I understand that you want to make defining patch-module directives easier in pom.xml. While that's a lovely idea, I think it is fundamentally flawed in usability. There is no connection in the mind of a developer between module-info directives and the matching command line arguments. By contrast developers know exactly how to setup the module-info.java file. I'm not proposing to add a new syntax in pom.xml for patch-module directives. Every time that a test directory has the same packages as the main code, a patch-module directive must be added. The rule is clear and simple because otherwise the JVM refuses to launch. So patch directives are added automatically with no need for the developer to think about that. For the --add-modules and --add-reads directives, this is also automatic with clear rules based on the scope of dependencies. The only directives at this time for which I do not yet have a rule that I can map to existing Maven concepts is --add-opens. Martin
Re: JPMS whitebox testing issues
Hello Stephen Thanks for testing. Le 2024-10-02 à 00 h 30, Stephen Colebourne a écrit : At present, the compile phase is asked to compile module-info.java, but the target/classes directory does not contain module-info.class. I've no idea where it is going I'm afraid. Is module-info missing for both test and main, or only for test? the current prototype does not allow anymore to have module-info in both main and test. The latter is replaced by automatically generated compiler options. I don't believe this is the right direction to go. The command line flags are difficult to understand, and connect to the matching meaning in module-info.java. By contrast, simply specifying a testing module-info.java directly expresses what needs to be expressed. Furthermore, IntelliJ already supports BlackBox testing in src/test/java, plus it is the recommended approach on the Maven website, so I don't think it is viable to remove that. The problem with module-info in both tests and main is that it works awkward. The previous compiler plugin had to took the project structure upside-down: the test code was considered the main code, and the main code was considered the patch. I just checked by compiling and running a dummy project with `java` and `javac` directly (without Maven) and I confirm: even if a `module-info` is present in the test output, it is ignored by the `java` command (the `module-info` of the main code has precedence) unless we pretend that the test are the main code and the main code are patching the tests. The behavior of `javac` is a little bit more complicated, but basically we also have to put things upside down to make it works. If nevertheless we still want to support having `module-info` in both main and test, I found a trick this morning for getting `javac` to compile the test `module-info` without putting the project structure upside-down. But it is undocumented and I don't know how reliable it would be. I can implement this trick (or try other tricks) if this is important, but I think that we should stop recommending the approach of overwriting `module-info`, or recommend to use it in last resort only. The alternative (compiler options) is indeed tedious, but this is precisely a major goal of this JPMS work to make that task easier than it currently is. As a separate point, it is very hard to know what the plugin is doing. AFAIK, the only approach to finding out what is going on is to use -X which produces far too many logs. Indeed, the current beta-2 compiler generates a `javac` file somewhere in the `target` directory with the arguments that it used, but only if the compilation fails or if Maven was launched with the `--debug` flag. My rational was that the developer may not be interested in details when the compilation is successful. But we can of course revisit that policy. Otherwise, you can ignore the verbose logs of -X and just look for that `javac` file (I forgot the exact name) in `target`. (And as a side note, I agree that Maven is too verbose, even in non-debug mode). * In project dependencies, add modular-jar for forcing the compiler to put the JAR dependency on the module path instead of trying to guess. * Conversely, use classpath-jar for forcing the compiler to put the JAR dependency on the class-path. I can't see many people wanting to set these. They are also restricting. (I believe use of these setting would mean that I cannot be used to run the same tests three times, as I am currently doing with Maven 3) The current Maven 3 behavior is still supported: jar (without specifying "modular-jar" or "classpath-jar") behaves as today, with the same heuristic rules. The "jar" type is still the default, so being specific is an "opt-in" feature. Regarding white-box testing in a modular project: * The plugin uses the --patch-module option for specifying the directories of test classes to the Java compiler. * The plugin adds the following options for test dependencies that are not present in the main dependencies (typically JUnit): o --add-modules and --add-reads for compile-time dependencies. o --add-modules only for runtime-time dependencies (typically services to be discovered with java.util.ServiceLoader). o --add-reads ALL-UNNAMED if at least one compile-time test dependency is not modular. You also need to ensure that - src/main/resources and src/test/resources are correctly patched - the module-info.java `requires static` directive is correctly followed. Yes, but those last points apply more to the surefire plugin than the compiler one. We did not started the work on surefire yet. Martin
Re: JPMS whitebox testing issues
Hello again Stephen If you are willing to test the development regarding JPMS, there is the steps: * Download Maven 4.0.0-beta4 * Use Maven 4 for building the master branch of https://github.com/apache/maven-plugin-tools * Use Maven 4 for building https://github.com/apache/maven-plugin-testing/pull/44. Use "mvn install -Denforcer.skip". * Use Maven 4 for building https://github.com/Geomatys/maven-compiler-plugin/tree/beta2. Use "mvn install -Denforcer.skip". * Set the compiler version to "4.0.0-beta-2-SNAPSHOT" in your project. * See https://github.com/Geomatys/maven-compiler-plugin/wiki/Incompatible-changes necessary changes in project configuration (note: I was wrong yesterday, the current prototype does not allowed anymore to have module-info in both main and test. The latter is replaced by automatically generated compiler options). Below are the new features. They are not all tested extensively. If they are not working as expected, I would like to know so we can fix them. Note: the following apply to the compiler plugin only. Other plugins have not yet been updated. * In project dependencies, add modular-jar for forcing the compiler to put the JAR dependency on the module path instead of trying to guess. * Conversely, use classpath-jar for forcing the compiler to put the JAR dependency on the class-path. * The default (jar) tries to guess with the same heuristic as the current compiler (Maven 3), but I would recommend to be explicit instead. * Use modular-processor or classpath-processor for adding the dependency on the annotation processor module-path or class-path. This is in replacement for the previous compiler configuration: dependencies are managed in a more unified way (same will apply for taglet, doclet, etc.). Regarding white-box testing in a modular project: * The plugin uses the --patch-module option for specifying the directories of test classes to the Java compiler. * The plugin adds the following options for test dependencies that are not present in the main dependencies (typically JUnit): o --add-modules and --add-reads for compile-time dependencies. o --add-modules only for runtime-time dependencies (typically services to be discovered with java.util.ServiceLoader). o --add-reads ALL-UNNAMED if at least one compile-time test dependency is not modular. Planed for Surefire (not yet done): * The plugin will adds the same --add-reads and --add-modules options as the compiler, maybe using shared Maven code for ensuring consistency. * The plugin will adds --add-exports module/package=junit (or whatever test engine is used) options for non-exported packages that have test classes. Above will be ajusted according experience and feedbacks. Martin
Re: JPMS whitebox testing issues
Hello Stephen Just a very quick reply, as I have to go. But I will try to provide more details tomorrow. I agree with the problem that you describe, and a massive amount of work is ongoing for improving Maven support of JPMS, including whitebox testing. The Maven compiler plugin has already been fully rewritten in a separated repository [1] but not yet merged on the Maven repository. One of the change is to get ride of Plexus compiler and uses directly the javax.tools compiler API, in part because this API provides specialized methods for JPMS. While having two module-info is still supported (I think, it would need extensive testing), I'm more inclined in having only one module-info in the main source code and letting the compiler and JUnit generates automatically the --patch-modules, --add-exports, --add-reads, etc. options. We will try to do that in documented and predictable way, as one goal of this rewrite is to avoid as much "black magic" as we can. This work will require Maven 4. Some changes are in progress in support for that. One of the change is that Maven 4 gives full control to the developer about whether to put a dependency on the class-path or module-path. This part is already in Maven core, but not yet honored by plugins, except the compiler prototype. Another work in progress right now is proposing to change the way that we declare the sources in Maven, for supporting both module source hierarchy and (as a side effect) more easily multi-release projects. I will try to give more details tomorrow. If you were available for testing and feedback, that would be greatly appreciated. However at this time, only the compiler plugin has been rewritten. We will also need to update Surefire, but this work hasn't started yet. Martin [1]https://github.com/Geomatys/maven-compiler-plugin/commits/beta2
Re: Question on shading and missing dependencies
Le 2024-05-13 à 22 h 52, Piotr P. Karwasz a écrit : If the CycloneDX Maven plugin learns to use those SBOMs as metadata source instead of POM files, your problem should be solved. I'm not familiar with CycloneDX, but I think that if any SBOM is used with a shaded artifact, then the metadata should said that the dependencies have been transformed that way. For modular dependencies, shading the artifact has major impacts: it breaks modules encapsulation, potentially creating security holes that did not existed in the original libraries. For non-modular dependencies, the impacts are typically smaller, but not necessarily null. For example, a library could perform security checks based on package names, and those checks may become invalid if the packages have been renamed. It does not mean that SBOM are useless, but I think that shading is a very significant transformation that should be declared in the metadata, and not give the impression that the dependencies are in their original form. Martin
Re: [VOTE] Require Java 17 for Maven 4
+1 (non binding) Java 17 is sufficient for resolving the compilation error that initially brought this discussion (HTML heading out of sequence). For that specific problem, Java 17 or 21 are both fine (but Java 11 was not). Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: maven-compiler-plugin adds all dependencies to the module-path when executing test-compile when it shouldn't
Hello Siddharth Le 2023-10-13 à 00 h 02, Siddharth Jain a écrit : IIUC, this is a separate issue. At minimum we need to make the behavior of compile vs. testCompile consistent. Currently it is not. Understood, but what I think is the common point is that Maven currently uses heuristic rules for making its decision. The current heuristic rules include the two points listed in MNG-7855, but I'm not sure that they are the only points. I suspect that there is more rules, intentional or not, depending on what the code is compiling. MNG-7855 can also be understood as "replace heuristic rules by developer control", which implies more predictable determinism. Martin
Re: maven-compiler-plugin adds all dependencies to the module-path when executing test-compile when it shouldn't
Hello Siddharth I do not have a precise answer to your question, but below is a few notes. Le 2023-10-12 à 23 h 07, Siddharth Jain a écrit : I have observed the maven-compiler-plugin behaves differently for test vs. main compilation. Which approach did you choose for the tests? One approach is to duplicate the module-info.java file in the test directory [1], but this is not the only way. I see during the test-compile phase the plugin is putting all the dependencies on the module-path whereas during the main compilation it only puts those dependencies on the module-path that are declared in module-info.java; the rest are on the classpath (which is the correct behavior). Indeed this is the correct behaviour in a lot of cases, but not always. Sometime we really want non-modular dependencies to nevertheless be on the module-path. So we need a way to give control to developers about which dependencies to put on the module-path versus class-path. This is the topic of JIRA issue MNG-7855 [2]. We had a little bit of discussion about that during the "Community Over Code" meeting (formerly "ApacheCon") that just finished two days ago. A proposal was to create a wiki page for designing a short-term fix. Is this a known issue? Is there any fix? Without it, how can we use JPMS? I think that MNG-7855 is a blocker issue. Without a fix, doing gradual migration to JPMS with Maven (i.e. being in a situation where we have a mix of JPMS and non-JPMS modules) is difficult except in some special cases. Martin [1]https://maven.apache.org/surefire/maven-surefire-plugin/examples/jpms.html [2]https://issues.apache.org/jira/browse/MNG-7855
Re: How to force Maven to put a dependency on the module-path?
Hello Thorsten
Le 02/08/2023 à 11:47, Thorsten Heit a écrit :
I see your point, but we're not living in a fully modularized Java
world: If you insist on having a Jar reside on the module path only,
this prevents others from using it on applications that are not fully
modularized or that cannot be modularized (yet).
No, an application does not need to be modularized for putting
dependencies on the module-path. This is what I tried to explain in my
yesterday's email. All applications, modularized or not, are already
using dependencies on the module path: the JDK itself. We can test that
with the following code:
System.out.println(String.class.getModule().getName());
The idea that a non-modularized application must put all its
dependencies on the class-path seems to be a widely-spread
misunderstanding of how JPMS works. Putting a library on the module-path
does not force an application to be modularized. Admittedly there is
some cases where a developer way want to use the class-path anyway, but
it should be under developer's control.
Furthermore I don't really see a problem of duplication. Normally you
don't have tons of service provider implementations in one Jar (IMO)
so this shouldn't be too much hassle to have a few lines in your
module descriptor AND in the provider configuration file under
META-INF/services...
In Apache SIS, there are 76 providers for coordinate operations (e.g.
map projections) alone, not counting other services. This is not huge,
but the risk of inconsistency exists. Furthermore, as explained
yesterday, it blocks us from providing singletons, so we have to
workaround with wrappers. Finally, contrarily to module-info,
META-INF/services/ files get no compile-time verification. JUnit tests
does not always help in discovering for example that a
META-INF/services/ file was in the wrong module (a mistake that we
discovered precisely thanks to the migration to module-info).
To be compatible with fully-modular applications and those that aren't
I'd extract the necessary parts from the provider implementation into
a singleton class and use that one instead. In this case you still
could have multiple instances of your provider implementation, and all
of them internally use a singleton for configuration data or whatever...
Yes this is the above-mentioned wrapper pattern. But things become more
complicated when some providers may implement more than one public
interface and we don't want to block advanced users from accessing those
additional API if they need to.
Can you show an example that depends on such methods? I personally
don't see a reason why a library should behave different if it is on
the module path or class path...
There is 167 occurrences of the @CallerSensitive annotation in OpenJDK
source code as of today. Not all of them are module-related, but many
are, especially in tasks involving reflections. When a modularized
library is put on the module-path, encapsulation is enforced. So for
example a call to the following code:
InputStream in = Foo.class.getResourceAsStream("Something.properties");
returns a non-null value (assuming the file exists) when the library
invoking above code is on the class-path, but a null value if the same
library is on the module-path and Foo is a class declared in a
non-exported package of another module. The same restriction applies to
all variants such as ClassLoader.getResource(String), there is no way to
bypass the encapsulation. It may look likes a regression, but the key
point is that the behavior is not the same and a library could very well
depend on encapsulation being enforced. Other methods annotated with
@CallerSensitive are some logging methods. I didn't tested but I see
that the source code uses module information, so I would not be
surprised if logging messages are formatted differently.
IMHO a library is buggy if it's *behavior* depends on whether it is
being used on the module path or on the classpath.
With 167 OpenJDK methods annotated with @CallerSensitive, there is
plenty of room for behavioral differences even if the library developers
where very careful. I think it is okay for a library to said "I have
been designed for working as a module, please put me on the
module-path". As said above, it does not force applications to be
themselves modularized. There is possibly different opinions about what
is best (and it may depend on the library), but the current problem is
that Maven imposes its black magic on everyone with no way I can see to
specify what we need.
Martin
Re: How to force Maven to put a dependency on the module-path?
Hello Garret Le 01/08/2023 à 18:32, Garret Wilson a écrit : On 7/26/2023 1:42 PM, Martin Desruisseaux wrote: … If a dependency is on the classpath, then the dependency is loaded as an unnamed module, its "module-info" file is ignored and the services that it contains are not discovered. Can you elaborate on the last point a little more? I haven't modularized my core libraries yet, and want to know how they would work with non-modularized applications. There is a very simple test case here, for both Maven and Gradle, with a README that tries to explain the problem. I tried to make the most trivial "hello world" reproducing the problem. The test case can also be run by invoking java directly on the command-line, which makes easy to experiment different Java options for understanding the problem: https://github.com/Geomatys/MavenModulepathBug The problem can be reproduced with the following command-line (omitting paths and version number for simplicity). This command-line reproduces what Maven and Gradle do: java --class-path service.jar:client.jar test.client.Main In this example: * "client" is the main application and is not modularized. * "service" is a modularized dependency. In this JAR, I put four services: o 2 services in "module-info.class" o 2 services in "META-INF/services/" but with different texts, so we can see which services is loaded. If you run above command-line, you will see the following text: |Start searching for services... Provider B declared in META-INF. Provider C declared in META-INF. Done. The dependency has been loaded as an unnamed module. Consequently its `module-info` file has been ignored, and the `META-INF/services` directory is used instead.| The call to "java.util.ServiceLoader" was done from the "service" JAR file, which was supposed to be modularized. Despite that fact, we see that "module-info.class" has been ignored and "META-INF/services/" used instead. In other words, the modularized "service.jar" is unable to access to its own "module-info.class". We are not even talking about the behavior of the non-modularized "client.jar" file here. The correct behavior can be reproduced with the following command-line (again omitting paths and version numbers). The main difference is that the modularized "service.jar" file is put on the module-path instead than the class-path. I will skip discussion about the "-add-modules" option for this email. java --module-path service.jar --class-path client.jar --add-modules ALL-MODULE-PATH test.client.Main The output is then: |Start searching for services... Provider A declared in module-info. Provider D declared in module-info. Done. The dependency has been loaded as named module. Great! This is what we need for the `module-info` to be used.| This time "module-info.class" has been used and "META-INF/services/" ignored (as it should be). I claim that it should be the Maven and Gradle behavior. If this claim is disputed (e.g. for compatibility reason), then at least it should be configurable. The fact that there is no way I could find for overriding the (in my opinion broken) Maven behavior is what I call the bug. Note that this is not a JPMS issue, this is a Maven/Gradle issue. Maven and Gradle uses the following heuristic rules for deciding if a dependency should be put on the module-path: 1. the dependency is modularized (i.e. contains a module-info.class file or an Automatic-Module-Name attribute in MANIFEST.MF), and 2. the project using the dependency is itself modularized. I claim that condition 2 should be removed, i.e. a modularized dependency should be put on the module-path regardless if the project using it is modularized or not. Or if condition 2 is not removed, then at least the developer should have some way to control that. You say that if the modularized library is put on the classpath, its services are not discovered. But wouldn't normal pre-module-era classpath-based service discovery still work via the `META-INF/services` configurations in the library JAR files? Please clarify what won't work if I modularize my core libraries, which contain services to be discovered, and I try to use that with a non-modularized application. "META-INF/services/" is not necessary for compatibility with non-modularized applications. It is necessary only for environments that decide to put the modules on the class-path rather than the module-path (reminder: putting dependencies on the module-path works as well with non-modules client application). Maintaining both "module-info.class" and "META-INF/services/" with identical content is duplication, with risk of inconsistent behavior if their content accid
Re: How to force Maven to put a dependency on the module-path?
Hello again I wrote a test case with a very small Maven project (only trivial code) reproducing the problem and a README.md file explaining the problem. The Maven behavior is then reproduced on the command-line, and the expected behavior provided by a command-line too. Unless there is some configuration options that I didn't see, the current Maven behavior regarding module-path seems to be a quasi-blocker for gradual migration of large projects to JPMS. https://github.com/Geomatys/MavenModulepathBug The ideal solution for me would be to have explicit control on whether to put a dependency on the module path versus the class-path. Is there some way to do that that I have missed? Thanks, Martin Le 26/07/2023 à 18:42, Martin Desruisseaux a écrit : Hello If I'm understanding right, Maven put a dependency on the module-path instead of the class-path only if: 1. the dependency is modularized (contains a "module-info" file), and 2. the project using the dependency is itself modularized. Condition #1 is fine, but #2 is problematic. If a dependency is on the classpath, then the dependency is loaded as an unnamed module, its "module-info" file is ignored and the services that it contains are not discovered. Not only the services that the non-modularized project may be looking for (I understand that it may not work), but the modularized dependency itself cannot find anymore its own services, or the services of its modularized dependencies, if those dependencies were not on the module-path. Is there a way to force Maven to ignore condition #2 and use only condition #1 for deciding whether to put a dependency on the module path or not? Thanks, Martin
How to force Maven to put a dependency on the module-path?
Hello If I'm understanding right, Maven put a dependency on the module-path instead of the class-path only if: 1. the dependency is modularized (contains a "module-info" file), and 2. the project using the dependency is itself modularized. Condition #1 is fine, but #2 is problematic. If a dependency is on the classpath, then the dependency is loaded as an unnamed module, its "module-info" file is ignored and the services that it contains are not discovered. Not only the services that the non-modularized project may be looking for (I understand that it may not work), but the modularized dependency itself cannot find anymore its own services, or the services of its modularized dependencies, if those dependencies were not on the module-path. Is there a way to force Maven to ignore condition #2 and use only condition #1 for deciding whether to put a dependency on the module path or not? Thanks, Martin
Re: Modular path vs. class path determination
Le 17/10/2022 à 12:33, Olivier Lamy a écrit : I wanted to have some opinions on what sort of configurations we could add but this didn't get much attention :) (https://issues.apache.org/jira/browse/SUREFIRE-2097) Maybe nobody really cares of jpms... On my side I do care a lot about JPMS. But I got the feeling that Maven model is not well suited to JPMS, which caused me to stop following JPMS issues on Maven and look for other build solution. The difficulty I have with JPMS in Maven is that I would like to break the "one Maven module = one JPMS module" relationship. Instead I would like "one Maven module = one compilation unit", i.e. a single call to javac, javadoc or other java tools, which may itself comprise many JPMS module. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Why maven-compiler-plugin seems to not take advantage of already compiled .class files?
Le 01/12/2018 à 21:17, Robert Scholte a écrit : > This is a dangerous advice. > > Yes, it'll only compile the touched files, but not the source files > using it. For example method signature changes are not detected and > you will hit that issue at runtime with a NoSuchMethodException, which > is too late > Yes I know. But running "mvn clean install" when we know that we have changed the API, or when we get a NoSuchMethodException, is a small price to pay for the gain in build speed. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Why maven-compiler-plugin seems to not take advantage of already compiled .class files?
Hello Francesco Have you tried following configuration? It seems counter-intuitive, but last time I tried it made compilation much faster (i.e. it seems to have the opposite effect of what we would expect): maven-compiler-plugin false Martin
Re: NetBeans - the UI for Maven
Le 24/11/2018 à 16:39, Robert Scholte a écrit : > **If And Only If** you want to make use of single tool invocation for > all you JPMS modules, then you cannot use Maven, it's architecture > doesn't support it and any plugin trying to solve this is a hack. > But isn't what you are going to do for fixing MJAVADOC-449 (aggregated javadoc), or is there another way? > Might be possible to add support for it in Maven 4 or 5, but focus is > on other improvements that block us right now. > Understood. Thanks! Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: NetBeans - the UI for Maven
Le 24/11/2018 à 15:10, Robert Scholte a écrit : > Today I started looking at MJAVADOC-449 again and it seems that just > nobody took serious time to solve this. I've been able to create the > proper commandline by moving some classpath entries to the modulepath. > Now it is a matter of implementing this. > Great, thanks! Indeed, this is what I described in https://github.com/Geomatys/maven-plugins/wiki#changes-to-maven-plugins - that implementing this functionality seems basically a matter of replacing some "--classpath" by "--module-path" and some "--source-path" by "--module-source-path" options. However while Javadoc is currently the main issue, the same capability is desirable for other tools too, which is why I would like to address this issue at a more fundamental level (the directory layout). >>> As response Remi Forax wrote Pro[1], which is build upon these new jdk >>> >>> tools features combined with the concepts of Maven. >>> >> Yes I'm aware of that, and my proposal is in part bringing some of Remi >> >> Forax's idea into Maven. >> > I'll wait for the PRs :) > Cool! I don't know yet when I will have time to start my experiment, but I would let you know when I will have a proposal that can be shown. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: NetBeans - the UI for Maven
Le 24/11/2018 à 13:53, Robert Scholte a écrit : > With the Java Platform Modular System you'll clearly see different > requirements between library and application developers. > Indeed (e.g. jlink is for application developers), but the requirements I'm talking about are for library developers. > Keep in mind that 1 pom-file can represent only 1 module. If you want > to distribute multiple modules, you will still need a multimodule > Maven project. > Yes and no. 1 pom-file can represent only one artifact. But the rest of the discussion assumed that 1 artifact = 1 module, or that 1 Maven module = 1 Jigsaw module. This relationship does not necessarily hold. >From recent discussion on OpenJDK public mailing list [1]: I'm sure the topic of executable modular JARs files be back on the table some day but it needs to be looked at in conjunction with the issues around having multiple modules in the same artifact. They would be likely be work needed in several Maven plugins and other parts of the eco system too. In the mean-time, it's not too hard to package multiple modules into the same JAR file and have a main class that creates a configuration and module layer from the modules in the JAR files. Another possibility is to generate the pom.xml files for each Jigsaw module, by intersecting the parent pom.xml with module-info. > Single invocation only makes sense for an application that won't share > modules. > No - a single javac invocation on a multi-modules projects still generate .class files in separated directories for each Jigsaw module, which result in one JAR file per module. I good demonstration of that is NetBeans modular projects (based on Ant), which does exactly that. I would like to see the same capability in Maven. > So yes: single invocation is an option for the jdk tools but doesn't > fit in the Maven architecture and is probably in general not that needed. > It does not fit with Maven standard directory layout. But with a change of layout and corresponding changes in Maven plugins, I do not see major obstacle (but of course I do not have your deep knowledge of Maven internal). However I would not said that this is generally not needed. We have at least MJAVADOC-449. Together with other issues, for me the choice is between an evolution of Maven (possibly with custom plugins) or migrating to another build system. Furthermore other Jigsaw evolution may come (e.g. above-cited multiple modules in same artifact). If Maven does not become more Jigsaw-friendly, the amount of difficulties may increase in the future. > There are no plans to make these changes in Maven. > > As response Remi Forax wrote Pro[1], which is build upon these new jdk > tools features combined with the concepts of Maven. > Yes I'm aware of that, and my proposal is in part bringing some of Remi Forax's idea into Maven. > No, module-path is not the issue, but module-source-path is, which > expects an extra folder reflection the module-name. > Indeed - this is the correction email I posted after my previous email. > I don't understand this issue. As a java developer wanting to add > module descriptor, you only have to create > src/main/java/module-info.java. You can build your project as you are > used to. > Adding module-info.java is sufficient for generating the JAR file, but this is not the end of development process. We also needs to generate javadoc. We sometime want to run annotation processing tools. Those extra tasks are where difficulties arise. There is also a circular dependency problem that would be solved with my proposal, but this one is admittedly quite specific to a project. Martin [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-November/056839.html [2] https://issues.apache.org/jira/browse/MJAVADOC-449
Re: NetBeans - the UI for Maven
Le 24/11/2018 à 13:28, Martin Desruisseaux a écrit : > The key part is "--module-path". Sorry I mean "--module-source-path" for the compilation and javadoc generation parts. Likewise I mean "--source-path" instead of "--classpath" for compilation with pre-jigsaw options. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: NetBeans - the UI for Maven
Hello Jan Le 24/11/2018 à 12:12, Jan Lahoda a écrit : > FWIW, there is StandardJavaFileManager.setLocationForModule: > https://docs.oracle.com/javase/10/docs/api/javax/tools/StandardJavaFileManager.html#setLocationForModule(javax.tools.JavaFileManager.Location,java.lang.String,java.util.Collection) > which might be useful in some cases. > Thanks, I was not aware of this method. It may indeed be useful for (if I understood correctly) plugins invoking the compiler from Java code. I will keep that in mind if I have a chance to do experiments. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: NetBeans - the UI for Maven
Hello Enrico Le 24/11/2018 à 12:13, Enrico Olivelli a écrit : > Have you already shared your thoughts and needs with Apache Maven group ? > Yes, on the mailing list. My feeling is that in order to be convincing, I need to create a prototype showing the feasibility of my proposal. This is the intent of https://github.com/Geomatys/maven-plugins - now I need to find time to work on it. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: NetBeans - the UI for Maven
Hello Robert Thanks for your reply. Le 24/11/2018 à 12:36, Robert Scholte a écrit : > Let me correct this part: there is *no* need to change the folder > structure to work with the Java Platform Modular System. The only > thing you need to do is add a module-info.java to src/main/java and > ensure you're using maven-compiler-plugin-3.8.0. That's it! > My email was saying that there is a need to change the folder structure for using Java Platform Modular System with "--module-path" options. The key part is "--module-path". What maven-compiler-plugin-3.8.0 does is to use the "--class-path" options, which are the legacy options before Jigsaw. The consequence is that it is currently not possible to build more than one Jigsaw module in a single invocation of "javac", "javadoc", "jlink" or other JDK tools with Maven. Consequently I'm not aware of any way to generate a multi-modules javadoc, or to execute annotation processors using information from more than one module (except by generating temporary files), or any other task for which there is advantages in being able to use standard JDK tools with their new "--module-path" options. What I want is to be able to use Maven with Java "--module-path" options instead than "--class-path". This is where the Maven standard directory layout become an obstacle, because the "--module-path" option imposes a directory layout which is different than the Maven's one. I realize that not everyone may want to build a multi-modular project using "--module-path" option. But before Java 9, Maven made the use of javac easier without blocking the use of other options when desired. Since Java 9, we have a difficulty for which I have not yet found a satisfying solution other than the proposed Maven changes. > The key library which might(/should!) be interesting for you is > Plexus-Java [1], which calculates the modulepath. I have not looked in details to Plexus-Java. Does it create a path for each Maven module? That would be a workaround, but it still more complicated than using Jigsaw in its "natural" way. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: NetBeans - the UI for Maven
I think differently. In Apache SIS for example, we maintain both a Maven and Ant project. The root source code directory is a classical Maven project with pom.xml file [1], but we also maintain a sub-directory with NetBeans Ant project configuration [2]. The official project configuration is the Maven one, but the Ant project configuration is keep in sync and takes its dependencies directly from the ~/.m2/repository directory. This means that the project needs to be built with Maven once before we can use the Ant project. We do that for the following reasons: * NetBeans Ant project builds faster. * Application and debugging sessions start faster in NetBeans Ant project. This is very convenient when we do a lot of "modify - test" cycles. * Allow to commit configurations that can not be done in Maven (header license, hints, additional words for check speller). * NetBeans Ant Jigsaw project configuration offers better compatibility with Jigsaw (more on it below). A major issue is that since Jigsaw, Maven standard directory layout is incompatible with java, javac and javadoc "--module-path" option, which is making me look for alternatives (Gradle?). I describe the issue at [3]. I would like to try changes in maven-compiler, maven-javadoc, maven-jar and maven-deploy plugins for supporting Jigsaw, but did not had the time to start yet. As far as I can see, I can not modularize my projects the way I want with Maven at it stands today. I also feel more concerned about Maven standard directory layout not only because of its incompatibility with Jigsaw module paths, but also because it does not seem well suited to project mixing two or more languages. For example in the given directory layout: * src/main/java * src/main/cpp * src/main/resources Does the resources applies to Java, to C++ or both? So Maven is a great tool, but its speed is not yet the same than the NetBeans Ant project configuration (admittedly it may be improved with time) and - more importantly - I think that the Maven standard directory layout needs to be modified at least for Jigsaw compatibility, maybe for better multi-language support. Regards, Martin [1] https://github.com/apache/sis [2] https://github.com/apache/sis/tree/master/ide-project/NetBeans/nbproject [3] https://github.com/Geomatys/maven-plugins/wiki
Re: Project Jigsaw Multi-module Compilation
Le 22/09/2018 à 20:39, Robert Scholte a écrit : > You can't. > > The Java 9 multiple module compilation doesn't fit into the Maven model. > > For Maven, every deliverable requires its own pom, hence a separate > Maven module. > Or alternatively, we can try to get Maven to evolve. Otherwise we can't have aggregated javadoc (unless I missed something), or cross-module annotation processing. Idea has been drafted there: https://github.com/Geomatys/maven-plugins/wiki . I didn't had the time to work on the plugins yet, but I hope to start after ApacheCon is over. Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: maven-compiler-plugin JPMS module resolution
Hello Robert Le 11/04/2018 à 18:45, Robert Scholte a écrit : > I am not aware of such problem. Did you create a Jira[1] issue for it? > Done: https://issues.apache.org/jira/browse/MCOMPILER-336 Thanks Martin - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: maven-compiler-plugin JPMS module resolution
Hello Robert Le 11/04/2018 à 18:45, Robert Scholte a écrit : > I am not aware of such problem. Did you create a Jira[1] issue for it? > > There should be no reason to have a workaround, all the information is > there so Maven is capable to divide jars properly over both paths. If > you can add a project so I can reproduce the issue, I can have a look > at it. > I have not yet created a JIRA issue. I tried last week to create a minimal test case reproducing the issue, without success. The issue happen with the real project; I have not yet isolated all the conditions that make the issue happen. The issue happen with GeoAPI [1]. The following works fine (assuming a JSR 385 2.0-SNAPSHOT is available on the local repository): mvn clean install -Pjdk9 But running the following immediately after: mvn install -Pjdk9 cause the following error message: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.7.0:compile (default-compile) on project geoapi: Compilation failure [ERROR] module not found: java.measure Executing in debug mode, we can see that the java.measure (JSR-385) dependency is correctly put in modulepath when maven-compiler-plugin compiles everything (e.g. when the "clean" phase is present), but not when the plugin compiles only the classes that have been modified and that set of classes do not include module-info.java. For example the following work touch geoapi/src/main/java9/module-info.java mvn install -Pjdk9 But maven-compiler-plugin then fails on the next module, unless I touch the module-info.java of that next module too, /etc./ I will see if I can do a new attempt at creating a minimal test case later this week. Martin [1] https://github.com/opengeospatial/geoapi
Re: maven-compiler-plugin JPMS module resolution
I also have the case where maven-compiler-plugin puts a dependency in classpath while it should be in modulepath. Strangely, the plugin does the correct thing when executed with "mvn _clean_ install" but not when executing "mvn install" without clean. This issue happen only when the useIncrementalCompilation option is set to false. In addition the javadoc plugin also seems to sometime put dependencies in unexpected classpath/modulepath option. A Maven option for specifying whether a dependency should be in the classpath or modulepath would be convenient as a workaround… Martin Le 11/04/2018 à 13:14, Roman Grigoriadi a écrit : > Thanks Robert, > > I tried to add requires for name which is resolved by JPMS for my > non-modularised jar, and it *somehow* made compiler-plugin to put this jar > on module-path. > > However if I don't want to reference this jar in my module descriptor, so I > let decision up to user if to put this dependency on CP or MP for app > runtime, would it make sense to add configuration parameter for > compiler-plugin which will force CP/MP for some of the dependencies during > compilation? > > The best solution I have found so far is to copy such dependencies > somewhere before compilation and pass --module-path, --add-modules and > --add-reads to javac.
Re: How can I emit the SNAPSHOT version after deploy?
Hello Jeff I also wanted to do something similar a while ago. But in my attempt in a multi-modules project, each modules got a slightly different timestamp. Building from the root of a project having modules A, B and C gave me the following timestamps: A: 20150702.165421-17 B: 20150702.165432-17 C: 20150702.165454-17 It looks like that Maven generates an individual timestamp from the time at which each sub-module is built, instead that a unique timestamp for the whole project. This make difficult the usage of a single property for the timestamp portion if the project has more than one module. Martin Le 21/08/15 11:08, Jeff a écrit : > I should clarify that I want to find the specific SNAPSHOT version assigned > by Nexus (or other maven repository manager applications) after the deploy. > > So after I deploy 'myapp.jar' to Nexus with GAV: > > groupID: mycompany.com > artifactID: myapp > version: 1.0-SNAPSHOT > > > Nexus assigns it to something > like /com/mycompany/myapp/1.0-SNAPSHOT/myapp-1.0-20150821.000538-120.jar > > I want to get the "20150821.000538-120" portion and also if possible the > SHA1 hash Nexus generated. > > Thanks!
[m2] How to use maven-plugin-tools-ant with a custom directory for scripts?
Hello all I'm trying to create a Maven 2 plugin using Ant as described there: http://maven.apache.org/guides/plugin/guide-ant-plugin-development.html But I wish to use a different directory layout than the Maven standard one. More specifically, my .build.xml and .mojos.xml files live elswhere than src/main/scripts. The above-cited page said that we can, but I didn't found any indication about how to do that. I searched in the mail archive and in the source code (http://svn.apache.org/viewcvs.cgi/maven/components/trunk/maven-plugin-tools/maven-plugin-tools-ant/src/main/java/org/apache/maven/tools/plugin/extractor/ant/AntMojoDescriptorExtractor.java?rev=326372&view=markup) with no luck. Does anyone know how to specify the source directory for the xml script files? Thanks, Martin. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [M2] How to add custom entries in a MANIFEST.MF file?
Jason van Zyl a écrit : I'm just working on the doco so maybe this will help: http://people.apache.org/~jvanzyl/maven2/guides/mini/guide-manifest.html Thanks a lot! It is exactly what I was looking for, and it worked perfectly well!!! Those kind of documentation save days :) As a side note (not a big issue), I noticed that the "Specification-Title:" entry in the manifest contains all the text specified in the project element. Is it the intended behavior? (I though that was for long description, something like a paragraph, and would be for shorter title applicable to "Specification-Title:" or "Implementation-Title:"). Martin. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[M2] How to add custom entries in a MANIFEST.MF file?
Hello all How to add custom entries in a JAR file (in addition to the current file created by Maven 2)? To be more specific, I would like to add a "RegistrationClassName: org.geotools.openoffice.Registration" entry for an OpenOffice.org add-in. I'm aware of the following email: http://mail-archives.apache.org/mod_mbox/maven-users/200509.mbox/[EMAIL PROTECTED] But the reply answer only about the specific case of "main-class" entry. I'm aware of the following patch: http://jira.codehaus.org/browse/MNG-742 But I fail to understand which class I should implement in order to invoke MavenArchiveConfiguration.addManifestEntry(key, value), and how to register my implementation to Maven 2. I'm aware of the and parameter in JAR plugin: http://maven.apache.org/maven2/plugins/maven-jar-plugin/jar-mojo.html But I do not understand how to use it in order to create a custom instance of MavenArchiveConfiguration (for example). I didn't found any example on internet. Any hint about how to add a "RegistrationClassName" entry in a JAR file would be greatly appreciated. Thanks, Martin. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
