Hm, I need to look more into MCOMPILER-496, as I found a m-compiler-p IT that does exactly what Thomas described: https://github.com/apache/maven-compiler-plugin/tree/master/src/it/MCOMPILER-203-processorpath
3 module multi module build: - an annotation processor - a verifier plugin - a "user" module that uses those above two within same build and it works as expected.... so more feedback or reproducer needed here as well. T On Fri, May 6, 2022 at 12:05 PM Tamás Cservenák <ta...@cservenak.net> wrote: > Howdy, > > a lot to unravel here... > > But let me start first with the obvious: I believe you are quite > experienced, and I'd bet you did not start with Maven3, but most probably > Maven2 or even Maven1 :) The reason why I mention this, is that you use > terms from the "Maven2 era", like "reactor". Maven2 had the well known > "maven-reactor-plugin", that ceased to exist in Maven3. IMHO, let's clear > up the terms as otherwise we will just accumulate more misunderstandings. > > For start, let's take Maven Extensions (core or build, does not matter). > > IMHO, having them both (the tool you build with, and the source you build) > in one multi module project is IMHO wrong. That would be like a carpenter > keeping his nails and other tools along with timber in the same > box/storage, which I believe is not true. There are two kinds of extensions > in Maven: "core extension" and "build extension". Yes, both of them need to > be loaded ahead of the build. So yes, in case you keep your nails and > timber in the same box, you need to "bootstrap" things somehow (like > installing extension), as otherwise Maven will be unable to load up your > build in the first place. > > But, let me ask this: how often your tooling changes compared to the > project? In other words, are you sure your extension you use to build with, > and the project itself, are changing at the same pace? Are you sure they > should share the same "release lifecycle"? > > My typical example here (not extension, but similar intent): if we examine > Maven build, look at maven-model module. With every (even dot) release of > Maven, we generate, compile, package etc the maven-model module, while the > Maven model did not change since Maven 3.0.0 (or so, did not fact check, > but the point is there: very rarely changes). IMHO, the Maven project > should simply "move out" the maven-model from Maven build, as it does not > belong there: they do not share any release lifecycle at all. > > If we get back to extensions kept in the same build where modules are > built as well, I wonder, how often does your extension change? Does it > really change with every dot release of your project? If so, then something > else may be improved in tooling (like missing some abstraction and/or > separation). If not, it is simply in the "wrong place": why do you > build/package/deploy it every time your project changes? > > But the same thing can be applied to examples like in this thread, for > example checkstyle configuration: is this the ONLY project in your company? > If not, is this the ONLY project having this checkstyle applied (and each > project has a different one?). Maven is all about "sharing" :) > > Your next example of an annotation processor falls under the same scrutiny > as above: does it really change with every release? But that said, and > reading your mail and in parallel reading m-compiler-p sources, I already > see a bug in m-compiler-p and how it resolves annotation processors: if you > look, it ends up in maven-compat (Maven2 compatibility layer) and does the > work in "Maven2 way", while you are right, this should work just fine out > of the box. Filed issue for this bug: > > https://issues.apache.org/jira/browse/MCOMPILER-496 > > So, your 2nd example (and 2nd paragraph) about the annotation processor is > IMHO a m-compiler-p bug, and shall be fixed. > > And your 3rd paragraph is a bit too vague for me. Could you, like Florent > did, create an example project that shows clearly what you want, and shows > what Maven does not as expected? > > > HTH > Tamas > > > > > > [1] http://takari.io/book/91-maven-classloading.html > > On Fri, May 6, 2022 at 10:40 AM Thomas Broyer <t.bro...@gmail.com> wrote: > >> >> Well, one limitation is with extensions: to use a project providing an >> extension (custom lifecycle, whatever), you have to install it first. You >> could argue that the project would otherwise be broken topologically (and >> I >> agree!) but there's really no way (or did I miss some new feature?) to do >> such things without installing the extension first (with Gradle, in >> addition to the buildSrc, you can use a "composite build" where one build >> provides plugins for the other, including "settings plugins" –where the >> settings.gradle file is also where you declare the included build–; Pants >> allows you to have in-repo plugins, it's a no-brainer with Bazel too as >> it's just how things work, I can't really tell for SBT but you could at >> least put plugins in a project/ and possibly symlink to another project to >> mimic an "included build" like Gradle) >> >> Other setups seem to work "by accident"; e.g. a project with 3 submodules: >> annotations, processor, whatever; where 'whatever' has a dependency on >> 'annotations' and uses 'processor' in >> maven-compiler-plugin's <annotationProcessorPaths>, the order of the >> <module> in the aggregator POM is crucial to make it work ('processor' has >> to come before 'whatever') *and* you'll have to explicitly include >> 'processor' in your project list: 'mvn -pl whatever -am verify' won't >> build >> 'processor', you have to use 'mvn -pl processor,whatever -am verify'. I >> would infer Maven's behavior for reactor ordering as taking the <modules> >> declaration order by default, when there's no dependency between modules, >> and then the maven-compiler-plugin happens to be able to resolve >> 'processor' from the reactor build; change the order <module> are declared >> and *kaboom*; in the end it does work only "by accident" (or "by chance" >> if >> you prefer). >> I wouldn't say the maven-compiler-plugin is "broken", it's just another >> Maven limitation: it has no way of knowing that the >> <annotationProcessorPaths> actually lists dependencies that could come >> from >> the reactor build and that it would take them into account for determining >> which modules to build and in which order. >> >> I don't have an actual use case at hand, this is all theoretical based on >> my knowledge of how Maven works so correct me if I'm wrong, but using >> custom lifecycles, and due to how build lifecycles work in Maven, running >> a >> goal "in a project" (it's never actually "in a project", it's always run >> in >> all projects of the reactor build, and the plugin itself has to detect or >> be configured to know in which module to actually run or skip) might only >> partially "build" the depended-on modules in such a way that they won't >> work when the plugin actually need them. >> Let's take a very hypothetical example, a plugin goal that scaffolds >> things >> in a project, so must not trigger the project's lifecycle as that would >> possibly break and we don't care (so either the plugin has no @execute, or >> it executes an "early" phase such as 'initialize', or maybe >> 'process-classes' if it scaffolds tests and somehow needs the project's >> main code to do that), but will use some dependency to determine what/how >> to do its job (so has a plugin dependency, let's say on another module of >> the reactor build). If we do 'mvn -pl target-project -am >> the-plugin:scaffold -Dwhatever', then the 'scaffolding-config' submodule >> declared as a plugin dependency of 'the-plugin' in project >> 'target-project' >> won't be built besides what the plugin's goal @execute declares. So you'd >> actually have to first 'mvn install -pl scaffolding-config' and then 'mvn >> -pl target-project the-plugin:scaffold -Dwhatever' (note that I haven't >> included -am here). >> You could argue this is somehow "broken", I wouldn't disagree, but then >> don't argue that Maven itself is not somehow "broken by design" or at a >> minimum "limited" in its capabilities: there are cases where using "mvn >> install" would be *required* to make things work. Either that or not using >> Maven for those things, and rather rely on third-party tooling outside >> Maven. >> >> -- >> Thomas Broyer >> /tɔ.ma.bʁwa.je/ <http://xn--nna.ma.xn--bwa-xxb.je/> < >> http://xn--nna.ma.xn--bwa-xxb.je/> >> >
