TL;DR your test project exposed two existing bugs, one change in behaviour and one quirk I can't explain
* Build `<extensions>` are loaded by two classloaders, which is a bug in DefaultProjectBuildingHelper#createProjectRealm and explains why you see extjar1/extjar2 in the output * ClassRealm does not allow same foreign-import from multiple classloaders, which is a bug and explains why it is not possible to load same resource from multiple plugins/extensions * TCCL does not have access to private (i.e. not exported) resources of this extensions plugin, which is a change of behaviour introduced by mng-6209 fix * Also, component injection order appears to be backwards, but maybe Stuart can explain why. Below is more detailed explanation of expected and observed behaviour ## Component injection depends on the currently running plugin and the injection site Currently running plugins have access to the following component implementations: * Regular plugin has access to components implemented by the plugin, project build extensions, if any (via project class realm foreign import) and Maven Core. * Extension plugin has access to components implemented by the project build extensions and Maven Core. * Without a running plugin (e.g., during project dependency resolution), components implemented by the project build extensions and Maven Core are accessible. Different injection sites have access to the following component interfaces: * Maven Core has access to component interfaces defined by the core itself (obviously) * Project build extensions have access to **public** component interfaces defined by Maven Core and component interfaces defined by the build extension itself (there is no way to access component interfaces defined in other extensions) * Regular plugins have access to **public** component interfaces defined by Maven Core, component interfaces **exported** by build extensions and component interfaces defined in the plugin itself For injection to work, injection site has to have access to the component interface and the component implementation must be accessible through the current context. >From what I can tell, in your example all plugins have access to the right components when using current 3.5.2-SNAPSHOT. The injection order does appear to be backwards from what I expected, however. ## Resources lookup fully depends on classpath visibility, specifically * Regular plugin class realm has access to resources from the plugin itself, from **exported** packages of the project build extensions and **public** Maven Core packages * Extensions plugin class realm has access to the resources from the extensions plugin itself and from **public** Maven Core packages * Project class realm has access to classes and resources **exported** by project build extensions and **public** Maven Core packages I see three problems here * Maven adds build single-jar `<extensions>` elements directly to project class realm **and** creates separate extensions class realms for them. Which results in duplicate classes/resources loaded by two classloaders and explains why you see extjar1/extjar2 output (which you shouldn't according to the explanation above) * ClassRealm does not allow foreign-import of the same package from multiple classloaders. This makes it impossible to load the same resource from multiple plugins/extensions. * Extensions plugins cannot access their own private (i.e. not exported) resources via TCCL, this is change in behaviour introduced by mng-6209 fix Hope this helps -- Regards, Igor On Mon, Sep 18, 2017, at 11:46 AM, Stephen Connolly wrote: > Oh if only... there is some subtleties going on here. > > Classes are managed by the "plexus" / "classworlds" stuff, so you cannot > override core classes etc. > > The problem is what extensions are visible and from which classloader > > On 18 September 2017 at 08:42, Charles Honton <c...@honton.org> wrote: > > > From a security perspective, I would expect that core classes can not be > > overridden by extensions or plugins. Likewise, extension classes can not > > be overridden by plugins. > > > > Given the use case of defaulting resources, I would expect that the plugin > > resources are first, followed by plugin specific extensions, followed by > > global extensions, finally core maven. (This allows resources to be > > specialized.) > > > > regards, > > chas > > > > > On Sep 18, 2017, at 3:20 AM, Stephen Connolly < > > stephen.alan.conno...@gmail.com> wrote: > > > > > > Hmmm, so I did some experiments: > > > > > > If you want to ride along, the experiments are at: > > > > > > https://github.com/stephenc/mng-6209 > > > > > > So basically I have a plugin that does three different tests: > > > > > > getLog().info("Injected by @Component:"); > > > for (Lifecycle l : lifecycles) { > > > if (l.getId().startsWith("mng-6209-")) { > > > getLog().info(" " + l.getId().substring(9)); > > > } > > > } > > > getLog().info(""); > > > getLog().info("On Plugin Class Loader:"); > > > try { > > > ClassLoader tccl = ListMojo.class.getClassLoader(); > > > for (URL url : > > > Collections.list(tccl.getResources("META-INF/probe.txt"))) { > > > InputStream is = url.openStream(); > > > try { > > > getLog().info(" " + IOUtil.toString(is).trim()); > > > } finally { > > > is.close(); > > > } > > > } > > > } catch (IOException e) { > > > throw new MojoExecutionException(e.getMessage(), e); > > > } > > > getLog().info(""); > > > getLog().info("On Thread Context Class Loader:"); > > > try { > > > ClassLoader tccl = > > > Thread.currentThread().getContextClassLoader(); > > > for (URL url : > > > Collections.list(tccl.getResources("META-INF/probe.txt"))) { > > > InputStream is = url.openStream(); > > > try { > > > getLog().info(" " + IOUtil.toString(is).trim()); > > > } finally { > > > is.close(); > > > } > > > } > > > } catch (IOException e) { > > > throw new MojoExecutionException(e.getMessage(), e); > > > } > > > > > > > > > First off, I hijack the @Component injection with some fake "lifecycles" > > to > > > see what "plexus" exposes to the plugins. > > > > > > Second, I look at the resources visible from the plugin's classloader. > > > > > > Finally, I look at the resources visible from the TCCL. > > > > > > Here's what 3.5.0 outputs: > > > > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Both extensions. Order: plugin1, plugin2 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe1 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe1 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Only plugin1 extensions. Order: plugin1, plugin2 > > > 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe2 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe2 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin2 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Both extensions. Order: plugin2, plugin1 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe3 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe3 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Both extensions. Order: plugin1, plugin2. Extra > > dependency > > > in plugin1 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe4 --- > > > [INFO] Injected by @Component: > > > [INFO] extjar2 > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe4 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > ------------------------------------------------------------------------ > > > > > > Now if we run with 3.5.1 (which contains the fix for MNG-6209) > > > > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Both extensions. Order: plugin1, plugin2 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe1 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > I would have expected the sequence to be: /build/extensions in pom order > > > followed by /build/plugins/plugin[extensions=true] in pom order. This > > seems > > > to be the reverse order. Is this a bug? > > > > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > > > > We haven't changed how the plugin classloader gets instantiated in > > > MNG-6209. It seems strange to me that this excludes the > > > /build/extensions... on the other hand this could be a side-effect of how > > > the classloader gets instantiated (which would mean using the plugin's > > > classloader is probably a bad idea, perhaps we need to provide the > > ability > > > to inject the classloader as a @Component or something) > > > > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar1 > > > [INFO] extjar2 > > > > > > OK, we see the change vs 3.5.0 as this is now the project realm... > > though I > > > would have expected the project realm to also include the plugins that > > were > > > marked as extensions... > > > > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe1 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar1 > > > [INFO] extjar2 > > > [INFO] > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Only plugin1 extensions. Order: plugin1, plugin2 > > > 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe2 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > As expected, given that plugin2 is not an extension here (apart from the > > > order being reverse of what I expect) > > > > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe2 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > As expected (modulo order) because a plugin should see any self-defined > > > extensions. I would expect the order to be plugin2, extjar1, extjar2, > > > plugin1 (because the plugin is not an extension, it should have its > > > extensions as priority over the project realms's) > > > > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > WAT! ok, this I do not understand. Something is wrong somewhere, either > > > plugin1's classloader should also include extjar2 and extjar1 or this one > > > shouldn't. And since we have extjar1 and extjar2 where is plugin1? (if > > the > > > inclusion is correct here I expect plugin2, extjar1, extjar2, plugin1 as > > a > > > plugin that is not an extension plugin should have its own implementation > > > first followed then by /build/extensions in pom order and then > > > /build/plugins/plugin[extension==true] > > > > > > This is making no sense at all! > > > > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] plugin2 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > So TCCL is the plugin classloader here as expected > > > > > > [INFO] > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Both extensions. Order: plugin2, plugin1 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe3 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > I note that the change in pom order has had no effect on the component > > > sequence. This leads me to suspect that plexus has some rule that is > > > defining the sequencing... it would be good if we could document that > > > somewhere... > > > > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe3 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin2 > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] Building Both extensions. Order: plugin1, plugin2. Extra > > dependency > > > in plugin1 1.0-SNAPSHOT > > > [INFO] > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- plugin1:1.0-SNAPSHOT:list (default) @ probe4 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] plugin2 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > > > > WAT! seems that @Component is somewhat in random order, in this case we > > > have modified the dependencies if plugin1 to also include extjar2, the > > > order I would expect is: > > > > > > extjar1, extjar2, plugin1, extjar2, plugin2 > > > > > > Given the previous executions and the fact that changing the sequence of > > > plugin1 and plugin2 in the pom did not affect the previous executions, if > > > plexus had a deterministic ordering then I would have been somewhat OK > > with: > > > > > > plugin2, plugin1, extjar2, extjar2, extjar1 > > > > > > But this seems to suggest that we have a completely non-deterministic > > > ordering... that is not good for reproducible builds... > > > > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin1 > > > [INFO] extjar2 > > > > > > Now we see extjar2 but only because it has been explicitly added to the > > > plugins dependencies, IMHO it should be here twice, e.g. > > > > > > extjar1, extjar2, plugin1, extjar2, plugin2 > > > > > > But it seems acceptable if this is instead plugin1, extjar2 (though it > > > makes the plugin classloader useless for discovery) > > > > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar1 > > > [INFO] extjar2 > > > [INFO] > > > [INFO] --- plugin2:1.0-SNAPSHOT:list (default) @ probe4 --- > > > [INFO] Injected by @Component: > > > [INFO] plugin1 > > > [INFO] extjar2 > > > [INFO] plugin2 > > > [INFO] extjar2 > > > [INFO] extjar1 > > > [INFO] > > > [INFO] On Plugin Class Loader: > > > [INFO] plugin2 > > > [INFO] > > > [INFO] On Thread Context Class Loader: > > > [INFO] extjar1 > > > [INFO] extjar2 > > > [INFO] > > > ------------------------------------------------------------------------ > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > > For additional commands, e-mail: dev-h...@maven.apache.org > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
