*http://jira.codehaus.org/browse/MNG-5652 <http://jira.codehaus.org/browse/MNG-5652>*
On 21 June 2014 15:28, Jason van Zyl <ja...@takari.io> wrote: > Throw it into a JIRA and tag the component FDPFC. I can only find 8 issues > related to features requiring POM format changes. I seem to recall more. > > On Jun 20, 2014, at 11:55 AM, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > > > Well I've had this proposal in one form or another on and off for the > past > > 2-3 years. Jason recently stated that he only wants to look at concrete > > proposals as the first stab for what/how to extend the pom format... so > > here is a very concrete example proposal ;-) > > > > > > On 20 June 2014 16:50, Paul Benedict <pbened...@apache.org> wrote: > > > >> Ah, the woes of fragmented jars. That's the bane of EE dependency > >> management. Figuring out the superset/subset stuff is indeed terrible. > Good > >> job, Stephen, to propose a solution for it. > >> > >> > >> Cheers, > >> Paul > >> > >> > >> On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly < > >> stephen.alan.conno...@gmail.com> wrote: > >> > >>> Well this started out from a need for a more effective <excludes>. > >>> > >>> The issue with <excludes> is that you have to apply it to each and > every > >>> dependency... when what you really want to say is "see this dependency > >>> here? well that is the same as X, Y and Z so don't pull them in from > the > >>> transitive dependencies" > >>> > >>> If you like that is the second use case in my proposal. > >>> > >>> The first use case is less relevant from the "better excludes" point of > >>> view, but I believe it to be more useful. > >>> > >>> The third use case is for when you don't trust upstream to get things > >>> right. Keep in mind that if, as a general principle, you don't trust > >>> upstream to express things mostly correct then what the hell are you > >> doing > >>> trusting them to express their transitive dependencies correctly? > >>> > >>> Tooling would allow us to validate a pom's claims of equivalence. But > >>> ultimately I think we need this concept to deal with te fragmentation > of > >>> API jars and implementations of APIs that we see taking place in > central. > >>> > >>> > >>> On 20 June 2014 15:27, Paul Benedict <pbened...@apache.org> wrote: > >>> > >>>> I am having trouble understanding how grouping together artifacts that > >>> are > >>>> "equivalent" gain me anything in my project building. I am already > >> doing > >>>> this in my POM today by excluding what's equivalent. Is this proposal > >>>> simply about adding semantic information to the POM so it's more > >> apparent > >>>> what the intent is? > >>>> > >>>> Furthermore, whoever defines the equivalence needs to be extremely > >>> careful. > >>>> Not all "equivalent" jars are actually equivalent. Most people have > >>>> learned, for example, that the javaee-api:6.0 jar in Maven is all > stubs > >>> and > >>>> can't be used for unit testing; so people go to find EE jars from > JBoss > >>> or > >>>> GlassFish, to get the full functionality. So I am not sure I'd ever > >> want > >>>> someone telling me what's equivalent during consuming. > >>>> > >>>> > >>>> Cheers, > >>>> Paul > >>>> > >>>> > >>>> On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly < > >>>> stephen.alan.conno...@gmail.com> wrote: > >>>> > >>>>> "supplies" concept proposal > >>>>> =========================== > >>>>> > >>>>> Introduction > >>>>> ------------ > >>>>> > >>>>> The following is a proposal for Maven in a post-modelVersion-4.0.0 > >> era. > >>>> The > >>>>> aim of this proposal is to simplify the management of dependency > >> trees > >>> in > >>>>> the decentralised era of artifact production that we find ourselves > >> in. > >>>>> > >>>>> The core issue is that different organisations can produce artifacts > >>> that > >>>>> may overlap. The easiest example is the servlet-api. If we restrict > >>>>> ourselves to version 2.5 of the servlet specification there are > >> quite a > >>>> few > >>>>> artifacts that all deliver the exact same content: > >>>>> > >>>>> * `jetty:servlet-api:2.5-6.0.2` > >>>>> * `org.daisy.libs:servlet-api:2.5.0` > >>>>> * `org.mortbay.jetty:servlet-api-2.5:6.1.14` > >>>>> * > >> `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final` > >>>>> * etc > >>>>> > >>>>> **Note:** this is a generic problem that is not restricted to the > >>>>> servlet-api, the servlet-api just provides the example that will be > >>> most > >>>>> familiar to everyone. > >>>>> > >>>>> So where these multiple artifacts supplying the equivalent content > >>>> becomes > >>>>> a problem is when the dependency tree is being calculated. If you > >> have > >>>> two > >>>>> dependencies each declaring transitive dependencies on different > >>>> artifacts > >>>>> that supply equivalent content, then you end up with two copies of > >> the > >>>> same > >>>>> JAR file in your classpath. > >>>>> > >>>>> In the case of the servlet-api, the hack most people use is to > >> declare > >>>> the > >>>>> servlet-api with scope `provided` thus preventing it from being > >>>> transitive. > >>>>> This is, however, a hack. In a more ideal world it would be better to > >>> let > >>>>> the servlet-api be transitive and only when we get to the WAR module > >>>> would > >>>>> we declare that a specific servlet-api is to be provided in the > >>>> containers > >>>>> that the WAR is targets for deployment into. > >>>>> > >>>>> We can take a second example that does not have the luxury of a *de > >>>> facto* > >>>>> hack. > >>>>> > >>>>> * `javax.faces:jsf-api:2.1` > >>>>> * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final` > >>>>> * `org.apache.myfaces.core:myfaces-api:2.1.13` > >>>>> > >>>>> Now in the case of the JSF API, you are supposed to bundle the JSF > >> API > >>> in > >>>>> your WAR file. So if I use three JSF component libraries, I could > >> very > >>>> well > >>>>> end up with three different but equivalent JSF API jars in my WAR > >> file. > >>>>> > >>>>> Ideally what we want is some way of telling Maven that these > >> artifacts > >>>> are > >>>>> equivalent. > >>>>> > >>>>> Proposal > >>>>> -------- > >>>>> > >>>>> Introduce the concept of "supplies" to the project model. The concept > >>>> needs > >>>>> three changes to the project model: > >>>>> > >>>>> 1. An explicit top level construct for a project to explicitly > >> declare > >>>>> up-front artifacts that it knows - at the time the project is being > >>>>> authored - to contain equivalent content to at least a subset of the > >>>>> project's content. Declarations could include a claim from: `subset`, > >>>>> `superset`, `disjoint` and `equivalent` with the default being > >>>> `disjoint`. > >>>>> 2. An explicit sub-element of the `dependency` construct to allow > >>>> consumers > >>>>> to *post-facto* declare a specific dependency as supplying equivalent > >>>>> content for other dependencies > >>>>> 3. An extension to the `dependency/excludes/exclude` construct to > >> allow > >>>>> consumers to remove claims a dependency makes with respect to > >> supplying > >>>>> equivalent content > >>>>> > >>>>> By way of illustration, here are some examples of these constructs > >>> mapped > >>>>> to a Model Version 4.0.0 like XML schema. As the > >>> post-modelVersion-4.0.0 > >>>>> schema is not yet known, this represents the best way to illustrate > >> how > >>>> the > >>>>> concept will work, but note that this proposal does not suggest a > >>> schema > >>>>> for this concept. > >>>>> > >>>>> ### Example 1 > >>>>> > >>>>> This illustrates how we would want, say, the `myfaces-api` project > >>> model > >>>> to > >>>>> look. > >>>>> > >>>>> ``` > >>>>> <project> > >>>>> <groupId>org.apache.myfaces.core</groupId> > >>>>> <artifactId>myfaces-api</artifactId> > >>>>> <version>2.1.3</version> > >>>>> ... > >>>>> <supplyManagement> > >>>>> <supplies> > >>>>> <groupId>javax.faces</groupId> > >>>>> <artifactId>jsf-api</artifactId> > >>>>> <version>[2.1,2.2)</version> > >>>>> <claim>superset</claim> > >>>>> <supplies> > >>>>> <supplies> > >>>>> <groupId>org.jboss.spec.javax.faces</groupId> > >>>>> <artifactId>jboss-jsf-api_2.1_spec</artifactId> > >>>>> <claim>equivalent</claim> > >>>>> <supplies> > >>>>> </supplyManagement> > >>>>> ... > >>>>> </project> > >>>>> ``` > >>>>> > >>>>> This indicates that the `myfaces-api` artifact is intended to be > >>> useable > >>>> as > >>>>> a drop-in replacement for either the `javax.faces:jsf-api` artifact > >>>> within > >>>>> a bounded range or for any version of the > >>>>> `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you > >>> get > >>>> a > >>>>> supplier conflict in your classpath, then Maven should fail the > >> build. > >>>>> > >>>>> For example if somebody forked `myfaces-api` but did not list > >>>> `myfaces-api` > >>>>> in the fork's supplyManagement and you end up with both `myfaces-api` > >>> and > >>>>> `myfaces-fork-api` in your classpath. Maven can detect that there are > >>> two > >>>>> dependencies that both claim to supply `javax.faces:jsf-api` and fail > >>> the > >>>>> build, thereby letting the user add either exclusions or additional > >>>>> supplies information to one of the artifacts and preventing duplicate > >>>>> artifacts on the classpath. The build need not be failed if the > >>> supplies > >>>>> claims provide a resolution. e.g. if the claim is `equivalent` then > >>> that > >>>>> implies that there is a 1:1 mapping and hence the artifacts are > >> drop-in > >>>>> replacements. However where the claim is `superset` we cannot know > >> that > >>>> the > >>>>> extra content in our artifact is the same as the extra content in > >>> another > >>>>> artifact which has a superset of `javax.faces:jsf-api`. > >>>>> > >>>>> ### Example 2 > >>>>> > >>>>> This illustrates a JSF component library that is working with the > >>>> existing > >>>>> JSF APIs > >>>>> > >>>>> ``` > >>>>> <project> > >>>>> ... > >>>>> <dependencies> > >>>>> <dependency> > >>>>> <groupId>javax.faces</groupId> > >>>>> <artifactId>jsf-api</artifactId> > >>>>> <version>2.1</version> > >>>>> <supplyManagement> > >>>>> <supplies> > >>>>> <groupId>org.jboss.spec.javax.faces</groupId> > >>>>> <artifactId>jboss-jsf-api_2.1_spec</artifactId> > >>>>> <claim>equivalent</claim> > >>>>> <supplies> > >>>>> <supplies> > >>>>> <groupId>org.apache.myfaces.core</groupId> > >>>>> <artifactId>myfaces-api</artifactId> > >>>>> <version>[2.1,2.2)</version> > >>>>> <claim>equivalent</claim> > >>>>> </supplies> > >>>>> </supplyManagement> > >>>>> <dependency> > >>>>> </dependencies> > >>>>> ... > >>>>> </project> > >>>>> ``` > >>>>> > >>>>> In this case we are publishing a transitive dependency with > >> additional > >>>>> supplyManagement injected. Consumers of this project would thus gain > >>> the > >>>>> benefit of collapsing their transitive dependencies for any of these > >>>> three > >>>>> artifacts. As all artifacts are declared with `equivalent` claim, > >> thus > >>>> the > >>>>> nearest of those three artifacts to the project will win as per the > >>>>> standard dependency resolution rules when dealing with conflicting > >>>> version > >>>>> requirements in the transitive dependency tree. > >>>>> > >>>>> ### Example 3 > >>>>> > >>>>> Finally, there is the case where you need to correct an incorrect > >> claim > >>>> of > >>>>> supply > >>>>> > >>>>> > >>>>> ``` > >>>>> <project> > >>>>> ... > >>>>> <dependencies> > >>>>> <dependency> > >>>>> <groupId>javax.faces</groupId> > >>>>> <artifactId>jsf-api</artifactId> > >>>>> <version>2.1</version> > >>>>> <exclusions> > >>>>> <exclusion> > >>>>> <groupId>org.jboss.spec.javax.faces</groupId> > >>>>> <artifactId>jboss-jsf-api_2.2_spec</artifactId> > >>>>> <scope>supplies</scope> > >>>>> <exclusion> > >>>>> </exclusions> > >>>>> <dependency> > >>>>> </dependencies> > >>>>> ... > >>>>> </project> > >>>>> ``` > >>>>> > >>>>> This would typically be coupled with adding back in a correct > >> supplies > >>>>> definition, but we need to allow for people to correct the graph > >> after > >>>> the > >>>>> fact of their dependencies being deployed to the remote repository. > >>>>> > >>>>> ### Claim conflict resolution > >>>>> > >>>>> The four classes of claim can be resolved using the following matrix > >>>>> > >>>>> ``` > >>>>> > >> +---------------------------------------------------+ > >>>>> | A > >> | > >>>>> > >> +------------+------------+------------+------------+ > >>>>> | subset | equivalent | superset | disjoint > >> | > >>>>> > >> +---+------------+------------+------------+------------+------------+ > >>>>> | | subset | conflict | A wins | A wins | conflict > >> | > >>>>> | > >> +------------+------------+------------+------------+------------+ > >>>>> | | equivalent | B wins | A or B | A wins | conflict > >> | > >>>>> | B > >> +------------+------------+------------+------------+------------+ > >>>>> | | superset | B wins | B wins | conflict | conflict > >> | > >>>>> | > >> +------------+------------+------------+------------+------------+ > >>>>> | | disjoint | conflict | conflict | conflict | conflict > >> | > >>>>> > >> +---+------------+------------+------------+------------+------------+ > >>>>> ``` > >>>>> > >>>>> The default unspecified claim is `disjoint` which indicates that some > >>> of > >>>>> the content is reproduced, but not all and there is additional > >> content > >>>>> added. With such a claim there will always be conflict and the build > >>>> should > >>>>> fail until the Project Model is updated to either remove some of the > >>>> claims > >>>>> or resolve the dependency clash. > >>>>> > >>>>> The ideal claim is `equivalent` which indicates that the two > >> artifacts > >>>> are > >>>>> bi-directionally substitutable. This does not mean that the contents > >>> are > >>>>> identical. It does mean that they both deliver on the same contract > >> in > >>> an > >>>>> equivalent fashion. > >>>>> > >>>>> The `subset` and `superset` claims are for aggregation APIs. So for > >>>> example > >>>>> the Java EE Web Profile API is a superset of the various spec APIs > >> that > >>>>> make up the Java EE Web Profile and a subset of the full Java EE > >>>>> specification. The use of the `subset` claim should be reserved to > >>> those > >>>>> cases that are strict subsets. If anything is added that is not in > >> the > >>>>> supplied artifact then the correct claim is `disjoint`. > >>>>> > >>>>> ### Validation of supplies claims > >>>>> > >>>>> We do not want to introduce Java bias with this feature. As a result > >>> the > >>>>> validation of claims and supplies directives will be left to plugins. > >>> For > >>>>> the Java case we should probably provide either/both an enforcer rule > >>> or > >>>> a > >>>>> maven hosted plugin to assist in checking JAR projects against the > >>>> declared > >>>>> supplies declarations, but Maven core should not be concerned with > >>>> solving > >>>>> the validation problem. > >>>>> > >>>>> Similarly, while there may be advantages in a more fine grained API > >>>>> contract negotiation between dependencies, to bind such a concept > >> into > >>>> the > >>>>> project model would significantly taint the Maven project model with > >>> more > >>>>> Java-centric concepts. Given that current software development > >>> typically > >>>>> uses at least two core languages: Java and JavaScript, we should be > >>>> aiming > >>>>> to reduce Java-centric constructs from Maven rather than increase > >> them. > >>>>> > >>>>> ### Backporting > >>>>> > >>>>> It will not be possible to fully express this concept in a > >> modelVersion > >>>>> 4.0.0 project model. Thus if generating 4.0.0 compatible project > >>> models, > >>>>> the aim should be to fully resolve the dependencies of the project > >>> using > >>>>> all available information and express that as the transitive > >>>> dependencies. > >>>>> > >>>>> Thus we will not expose the "supplies" information to modelVersion > >>> 4.0.0 > >>>>> parsers but we will expose the end results of that and present the > >>> final > >>>>> effective flattened dependency tree. > >>>>> > >>>>> modelVersion 4.0.0 consumers will thus be no worse off than they > >>> already > >>>>> are and those consumers understanding newer modelVersions can get the > >>>>> enhanced tree resolution that they would not get otherwise. > >>>>> > >>>> > >>> > >> > > Thanks, > > Jason > > ---------------------------------------------------------- > Jason van Zyl > Founder, Apache Maven > http://twitter.com/jvanzyl > http://twitter.com/takari_io > --------------------------------------------------------- > > We all have problems. How we deal with them is a measure of our worth. > > -- Unknown > > > > > > > > > >
