Well I've had this proposal in one form or another on and off for the past 2-3 years. Jason recently stated that he only wants to look at concrete proposals as the first stab for what/how to extend the pom format... so here is a very concrete example proposal ;-)
On 20 June 2014 16:50, Paul Benedict <[email protected]> wrote: > Ah, the woes of fragmented jars. That's the bane of EE dependency > management. Figuring out the superset/subset stuff is indeed terrible. Good > job, Stephen, to propose a solution for it. > > > Cheers, > Paul > > > On Fri, Jun 20, 2014 at 10:03 AM, Stephen Connolly < > [email protected]> wrote: > > > Well this started out from a need for a more effective <excludes>. > > > > The issue with <excludes> is that you have to apply it to each and every > > dependency... when what you really want to say is "see this dependency > > here? well that is the same as X, Y and Z so don't pull them in from the > > transitive dependencies" > > > > If you like that is the second use case in my proposal. > > > > The first use case is less relevant from the "better excludes" point of > > view, but I believe it to be more useful. > > > > The third use case is for when you don't trust upstream to get things > > right. Keep in mind that if, as a general principle, you don't trust > > upstream to express things mostly correct then what the hell are you > doing > > trusting them to express their transitive dependencies correctly? > > > > Tooling would allow us to validate a pom's claims of equivalence. But > > ultimately I think we need this concept to deal with te fragmentation of > > API jars and implementations of APIs that we see taking place in central. > > > > > > On 20 June 2014 15:27, Paul Benedict <[email protected]> wrote: > > > > > I am having trouble understanding how grouping together artifacts that > > are > > > "equivalent" gain me anything in my project building. I am already > doing > > > this in my POM today by excluding what's equivalent. Is this proposal > > > simply about adding semantic information to the POM so it's more > apparent > > > what the intent is? > > > > > > Furthermore, whoever defines the equivalence needs to be extremely > > careful. > > > Not all "equivalent" jars are actually equivalent. Most people have > > > learned, for example, that the javaee-api:6.0 jar in Maven is all stubs > > and > > > can't be used for unit testing; so people go to find EE jars from JBoss > > or > > > GlassFish, to get the full functionality. So I am not sure I'd ever > want > > > someone telling me what's equivalent during consuming. > > > > > > > > > Cheers, > > > Paul > > > > > > > > > On Fri, Jun 20, 2014 at 7:51 AM, Stephen Connolly < > > > [email protected]> wrote: > > > > > > > "supplies" concept proposal > > > > =========================== > > > > > > > > Introduction > > > > ------------ > > > > > > > > The following is a proposal for Maven in a post-modelVersion-4.0.0 > era. > > > The > > > > aim of this proposal is to simplify the management of dependency > trees > > in > > > > the decentralised era of artifact production that we find ourselves > in. > > > > > > > > The core issue is that different organisations can produce artifacts > > that > > > > may overlap. The easiest example is the servlet-api. If we restrict > > > > ourselves to version 2.5 of the servlet specification there are > quite a > > > few > > > > artifacts that all deliver the exact same content: > > > > > > > > * `jetty:servlet-api:2.5-6.0.2` > > > > * `org.daisy.libs:servlet-api:2.5.0` > > > > * `org.mortbay.jetty:servlet-api-2.5:6.1.14` > > > > * > `org.jboss.spec.javax.servlet:jboss-servlet-api_2.5_spec:1.0.1.Final` > > > > * etc > > > > > > > > **Note:** this is a generic problem that is not restricted to the > > > > servlet-api, the servlet-api just provides the example that will be > > most > > > > familiar to everyone. > > > > > > > > So where these multiple artifacts supplying the equivalent content > > > becomes > > > > a problem is when the dependency tree is being calculated. If you > have > > > two > > > > dependencies each declaring transitive dependencies on different > > > artifacts > > > > that supply equivalent content, then you end up with two copies of > the > > > same > > > > JAR file in your classpath. > > > > > > > > In the case of the servlet-api, the hack most people use is to > declare > > > the > > > > servlet-api with scope `provided` thus preventing it from being > > > transitive. > > > > This is, however, a hack. In a more ideal world it would be better to > > let > > > > the servlet-api be transitive and only when we get to the WAR module > > > would > > > > we declare that a specific servlet-api is to be provided in the > > > containers > > > > that the WAR is targets for deployment into. > > > > > > > > We can take a second example that does not have the luxury of a *de > > > facto* > > > > hack. > > > > > > > > * `javax.faces:jsf-api:2.1` > > > > * `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec:2.1.28.Final` > > > > * `org.apache.myfaces.core:myfaces-api:2.1.13` > > > > > > > > Now in the case of the JSF API, you are supposed to bundle the JSF > API > > in > > > > your WAR file. So if I use three JSF component libraries, I could > very > > > well > > > > end up with three different but equivalent JSF API jars in my WAR > file. > > > > > > > > Ideally what we want is some way of telling Maven that these > artifacts > > > are > > > > equivalent. > > > > > > > > Proposal > > > > -------- > > > > > > > > Introduce the concept of "supplies" to the project model. The concept > > > needs > > > > three changes to the project model: > > > > > > > > 1. An explicit top level construct for a project to explicitly > declare > > > > up-front artifacts that it knows - at the time the project is being > > > > authored - to contain equivalent content to at least a subset of the > > > > project's content. Declarations could include a claim from: `subset`, > > > > `superset`, `disjoint` and `equivalent` with the default being > > > `disjoint`. > > > > 2. An explicit sub-element of the `dependency` construct to allow > > > consumers > > > > to *post-facto* declare a specific dependency as supplying equivalent > > > > content for other dependencies > > > > 3. An extension to the `dependency/excludes/exclude` construct to > allow > > > > consumers to remove claims a dependency makes with respect to > supplying > > > > equivalent content > > > > > > > > By way of illustration, here are some examples of these constructs > > mapped > > > > to a Model Version 4.0.0 like XML schema. As the > > post-modelVersion-4.0.0 > > > > schema is not yet known, this represents the best way to illustrate > how > > > the > > > > concept will work, but note that this proposal does not suggest a > > schema > > > > for this concept. > > > > > > > > ### Example 1 > > > > > > > > This illustrates how we would want, say, the `myfaces-api` project > > model > > > to > > > > look. > > > > > > > > ``` > > > > <project> > > > > <groupId>org.apache.myfaces.core</groupId> > > > > <artifactId>myfaces-api</artifactId> > > > > <version>2.1.3</version> > > > > ... > > > > <supplyManagement> > > > > <supplies> > > > > <groupId>javax.faces</groupId> > > > > <artifactId>jsf-api</artifactId> > > > > <version>[2.1,2.2)</version> > > > > <claim>superset</claim> > > > > <supplies> > > > > <supplies> > > > > <groupId>org.jboss.spec.javax.faces</groupId> > > > > <artifactId>jboss-jsf-api_2.1_spec</artifactId> > > > > <claim>equivalent</claim> > > > > <supplies> > > > > </supplyManagement> > > > > ... > > > > </project> > > > > ``` > > > > > > > > This indicates that the `myfaces-api` artifact is intended to be > > useable > > > as > > > > a drop-in replacement for either the `javax.faces:jsf-api` artifact > > > within > > > > a bounded range or for any version of the > > > > `org.jboss.spec.javax.faces:jboss-jsf-api_2.1_spec` artifact. If you > > get > > > a > > > > supplier conflict in your classpath, then Maven should fail the > build. > > > > > > > > For example if somebody forked `myfaces-api` but did not list > > > `myfaces-api` > > > > in the fork's supplyManagement and you end up with both `myfaces-api` > > and > > > > `myfaces-fork-api` in your classpath. Maven can detect that there are > > two > > > > dependencies that both claim to supply `javax.faces:jsf-api` and fail > > the > > > > build, thereby letting the user add either exclusions or additional > > > > supplies information to one of the artifacts and preventing duplicate > > > > artifacts on the classpath. The build need not be failed if the > > supplies > > > > claims provide a resolution. e.g. if the claim is `equivalent` then > > that > > > > implies that there is a 1:1 mapping and hence the artifacts are > drop-in > > > > replacements. However where the claim is `superset` we cannot know > that > > > the > > > > extra content in our artifact is the same as the extra content in > > another > > > > artifact which has a superset of `javax.faces:jsf-api`. > > > > > > > > ### Example 2 > > > > > > > > This illustrates a JSF component library that is working with the > > > existing > > > > JSF APIs > > > > > > > > ``` > > > > <project> > > > > ... > > > > <dependencies> > > > > <dependency> > > > > <groupId>javax.faces</groupId> > > > > <artifactId>jsf-api</artifactId> > > > > <version>2.1</version> > > > > <supplyManagement> > > > > <supplies> > > > > <groupId>org.jboss.spec.javax.faces</groupId> > > > > <artifactId>jboss-jsf-api_2.1_spec</artifactId> > > > > <claim>equivalent</claim> > > > > <supplies> > > > > <supplies> > > > > <groupId>org.apache.myfaces.core</groupId> > > > > <artifactId>myfaces-api</artifactId> > > > > <version>[2.1,2.2)</version> > > > > <claim>equivalent</claim> > > > > </supplies> > > > > </supplyManagement> > > > > <dependency> > > > > </dependencies> > > > > ... > > > > </project> > > > > ``` > > > > > > > > In this case we are publishing a transitive dependency with > additional > > > > supplyManagement injected. Consumers of this project would thus gain > > the > > > > benefit of collapsing their transitive dependencies for any of these > > > three > > > > artifacts. As all artifacts are declared with `equivalent` claim, > thus > > > the > > > > nearest of those three artifacts to the project will win as per the > > > > standard dependency resolution rules when dealing with conflicting > > > version > > > > requirements in the transitive dependency tree. > > > > > > > > ### Example 3 > > > > > > > > Finally, there is the case where you need to correct an incorrect > claim > > > of > > > > supply > > > > > > > > > > > > ``` > > > > <project> > > > > ... > > > > <dependencies> > > > > <dependency> > > > > <groupId>javax.faces</groupId> > > > > <artifactId>jsf-api</artifactId> > > > > <version>2.1</version> > > > > <exclusions> > > > > <exclusion> > > > > <groupId>org.jboss.spec.javax.faces</groupId> > > > > <artifactId>jboss-jsf-api_2.2_spec</artifactId> > > > > <scope>supplies</scope> > > > > <exclusion> > > > > </exclusions> > > > > <dependency> > > > > </dependencies> > > > > ... > > > > </project> > > > > ``` > > > > > > > > This would typically be coupled with adding back in a correct > supplies > > > > definition, but we need to allow for people to correct the graph > after > > > the > > > > fact of their dependencies being deployed to the remote repository. > > > > > > > > ### Claim conflict resolution > > > > > > > > The four classes of claim can be resolved using the following matrix > > > > > > > > ``` > > > > > +---------------------------------------------------+ > > > > | A > | > > > > > +------------+------------+------------+------------+ > > > > | subset | equivalent | superset | disjoint > | > > > > > +---+------------+------------+------------+------------+------------+ > > > > | | subset | conflict | A wins | A wins | conflict > | > > > > | > +------------+------------+------------+------------+------------+ > > > > | | equivalent | B wins | A or B | A wins | conflict > | > > > > | B > +------------+------------+------------+------------+------------+ > > > > | | superset | B wins | B wins | conflict | conflict > | > > > > | > +------------+------------+------------+------------+------------+ > > > > | | disjoint | conflict | conflict | conflict | conflict > | > > > > > +---+------------+------------+------------+------------+------------+ > > > > ``` > > > > > > > > The default unspecified claim is `disjoint` which indicates that some > > of > > > > the content is reproduced, but not all and there is additional > content > > > > added. With such a claim there will always be conflict and the build > > > should > > > > fail until the Project Model is updated to either remove some of the > > > claims > > > > or resolve the dependency clash. > > > > > > > > The ideal claim is `equivalent` which indicates that the two > artifacts > > > are > > > > bi-directionally substitutable. This does not mean that the contents > > are > > > > identical. It does mean that they both deliver on the same contract > in > > an > > > > equivalent fashion. > > > > > > > > The `subset` and `superset` claims are for aggregation APIs. So for > > > example > > > > the Java EE Web Profile API is a superset of the various spec APIs > that > > > > make up the Java EE Web Profile and a subset of the full Java EE > > > > specification. The use of the `subset` claim should be reserved to > > those > > > > cases that are strict subsets. If anything is added that is not in > the > > > > supplied artifact then the correct claim is `disjoint`. > > > > > > > > ### Validation of supplies claims > > > > > > > > We do not want to introduce Java bias with this feature. As a result > > the > > > > validation of claims and supplies directives will be left to plugins. > > For > > > > the Java case we should probably provide either/both an enforcer rule > > or > > > a > > > > maven hosted plugin to assist in checking JAR projects against the > > > declared > > > > supplies declarations, but Maven core should not be concerned with > > > solving > > > > the validation problem. > > > > > > > > Similarly, while there may be advantages in a more fine grained API > > > > contract negotiation between dependencies, to bind such a concept > into > > > the > > > > project model would significantly taint the Maven project model with > > more > > > > Java-centric concepts. Given that current software development > > typically > > > > uses at least two core languages: Java and JavaScript, we should be > > > aiming > > > > to reduce Java-centric constructs from Maven rather than increase > them. > > > > > > > > ### Backporting > > > > > > > > It will not be possible to fully express this concept in a > modelVersion > > > > 4.0.0 project model. Thus if generating 4.0.0 compatible project > > models, > > > > the aim should be to fully resolve the dependencies of the project > > using > > > > all available information and express that as the transitive > > > dependencies. > > > > > > > > Thus we will not expose the "supplies" information to modelVersion > > 4.0.0 > > > > parsers but we will expose the end results of that and present the > > final > > > > effective flattened dependency tree. > > > > > > > > modelVersion 4.0.0 consumers will thus be no worse off than they > > already > > > > are and those consumers understanding newer modelVersions can get the > > > > enhanced tree resolution that they would not get otherwise. > > > > > > > > > >
