On Thursday 25 August 2016, Robert Scholte <rfscho...@apache.org> wrote:
> On Thu, 25 Aug 2016 01:10:36 +0200, Stephen Connolly < > stephen.alan.conno...@gmail.com> wrote: > > On 24 August 2016 at 04:50, Robert Scholte <rfscho...@apache.org> wrote: >> >> I realized last ApacheCon that I wasn't clear about my definiton of the >>> consumer-pom. >>> I don't think it should be a flattened pom with only the dependency >>> information. Instead it shouldn't be a pom (matching the pom.xsd) at >>> all. >>> >>> >> I am not in disagreement, but as a concept the name "consumer pom" has >> more >> traction. >> > > I'd prefer consumer pom too, but it has led to confusion due to the > assumption that consumer pom and flattened pom are the same thing. To me it > makes sense to consider a new fileformat for the consumer pom which matches > better the requirements. And yes, the "provides" would fit here. New format or new schema? If we accept this is machine generated then we can use a format that does not support comments (eg JSON) but it's hard to beat XSLT for a cross platform schema migration tool... So XSLT would mandate XML (with all the irrational hate people have for that - I say irrational as the people laying the hate usually love writing HTML ;-) ) > > >> I see the builder pom as probably ending up not even being XML at all. >> Rather IMHO it will end up being a DSL that is easy to author and not >> verbose... and certainly not XML >> >> So the consumer pom should be stripped back to include two sets of >> information: >> >> * dependencies - we are mostly familiar with this... though we may not >> expose all the scopes... depends on how we think about things and how we >> view scopes moving forward >> >> * provides - this is vitally important IMHO and not handled currently. >> >> To understand provides we have to look at things like JavaEE (but the >> concept has general utility... though I suspect only for about 10-25% of >> projects) >> >> If I have a project that says: provides javax.servlet:servlet-api:3.1 then >> that is saying that my project is providing the equivalent content as in >> javax.servlet:servlet-api:3.1 so for example org.jboss.spec.javax.servlet: >> jboss-servlet-api_3.1_spec:1.0.0.Final would say >> javax.servlet:servlet-api:3.1 >> >> When resolving the dependency tree, maven then knows that any transitive >> requirement for javax.servlet:servlet-api:3.1 has already been met by my >> direct dependency on org.jboss.spec.javax.servlet:j >> boss-servlet-api_3.1_spec >> :1.0.0.Final if we have a direct dependency on >> org.apache.tomcat:tomcat-servlet-api:8.5.4 (which would also say >> javax.servlet:servlet-api:3.1) then Maven can report an error and fail the >> build due to dependency conflict. >> >> There are lots of other improvements we can add, but to start we need to >> have a way to declare when a project includes duplicate content of another >> artifact. Convention will be required to make this work correctly... >> perhaps we can even introduce a new project type that provides needs to >> point at so that a provides has to point at an "empty" specification >> project... >> >> Finally, for the consumer pom refactoring I believe we need to address >> architecture specific artifacts as these are a sort of implicit provides. >> >> >> Maybe it should be called something like consumer-dom (dependency object >>> model, though dom is confusing...). >>> It should be the most fast and efficient way to resolve the dependency >>> tree. That means it should do less roundtrips like Maven must do do right >>> now: for every dependency download the pom, for all transitive >>> dependencies >>> download all poms, etc. >>> Why can't it be a pom? I'd like to add the (file)extension too. Otherwise >>> Maven needs to initialize the matching ArtifactHandler and translate the >>> type to the proper extension. >>> >> >> >> I think the consumer pom needs to embed some of the artifact handler >> information for it self as otherwise non-maven tooling cannot be expected >> to understand those details... also we should be making the consumer pom >> both Maven and Java agnostic... but this is a grand problem alright! >> >> >> The pom doesn't have room for this. >>> >>> >> I saw the whole flattened pom experiment as mostly a waste of time for the >> consumer pom effort as I envision the consumer pom not looking anything >> like the current pom at all... but it is a project model... >> >> Oh and the consumer pom should include information for the side-artifacts >> (which would help with reusing tests as we could attach the test >> dependencies to the test artifact details in the consumer pom) >> >> Consumer-dom is an extraction and enriched version of the pom and will be >> a >> >>> separate upload to the repository. Build tools who can understand this >>> file >>> can use it or fall back by downloading all poms. >>> >>> thanks, >>> Robert >>> >>> >>> On Wed, 24 Aug 2016 09:22:03 +0200, Stephen Connolly < >>> stephen.alan.conno...@gmail.com> wrote: >>> >>> The consumer Pom is for machine to machine, it should be human readable >>> >>>> because that is nice, but intent is never human generated. >>>> >>>> Switching to this separation allows us to be more radical in the changes >>>> to >>>> the build Pom. >>>> >>>> The only reason we deploy packaging Pom's build Pom is for inheritance. >>>> If >>>> we didn't have to deal with that we wouldn't need to deploy any build >>>> poms >>>> ever. >>>> >>>> For using a build Pom as a parent, you implicitly pick up the minimum >>>> version of maven that your parent requires, so we then are free to >>>> evolve >>>> the build Pom format without worrying about forward compatibility, only >>>> backwards compatibility on the *build* Pom. >>>> >>>> The consumer Pom needs partial *forward* compatibility, so that older >>>> clients are able to *attempt* to consume... >>>> >>>> In short there are totally different compatibility constraints on the >>>> two, >>>> so they should be separate. >>>> >>>> Mixins would probably also be deployed, once we figure out how they work >>>> with build poms. >>>> >>>> I think we probably need to rethink version ranges. What I'd like is to >>>> let >>>> the consumer Pom treat version ranges more as guidance rather than hard >>>> requirements. It's a pain if you depend transitiveky on Foo:[1.0] but >>>> need >>>> Foo:[1.0.1,1.1) for the critical security fix... Having to run around >>>> applying excludes is not a good plan... Yes the build should initially >>>> fail >>>> if I depend on [1.0] and [1.0.1,1.1) in my graph, but I should be able >>>> to >>>> resolve the conflict for all my consumers (unless they pull in the >>>> conflict >>>> again themselves) >>>> >>>> On Tuesday 23 August 2016, Fred Cooke <fred.co...@gmail.com> wrote: >>>> >>>> I still find it a bit off that the original real POM won't always be >>>> >>>>> directly available anymore. >>>>> >>>>> Other tools create fake POMs because they *have* to - there's no other >>>>> option. >>>>> >>>>> I feel like some fidelity would be lost. Diffability would be lost >>>>> (from >>>>> a >>>>> scenario where there's nothing to diff). >>>>> >>>>> Unrelated, really, but kind of related: top level deployable artefact >>>>> deployments, debs, wars, exes, etc. When ranges are in use it'd be nice >>>>> to >>>>> deploy a sort of strict effective pom with fully hard [] versions for >>>>> all >>>>> things. This can be achieved in other ways, though. >>>>> >>>>> I guess that is to say, don't forget about non-central deployments. I >>>>> suppose if there's a way to always deploy pom.xml.build through some >>>>> plugin >>>>> or configuration or some such, then that's fine with me. >>>>> >>>>> On Wed, Aug 24, 2016 at 6:49 PM, Hervé BOUTEMY <herve.bout...@free.fr >>>>> <javascript:;>> >>>>> >>>>> wrote: >>>>> >>>>> > Le mercredi 24 août 2016 18:41:59 Fred Cooke a écrit : >>>>> > > Fair call re embedded pom, however it's quite convenient to just >>>>> browse >>>>> > to >>>>> > > it and read. >>>>> > > >>>>> > > I've occasionally gone looking for details from poms of artefacts >>>>> and >>>>> > found >>>>> > > a mess and missing stuff, and been annoyed. It probably wasn't >>>>> gradle's >>>>> > > fault, though. Just a sloppy author. If I'm honest I wouldn't even >>>>> know >>>>> > if >>>>> > > I've ever consumed a non-maven artefact, certainly none of mine! >>>>> :-) >>>>> > > >>>>> > > No, I am sure, I have, at least one, and it's an ant one with a >>>>> hard >>>>> > coded >>>>> > > POM that doesn't always reflect the contents of the jar. Yuck. >>>>> Clearly >>>>> > not >>>>> > > an issue with automated stuff, though. >>>>> > > >>>>> > > My only argument now is ease of reading things like project >>>>> descriptions, >>>>> > > contributors, SCM details, etc rather than having to unpack the >>>>> jar. >>>>> And >>>>> > if >>>>> > > you do, and end up with two pom.xmls laying around, that's not >>>>> nice. >>>>> > Better >>>>> > > to just start always suffixing one with pom.xml.build or some >>>>> such? I >>>>> > think >>>>> > > so, but I haven't thought deeply about it aside from reading this >>>>> thread. >>>>> > our consumer pom will be generated from build pom: we can automate >>>>> copy >>>>> of >>>>> > any >>>>> > information we want, and for sure, I expect to put at least >>>>> description, >>>>> > scm >>>>> > details, issueManagement, license (of course). >>>>> > In your list, there is only constributors that I was not counting on >>>>> it: >>>>> > but >>>>> > it's a detailed decision we'll have to make >>>>> > >>>>> > for sure, Maven consumer poms, since generated from Maven build pom, >>>>> can >>>>> > have >>>>> > much more details (and be sure they are accurrate) than build tools >>>>> that >>>>> > don't >>>>> > generate it from data that exists in their original build format >>>>> > >>>>> > Regards, >>>>> > >>>>> > Hervé >>>>> > >>>>> > > >>>>> > > >>>>> > > >>>>> > > On Wed, Aug 24, 2016 at 6:32 PM, Hervé BOUTEMY < >>>>> herve.bout...@free.fr >>>>> <javascript:;>> >>>>> >>>>> > > >>>>> > > wrote: >>>>> > > > Le mercredi 24 août 2016 14:04:12 Fred Cooke a écrit : >>>>> > > > > That should have separation between builder Pom and consumer >>>>> Pom. >>>>> For >>>>> > > > > packaging=pom we deploy the builder Pom using classifier=build >>>>> > > > > *for all other packaging a we do not deploy the builder Pom*. >>>>> > > > > >>>>> > > > > I don't like the sound of this. The deployed artefacts should >>>>> include >>>>> > > > > the >>>>> > > > > exact POM in use to build the project, as a reference, even if >>>>> under >>>>> > a >>>>> > > > > different name. Yes, they should be in SCM, however down stream >>>>> it's >>>>> > > > >>>>> > > > useful >>>>> > > > >>>>> > > > > to see these even when the SCM is offline or gone or private or >>>>> > > > > whatever. >>>>> > > > > Or did I misunderstand? If so, please clarify? >>>>> > > > >>>>> > > > when you consume an artifact not build with Maven, do you get the >>>>> full >>>>> > > > build >>>>> > > > script? >>>>> > > > no >>>>> > > > I know that, as Maven users, we got used to have the build pom >>>>> > published >>>>> > > > in >>>>> > > > central: this is now an issue, but we like that >>>>> > > > >>>>> > > > notice: the build pom can be injected in the artifact, in >>>>> > META-INF/maven, >>>>> > > > like >>>>> > > > it is currently done >>>>> > > > but I don't see the point in requiring it to be pbulished >>>>> separately >>>>> in >>>>> > > > central: no other build tool does that, and they don't have any >>>>> issue >>>>> > with >>>>> > > > that (and eventually it's a feature: don't publish internal >>>>> details >>>>> you >>>>> > > > don't >>>>> > > > really want people to see) >>>>> > > > >>>>> > > > Regards, >>>>> > > > >>>>> > > > Hervé >>>>> > > > >>>>> > > > > On Wed, Aug 24, 2016 at 1:52 PM, Stephen Connolly < >>>>> > > > > >>>>> > > > > stephen.alan.conno...@gmail.com <javascript:;>> wrote: >>>>> > > > > > On Tuesday 23 August 2016, Paul Benedict < >>>>> pbened...@apache.org >>>>> <javascript:;>> >>>>> >>>>> > wrote: >>>>> > > > > > > On Tue, Aug 23, 2016 at 5:27 PM, Christian Schulte < >>>>> > c...@schulte.it <javascript:;> >>>>> > > > > > > >>>>> > > > > > > <javascript:;>> wrote: >>>>> > > > > > > > Am 08/24/16 um 00:08 schrieb Paul Benedict: >>>>> > > > > > > > > POM and a future major version POM? I am hinting at a >>>>> > strategy >>>>> > > > >>>>> > > > for >>>>> > > > >>>>> > > > > > > > forward >>>>> > > > > > > > >>>>> > > > > > > > > compatibility. >>>>> > > > > > > > >>>>> > > > > > > > Is forward compatibility really needed/required? >>>>> > > > > > > >>>>> > > > > > > I honestly don't know, which is why I am asking. An answer >>>>> of >>>>> "no >>>>> > > > > > > compatibility" is possible, too. >>>>> > > > > > > >>>>> > > > > > > I can completely see the argument that says POMs must be >>>>> > > > > > > all-parseable-or-nothing -- for the sake of >>>>> reproducibility. >>>>> I >>>>> > > > >>>>> > > > actually >>>>> > > > >>>>> > > > > > > prefer this answer. I think it is sensible to fail a build >>>>> when >>>>> > > > > > > encountering a POM version too advanced. If your client >>>>> only >>>>> > > > >>>>> > > > supports up >>>>> > > > >>>>> > > > > > to >>>>> > > > > > >>>>> > > > > > > model 4.0.0, then all artifacts must be specified by 4.0.0 >>>>> > models, >>>>> > > > >>>>> > > > too. >>>>> > > > >>>>> > > > > > > On the other hand, I wanted to give the benefit of the >>>>> doubt >>>>> to >>>>> > the >>>>> > > > > > > opposite argument. At least one person spoke up and said >>>>> it's >>>>> > > > > > >>>>> > > > > > unacceptable >>>>> > > > > > >>>>> > > > > > > to fail a build on configuration you don't understand. >>>>> Well, >>>>> > that's >>>>> > > > >>>>> > > > an >>>>> > > > >>>>> > > > > > > interesting argument, too. That person doesn't want to tank >>>>> the >>>>> > > > > > > build >>>>> > > > > > > for >>>>> > > > > > > the 1% of configuration that can't be understood.... but I >>>>> fail >>>>> > to >>>>> > > > >>>>> > > > see >>>>> > > > >>>>> > > > > > > an >>>>> > > > > > > escape hatch here. If a client can't understand what's >>>>> being >>>>> > > > >>>>> > > > specified, >>>>> > > > >>>>> > > > > > > then what else can be done but fail? >>>>> > > > > > >>>>> > > > > > Strip the dependencies a and let the user fix up manually >>>>> (ideally >>>>> > by >>>>> > > > > > logging a warning that you are consuming a dependency using a >>>>> newer >>>>> > > > > > modelVersion) >>>>> > > > > > >>>>> > > > > > Everyone forgets that the 4.0.0 ship has sailed already, so >>>>> we >>>>> > have to >>>>> > > > > > deploy best-effort 4.0.0 poms >>>>> > > > > > >>>>> > > > > > Now I say that 3.4 should not have a new modelVersion but >>>>> what >>>>> it >>>>> > > > >>>>> > > > could do >>>>> > > > >>>>> > > > > > is be more forgiving of newer modelVersions or try and >>>>> download >>>>> and >>>>> > > > >>>>> > > > use an >>>>> > > > >>>>> > > > > > XSLT to convert newer modelVersions to 4.0.0 (while logging a >>>>> > warning) >>>>> > > > > > with >>>>> > > > > > option flags to allow failing the build if XSLT had to be >>>>> applied >>>>> > > > > > >>>>> > > > > > So let's bump the modelVersion in Maven 5.0.0 (there is no >>>>> Maven >>>>> > 4.x >>>>> > > > >>>>> > > > let's >>>>> > > > >>>>> > > > > > align on the modelVersion) >>>>> > > > > > >>>>> > > > > > That should have separation between builder Pom and consumer >>>>> Pom. >>>>> > For >>>>> > > > > > packaging=pom we deploy the builder Pom using >>>>> classifier=build >>>>> for >>>>> > all >>>>> > > > > > other packaging a we do not deploy the builder Pom. >>>>> > > > > > >>>>> > > > > > We deploy a *best effort* conversion of the consumer Pom to >>>>> > > > >>>>> > > > modelVersion >>>>> > > > >>>>> > > > > > 4.0.0 without a classifier and the consumer Pom gets deployed >>>>> as >>>>> > > > > > classifier >>>>> > > > > > consumer. >>>>> > > > > > >>>>> > > > > > The consumer Pom format allows for XSLT to transform to a >>>>> parsable >>>>> > > > >>>>> > > > syntax, >>>>> > > > >>>>> > > > > > if transform is required we log a warning (or fail the build >>>>> if >>>>> the >>>>> > > > > > builder >>>>> > > > > > Pom indicates strict modelVersion adherence) >>>>> > > > > > >>>>> > > > > > So yeah maven 5.x will be able to parse dependencies with >>>>> > modelVersion >>>>> > > > >>>>> > > > 6.x >>>>> > > > >>>>> > > > > > while logging warnings that the user may not have the correct >>>>> > > > >>>>> > > > dependency >>>>> > > > >>>>> > > > > > tree. That is IMHO acceptable forward compatibility >>>>> > > > > > >>>>> > > > > > HTH >>>>> > > > > > >>>>> > > > > > -Stephen >>>>> > > > > > >>>>> > > > > > Ps I'm really hoping someone has a less crappy solution that >>>>> > this... >>>>> > > > >>>>> > > > But I >>>>> > > > >>>>> > > > > > believe this is actually *a* solution... And prior to this I >>>>> have >>>>> > not >>>>> > > > >>>>> > > > seen >>>>> > > > >>>>> > > > > > any solution >>>>> > > > > > >>>>> > > > > > > Cheers, >>>>> > > > > > > Paul >>>>> > > > > > >>>>> > > > > > -- >>>>> > > > > > Sent from my phone >>>>> > > > >>>>> > > > ------------------------------------------------------------ >>>>> --------- >>>>> > > > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> <javascript:;> >>>>> > > > For additional commands, e-mail: dev-h...@maven.apache.org >>>>> <javascript:;> >>>>> > >>>>> > >>>>> > ------------------------------------------------------------ >>>>> --------- >>>>> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>>>> <javascript:;> >>>>> > For additional commands, e-mail: dev-h...@maven.apache.org >>>>> <javascript:;> >>>>> > >>>>> > >>>>> >>>>> >>>>> >>>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org >>> For additional commands, e-mail: dev-h...@maven.apache.org >>> >>> > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > -- Sent from my phone
