Re: Fwd: FOSDEM 19 Debian Java talk
Le samedi 16 février 2019, 11:38:59 CET Emmanuel Bourg a écrit : > Le 15/02/2019 à 17:34, Hervé BOUTEMY a écrit : > > IIUC you don't use the same versions of dependencies (because of > > uniqueness of each lib in Debian [1]) as those used in a "normal > > unconstrained" build from source [2] > > > > I don't yet understand the details of Debian build system, if the symlinks > > are used at build time or only at runtime, but even if you the resulting > > jars of the Debian build from Maven source are the same binaries, the > > execution of Maven does not run with the same dependencies, then there > > may be subtle differences during the run (for example from jansi, or > > guice, or commons-cli or ...) > I try to stick as much as possible to the same versions of the > dependencies, especially the sensitive ones like Guice or Aether, but > there are still some divergences. I've reviewed the dependencies > installed in the /usr/share/maven/lib directory by the maven/3.6.0 > package in Debian unstable and we currently have the following differences: > > MavenDebian > commons-lang3 3.8.13.8 > commons-io 2.5 2.6 > guava 25.1 19.0 > jansi 1.17 1.17.1 > jansi-native1.17 1.18 > maven-shared-utils 3.2.13.3.0 > plexus-utils3.1.03.1.1 > wagon 3.2.03.3.1 > > The versions are slightly ahead to the ones in Maven 3.6.0, ahead does not always mean better :) but it's good to see a precise measure of the intentionally managed difference between unconstrained Maven distribution (which will become in the future a reproducible build) and Debian choice = where subtle differences in behaviour can happen for users = what should be clearly advertised to users > except for > Guava. Guava is only used by Guice for Preconditions, immutable > collections and other basic operations. I don't think the difference is > significant. At least, I've verified that the Guice 4.2.1 tests pass > with Guava 19.0. > > > IIUC, that uniqueness of every library is a Debian principle, then a Maven > > distribution from Debian cannot be equivalent to a "normal" distribution > > where library versions are purely selected by Maven source's pom.xml, > > without adding any constraint This is why having Debian's distribution > > display its name in "mvn -v" would make that more clear. > > > > Let's discuss a little bit about plugins: when Debian launches the "mvn" > > command to build some Debian package, does it use Maven Central like > > anybody? = where do the Maven plugins used during build come from? > There are two different cases: > 1. What Maven does when launched by a user > 2. What Maven does when it's used in a Debian package build > > > 1. When the user launches "mvn" to build a project it behaves as usual > by downloading the dependencies from Maven Central or the other > repositories defined in the pom. The Maven plugins also available as > Debian packages are never used. The only code compiled by Debian that is > executed is the set of dependencies found in the Maven lib directory > (/usr/share/maven/lib). > > The default plugin versions are also exactly identical. Before Maven > 3.5.0 we patched default-bindings.xml to align them with the versions > packaged in Debian. We stopped to preserve the same default behavior. > This is new in Debian 10 "Buster" (and the latest Ubuntu 18.04 LTS), the > current stable version Debian 9 "Stretch" has Maven 3.3.9 with different > default plugins. for users, depending on default plugins versions injected by default biondings is a bad practice: in Maven 3.7.0, MNG-6562 will provide a warning to help users avoid that https://issues.apache.org/jira/browse/MNG-6562 > > > 2. Debian builds are offline, completely disconnected from the network. > It's impossible to download dependencies or plugins from Maven Central. > That's why we recreate a Maven repository in /usr/share/maven-repo/ and > instruct Maven to fetch the dependencies there instead of Maven Central > when building a Debian package. This repository is populated by > installing the packaged plugins and dependencies. great, this is the key strategy that I wanted to discover > > > If there are Maven developers interested I can do a live demo showing > how Maven projects are built into Debian packages. yes, I'm really interested, because I have so many more detailed questions if we do that face to face in a Paris bar, I'll pay you a few beers :) but if many people are interested, having a remote session will be better: we'll have to wait for the beers... Regards, Hervé > > Emmanuel Bourg > > - > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail:
Re: Fwd: FOSDEM 19 Debian Java talk
Le 15/02/2019 à 17:34, Hervé BOUTEMY a écrit : > IIUC you don't use the same versions of dependencies (because of uniqueness > of each lib in Debian [1]) as those used in a "normal unconstrained" build > from source [2] > I don't yet understand the details of Debian build system, if the symlinks > are used at build time or only at runtime, but even if you the resulting jars > of the Debian build from Maven source are the same binaries, the execution of > Maven does not run with the same dependencies, then there may be subtle > differences during the run (for example from jansi, or guice, or commons-cli > or ...) I try to stick as much as possible to the same versions of the dependencies, especially the sensitive ones like Guice or Aether, but there are still some divergences. I've reviewed the dependencies installed in the /usr/share/maven/lib directory by the maven/3.6.0 package in Debian unstable and we currently have the following differences: MavenDebian commons-lang3 3.8.13.8 commons-io 2.5 2.6 guava 25.1 19.0 jansi 1.17 1.17.1 jansi-native1.17 1.18 maven-shared-utils 3.2.13.3.0 plexus-utils3.1.03.1.1 wagon 3.2.03.3.1 The versions are slightly ahead to the ones in Maven 3.6.0, except for Guava. Guava is only used by Guice for Preconditions, immutable collections and other basic operations. I don't think the difference is significant. At least, I've verified that the Guice 4.2.1 tests pass with Guava 19.0. > IIUC, that uniqueness of every library is a Debian principle, then a Maven > distribution from Debian cannot be equivalent to a "normal" distribution > where library versions are purely selected by Maven source's pom.xml, without > adding any constraint > This is why having Debian's distribution display its name in "mvn -v" would > make that more clear. > > Let's discuss a little bit about plugins: when Debian launches the "mvn" > command to build some Debian package, does it use Maven Central like anybody? > = where do the Maven plugins used during build come from? There are two different cases: 1. What Maven does when launched by a user 2. What Maven does when it's used in a Debian package build 1. When the user launches "mvn" to build a project it behaves as usual by downloading the dependencies from Maven Central or the other repositories defined in the pom. The Maven plugins also available as Debian packages are never used. The only code compiled by Debian that is executed is the set of dependencies found in the Maven lib directory (/usr/share/maven/lib). The default plugin versions are also exactly identical. Before Maven 3.5.0 we patched default-bindings.xml to align them with the versions packaged in Debian. We stopped to preserve the same default behavior. This is new in Debian 10 "Buster" (and the latest Ubuntu 18.04 LTS), the current stable version Debian 9 "Stretch" has Maven 3.3.9 with different default plugins. 2. Debian builds are offline, completely disconnected from the network. It's impossible to download dependencies or plugins from Maven Central. That's why we recreate a Maven repository in /usr/share/maven-repo/ and instruct Maven to fetch the dependencies there instead of Maven Central when building a Debian package. This repository is populated by installing the packaged plugins and dependencies. If there are Maven developers interested I can do a live demo showing how Maven projects are built into Debian packages. Emmanuel Bourg - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Fwd: FOSDEM 19 Debian Java talk
not sure we're on the same understanding. I'll try to focus on 2 specific topics Le jeudi 14 février 2019, 22:57:26 CET Emmanuel Bourg a écrit : > > - Are you making clear that you're not using the official Maven > > distribution, e.g. by adjust the info from 'mvn --version'? > > No we aren't, but that's worth considering. Note that as the Maven > reproducibility improves this will become unnecessary because at some > point we'll be able to build binaries strictly identical to the Apache ones. IIUC you don't use the same versions of dependencies (because of uniqueness of each lib in Debian [1]) as those used in a "normal unconstrained" build from source [2] [1] https://salsa.debian.org/java-team/maven/blob/master/debian/maven.links [2] https://maven.apache.org/ref/3.6.0/apache-maven/dependencies.html#Project_Transitive_Dependencies I don't yet understand the details of Debian build system, if the symlinks are used at build time or only at runtime, but even if you the resulting jars of the Debian build from Maven source are the same binaries, the execution of Maven does not run with the same dependencies, then there may be subtle differences during the run (for example from jansi, or guice, or commons-cli or ...) IIUC, that uniqueness of every library is a Debian principle, then a Maven distribution from Debian cannot be equivalent to a "normal" distribution where library versions are purely selected by Maven source's pom.xml, without adding any constraint This is why having Debian's distribution display its name in "mvn -v" would make that more clear. > > - What is the optimum way of distributing Maven sources? For example: > > also providing compile and package scripts (instead of calling > > maven-plugins)? > > The current system is mostly fine to us. The only issue I got recently > was the embedding of the SLF4J sources at build time, because we don't > build binary packages installing the sources artifacts of the libraries > and they aren't available at build time (we could though, that's a > little more work). In this case I patched the build [4] to embed the > compiled classes with the shade plugin instead (the patch was refused > though [5], but that's a fairly minor divergence). Let's discuss a little bit about plugins: when Debian launches the "mvn" command to build some Debian package, does it use Maven Central like anybody? = where do the Maven plugins used during build come from? Regards, Hervé - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Fwd: FOSDEM 19 Debian Java talk
Hi Robert, Some comments with my Maven package maintainer hat. Le 12/02/2019 à 20:09, Robert Scholte a écrit : > I'm also wondering how you build Maven, since Maven is > being built with Maven. That should be a challenge to also rebuild all > plugins, etc. The maven package is rebuilt with the version of Maven previously packaged, that's not really a problem now. In the past we used a convoluted Ant build mimicking the Maven phases but that's history now. The source code is fetched from the release tags on the Git repositories (the tags are polled everyday and we're warned when a new release is available). One project (library, plugin, etc) is mapped to one package, and the upgrade process is done manually. It isn't terribly difficult but that's quite time consuming. If there are Debian/Ubuntu users in the Maven community interested in helping they are very welcome. > And how do you test this and confirm that it works as the official > distribution? > Sure, *IF* Maven is 100% reproducible then you can rely on our > test-infra, but that's not the situation. Debian has a CI infra rebuilding 550+ packages using Maven as build tool, so regressions tend to be caught fairly quickly (and immediately [1] reported [2][3]). Also the version provided in the stable release is available after months of user testing. So I'm pretty confident the Debian package is true to the Apache binaries. > So here are my main questions: > - Are you making clear that you're not using the official Maven > distribution, e.g. by adjust the info from 'mvn --version'? No we aren't, but that's worth considering. Note that as the Maven reproducibility improves this will become unnecessary because at some point we'll be able to build binaries strictly identical to the Apache ones. > - What is the optimum way of distributing Maven sources? For example: > also providing compile and package scripts (instead of calling > maven-plugins)? The current system is mostly fine to us. The only issue I got recently was the embedding of the SLF4J sources at build time, because we don't build binary packages installing the sources artifacts of the libraries and they aren't available at build time (we could though, that's a little more work). In this case I patched the build [4] to embed the compiled classes with the shade plugin instead (the patch was refused though [5], but that's a fairly minor divergence). > Interesting :) We've been discussing how we could get more contributors > and Kotlin was one idea, but that one didn't make it. > Even though we as Maven developers don't like the wrapper, the community > is asking for it, so we're seriously considering to add it as part of > Maven Core. The wrapper would have no significant impact on the Debian packaging, we just remove it before assembling the source package (no binaries are allowed). Emmanuel Bourg [1] https://issues.apache.org/jira/browse/MJAVADOC-504 [2] https://issues.apache.org/jira/browse/MPLUGIN-339 [3] https://issues.apache.org/jira/browse/SUREFIRE-1422 [4] https://salsa.debian.org/java-team/maven/blob/master/debian/patches/slf4j-compatibility.patch [5] https://github.com/apache/maven/pull/118 - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Re: Fwd: FOSDEM 19 Debian Java talk
I have proposed a similar approach with maven.vendor in the build.properties. > Gesendet: Donnerstag, 14. Februar 2019 um 14:24 Uhr > Von: "Gary Gregory" > An: "Maven Developers List" > Cc: "Robert Scholte" , "Dalibor Topic" > , debian-j...@lists.debian.org, "Matthias Klose" > > Betreff: Re: Fwd: FOSDEM 19 Debian Java talk > > Jar manifest files carry data like "built-by" and implementation > information: > > https://docs.oracle.com/javase/tutorial/deployment/jar/packageman.html > > Why not reuse "Implementation-Vendor" or invent a new entry and put > "Debian" in there. Maven can display this additional information on "mvn > -version". > > Gary > > On Thu, Feb 14, 2019 at 4:56 AM Markus Koschany wrote: > > > Hi Robert, > > > > Am 12.02.19 um 20:09 schrieb Robert Scholte: > > [...] > > > Hi Markus, > > > > > > first of all thanks for the insights, it is important for us to know how > > > Maven is used and in which way we can improve that way-of-work. Hervé is > > > already working hard on the reproducible builds specs with your team in > > > order to find out how we can improve our maven-plugins to get > > > reproducible artifacts. > > > > > > Maven itself is not 100% reproducible. We've learned that some Linux > > > vendors rebuild Maven and the presentation confirmed that Debian is one > > > of those vendors. What we've seen in the past is that sometimes people > > > are having issues with Maven and after a while we discovered that they > > > were not using the official Apache Maven distribution[1]. For us it is > > > quite easy to say: sorry, not our official distribution, please contact > > > your Linux distributor. > > > In such case we have 3 losers: the user, the Apache Maven project and > > > the Linux Distributor. If only the official Maven distribution was used, > > > then we would have had 3 winners. > > > > > > When you decide to rebuild Maven, you're also taking all related > > > responsibilities. > > > > We hear this a lot and it seems to be more common in the Java community. > > From Debian's point of view (other distributions like Fedora share the > > same view) it is essential being able to rebuild software from source. > > The prerequisite is the availability of source code of course. Most of > > us find it even strange when upstream developers recommend to use their > > prebuilt versions. Do they have something to hide? Why can't they just > > make building from source easy and trivial? > > > > We believe that every user should have the ability to modify software > > but this is only possible if she can build it from source. We go to > > great lengths to ensure that all software complies with Debian's free > > software guidelines. For the enduser the language and build tools of a > > certain piece of software almost become meaningless. They just type "apt > > source maven", change into the maven directory and rebuild the software > > with another one-liner. > > > > In case of Maven I don't see where our release differs fundamentally > > from your binary releases. As I said there is only one reproducibility > > patch left that doesn't change the functionality at all. So what we do > > is grab your sources from https://github.com/apache/maven/releases or > > maven.apache.org. In our opinion, without making any significant > > changes, it should just behave as your binary release when we build it > > from source. Otherwise there is source code missing or different. > > > > > I'm also wondering how you build Maven, since Maven is > > > being built with Maven. That should be a challenge to also rebuild all > > > plugins, etc. > > > And how do you test this and confirm that it works as the official > > > distribution? > > > Sure, *IF* Maven is 100% reproducible then you can rely on our > > > test-infra, but that's not the situation. > > > > It is a challenge to build Maven from source. We solved the > > bootstrapping problem and now we can use Debian's Maven version to build > > newer versions but we have to follow a certain build order when we make > > an update. > > > > > So here are my main questions: > > > - Are you making clear that you're not using the official Maven > > > distribution, e.g. by adjust the info from 'mvn --version'? > > > > This is how it looks on Debian 9 "Stretch" at the moment. > > > &g
Re: Fwd: FOSDEM 19 Debian Java talk
Jar manifest files carry data like "built-by" and implementation information: https://docs.oracle.com/javase/tutorial/deployment/jar/packageman.html Why not reuse "Implementation-Vendor" or invent a new entry and put "Debian" in there. Maven can display this additional information on "mvn -version". Gary On Thu, Feb 14, 2019 at 4:56 AM Markus Koschany wrote: > Hi Robert, > > Am 12.02.19 um 20:09 schrieb Robert Scholte: > [...] > > Hi Markus, > > > > first of all thanks for the insights, it is important for us to know how > > Maven is used and in which way we can improve that way-of-work. Hervé is > > already working hard on the reproducible builds specs with your team in > > order to find out how we can improve our maven-plugins to get > > reproducible artifacts. > > > > Maven itself is not 100% reproducible. We've learned that some Linux > > vendors rebuild Maven and the presentation confirmed that Debian is one > > of those vendors. What we've seen in the past is that sometimes people > > are having issues with Maven and after a while we discovered that they > > were not using the official Apache Maven distribution[1]. For us it is > > quite easy to say: sorry, not our official distribution, please contact > > your Linux distributor. > > In such case we have 3 losers: the user, the Apache Maven project and > > the Linux Distributor. If only the official Maven distribution was used, > > then we would have had 3 winners. > > > > When you decide to rebuild Maven, you're also taking all related > > responsibilities. > > We hear this a lot and it seems to be more common in the Java community. > From Debian's point of view (other distributions like Fedora share the > same view) it is essential being able to rebuild software from source. > The prerequisite is the availability of source code of course. Most of > us find it even strange when upstream developers recommend to use their > prebuilt versions. Do they have something to hide? Why can't they just > make building from source easy and trivial? > > We believe that every user should have the ability to modify software > but this is only possible if she can build it from source. We go to > great lengths to ensure that all software complies with Debian's free > software guidelines. For the enduser the language and build tools of a > certain piece of software almost become meaningless. They just type "apt > source maven", change into the maven directory and rebuild the software > with another one-liner. > > In case of Maven I don't see where our release differs fundamentally > from your binary releases. As I said there is only one reproducibility > patch left that doesn't change the functionality at all. So what we do > is grab your sources from https://github.com/apache/maven/releases or > maven.apache.org. In our opinion, without making any significant > changes, it should just behave as your binary release when we build it > from source. Otherwise there is source code missing or different. > > > I'm also wondering how you build Maven, since Maven is > > being built with Maven. That should be a challenge to also rebuild all > > plugins, etc. > > And how do you test this and confirm that it works as the official > > distribution? > > Sure, *IF* Maven is 100% reproducible then you can rely on our > > test-infra, but that's not the situation. > > It is a challenge to build Maven from source. We solved the > bootstrapping problem and now we can use Debian's Maven version to build > newer versions but we have to follow a certain build order when we make > an update. > > > So here are my main questions: > > - Are you making clear that you're not using the official Maven > > distribution, e.g. by adjust the info from 'mvn --version'? > > This is how it looks on Debian 9 "Stretch" at the moment. > > Apache Maven 3.3.9 > Maven home: /usr/share/maven > Java version: 1.8.0_181, vendor: Oracle Corporation > Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre > Default locale: de_DE, platform encoding: UTF-8 > OS name: "linux", version: "4.9.0-8-amd64", arch: "amd64", family: "unix" > > It is obvious from Maven home I guess but there is no special emphasis > on Debian because when you install Maven from Debian you will never get > a prebuilt binary release, this is implicit for all software in Debian's > main distribution. > > > - What is the optimum way of distributing Maven sources? For example: > > also providing compile and package scripts (instead of calling > > maven-plugins)? > > The major headache for us is the initial bootstrapping of new compilers > or build tools. We often write our own Ant scripts to solve the chicken > and egg problem. For Maven this is currently solved but if I recall > correctly there are sometimes plugins that require a certain Maven > version which in turn requires a specific plugin version, a classic > dependency loop. So if there was a way to build Maven without Maven or > certain plugins that would obviously make our life easier. >
Re: Fwd: FOSDEM 19 Debian Java talk
Hi Robert, Am 12.02.19 um 20:09 schrieb Robert Scholte: [...] > Hi Markus, > > first of all thanks for the insights, it is important for us to know how > Maven is used and in which way we can improve that way-of-work. Hervé is > already working hard on the reproducible builds specs with your team in > order to find out how we can improve our maven-plugins to get > reproducible artifacts. > > Maven itself is not 100% reproducible. We've learned that some Linux > vendors rebuild Maven and the presentation confirmed that Debian is one > of those vendors. What we've seen in the past is that sometimes people > are having issues with Maven and after a while we discovered that they > were not using the official Apache Maven distribution[1]. For us it is > quite easy to say: sorry, not our official distribution, please contact > your Linux distributor. > In such case we have 3 losers: the user, the Apache Maven project and > the Linux Distributor. If only the official Maven distribution was used, > then we would have had 3 winners. > > When you decide to rebuild Maven, you're also taking all related > responsibilities. We hear this a lot and it seems to be more common in the Java community. From Debian's point of view (other distributions like Fedora share the same view) it is essential being able to rebuild software from source. The prerequisite is the availability of source code of course. Most of us find it even strange when upstream developers recommend to use their prebuilt versions. Do they have something to hide? Why can't they just make building from source easy and trivial? We believe that every user should have the ability to modify software but this is only possible if she can build it from source. We go to great lengths to ensure that all software complies with Debian's free software guidelines. For the enduser the language and build tools of a certain piece of software almost become meaningless. They just type "apt source maven", change into the maven directory and rebuild the software with another one-liner. In case of Maven I don't see where our release differs fundamentally from your binary releases. As I said there is only one reproducibility patch left that doesn't change the functionality at all. So what we do is grab your sources from https://github.com/apache/maven/releases or maven.apache.org. In our opinion, without making any significant changes, it should just behave as your binary release when we build it from source. Otherwise there is source code missing or different. > I'm also wondering how you build Maven, since Maven is > being built with Maven. That should be a challenge to also rebuild all > plugins, etc. > And how do you test this and confirm that it works as the official > distribution? > Sure, *IF* Maven is 100% reproducible then you can rely on our > test-infra, but that's not the situation. It is a challenge to build Maven from source. We solved the bootstrapping problem and now we can use Debian's Maven version to build newer versions but we have to follow a certain build order when we make an update. > So here are my main questions: > - Are you making clear that you're not using the official Maven > distribution, e.g. by adjust the info from 'mvn --version'? This is how it looks on Debian 9 "Stretch" at the moment. Apache Maven 3.3.9 Maven home: /usr/share/maven Java version: 1.8.0_181, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre Default locale: de_DE, platform encoding: UTF-8 OS name: "linux", version: "4.9.0-8-amd64", arch: "amd64", family: "unix" It is obvious from Maven home I guess but there is no special emphasis on Debian because when you install Maven from Debian you will never get a prebuilt binary release, this is implicit for all software in Debian's main distribution. > - What is the optimum way of distributing Maven sources? For example: > also providing compile and package scripts (instead of calling > maven-plugins)? The major headache for us is the initial bootstrapping of new compilers or build tools. We often write our own Ant scripts to solve the chicken and egg problem. For Maven this is currently solved but if I recall correctly there are sometimes plugins that require a certain Maven version which in turn requires a specific plugin version, a classic dependency loop. So if there was a way to build Maven without Maven or certain plugins that would obviously make our life easier. [...] >> Our biggest challenge is Gradle. If Robert wants to help us then he >> should never rewrite parts of Maven in Kotlin or encourage developers to >> use a specific prebuilt version of Maven and to ship a script in every >> project that downloads it from the internet (gradle-wrapper). ;) > > Interesting :) We've been discussing how we could get more contributors > and Kotlin was one idea, but that one didn't make it. > Even though we as Maven developers don't like the wrapper, the community > is asking for it, so we're
Re: Fwd: FOSDEM 19 Debian Java talk
Am 2019-02-12 um 20:09 schrieb Robert Scholte: On Tue, 12 Feb 2019 12:34:56 +0100, Markus Koschany wrote: Hello, Dalibor Topic (Oracle) and Robert Scholte (Apache Maven) contacted me and were so kind to agree to make this discussion public, so that others can chime in too. I would like to use the opportunity to answer the initial question "what we are interested in seeing better supported from build tools" and give some general feedback about integrating Java into Debian. First of all Ant and Maven are most likely the best supported build systems at the moment. We carry only two patches for Maven, one because we use a newer version of SLF4j [1] and the second one is to make Maven builds reproducible. [2] It looks like [1] has been already merged upstream but [2] has not been forwarded yet. It would be great of course, if Maven builds would be reproducible out-of-the-box. In general I would like to see reproducible builds everywhere. Hi Markus, first of all thanks for the insights, it is important for us to know how Maven is used and in which way we can improve that way-of-work. Hervé is already working hard on the reproducible builds specs with your team in order to find out how we can improve our maven-plugins to get reproducible artifacts. Maven itself is not 100% reproducible. We've learned that some Linux vendors rebuild Maven and the presentation confirmed that Debian is one of those vendors. What we've seen in the past is that sometimes people are having issues with Maven and after a while we discovered that they were not using the official Apache Maven distribution[1]. For us it is quite easy to say: sorry, not our official distribution, please contact your Linux distributor. In such case we have 3 losers: the user, the Apache Maven project and the Linux Distributor. If only the official Maven distribution was used, then we would have had 3 winners. When you decide to rebuild Maven, you're also taking all related responsibilities. I'm also wondering how you build Maven, since Maven is being built with Maven. That should be a challenge to also rebuild all plugins, etc. And how do you test this and confirm that it works as the official distribution? Sure, *IF* Maven is 100% reproducible then you can rely on our test-infra, but that's not the situation. So here are my main questions: - Are you making clear that you're not using the official Maven distribution, e.g. by adjust the info from 'mvn --version'? I expressed my proposal to Hervé that we need a new property: maven.vendor. Our official distribution will carry the value: ASF. Everyone else who modifies the content must change the value in the build.properties. Thus, we will quickly know that this distro has been modified by someone else. Michael - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org
Re: Fwd: FOSDEM 19 Debian Java talk
On Tue, 12 Feb 2019 12:34:56 +0100, Markus Koschany wrote: Hello, Dalibor Topic (Oracle) and Robert Scholte (Apache Maven) contacted me and were so kind to agree to make this discussion public, so that others can chime in too. I would like to use the opportunity to answer the initial question "what we are interested in seeing better supported from build tools" and give some general feedback about integrating Java into Debian. First of all Ant and Maven are most likely the best supported build systems at the moment. We carry only two patches for Maven, one because we use a newer version of SLF4j [1] and the second one is to make Maven builds reproducible. [2] It looks like [1] has been already merged upstream but [2] has not been forwarded yet. It would be great of course, if Maven builds would be reproducible out-of-the-box. In general I would like to see reproducible builds everywhere. Hi Markus, first of all thanks for the insights, it is important for us to know how Maven is used and in which way we can improve that way-of-work. Hervé is already working hard on the reproducible builds specs with your team in order to find out how we can improve our maven-plugins to get reproducible artifacts. Maven itself is not 100% reproducible. We've learned that some Linux vendors rebuild Maven and the presentation confirmed that Debian is one of those vendors. What we've seen in the past is that sometimes people are having issues with Maven and after a while we discovered that they were not using the official Apache Maven distribution[1]. For us it is quite easy to say: sorry, not our official distribution, please contact your Linux distributor. In such case we have 3 losers: the user, the Apache Maven project and the Linux Distributor. If only the official Maven distribution was used, then we would have had 3 winners. When you decide to rebuild Maven, you're also taking all related responsibilities. I'm also wondering how you build Maven, since Maven is being built with Maven. That should be a challenge to also rebuild all plugins, etc. And how do you test this and confirm that it works as the official distribution? Sure, *IF* Maven is 100% reproducible then you can rely on our test-infra, but that's not the situation. So here are my main questions: - Are you making clear that you're not using the official Maven distribution, e.g. by adjust the info from 'mvn --version'? - What is the optimum way of distributing Maven sources? For example: also providing compile and package scripts (instead of calling maven-plugins)? [1] https://maven.apache.org/download.cgi Otherwise we have two build tools / Maven plugins called maven-debian-helper [3] and maven-repo-helper [4] that do all the Debian specific operations. Maven already supports local repositories and offline mode but I would like to see native support for unversioned jar files and dependencies too. At the moment we create our own local repository in /usr/share/maven-repo and in addition to the normal version, we have a so called "debian" version. Other Java projects in Debian only reference the debian version, so that we have to maintain only one library or application and when we decide to upgrade a package, reverse-depdencies continue to work because they use the unversioned "debian" instead of a specific version. In my experience other languages like Perl or Python, which are less version-centric, support this use case better. Regarding javadoc generation we would like to see an option that basically reverts to pre OpenJDK8 and simply is less strict than the current implementation. We currently use the undocumented and unsupported --ignore-source-errors option when we build javadoc. It is not feasible for us to fix all those syntax errors ourselves and we will rather ditch our documentation packages should --ignore-source-errors go away. Our biggest challenge is Gradle. If Robert wants to help us then he should never rewrite parts of Maven in Kotlin or encourage developers to use a specific prebuilt version of Maven and to ship a script in every project that downloads it from the internet (gradle-wrapper). ;) Interesting :) We've been discussing how we could get more contributors and Kotlin was one idea, but that one didn't make it. Even though we as Maven developers don't like the wrapper, the community is asking for it, so we're seriously considering to add it as part of Maven Core. regards, Robert That is all off the top of my head. Maybe someone else on the list wants to chime in here. Regards, Markus [1] https://sources.debian.org/src/maven/3.6.0-1/debian/patches/slf4j-compatibility.patch/ [2] https://sources.debian.org/src/maven/3.6.0-1/debian/patches/reproducible-build-timestamp.patch/ [3] https://tracker.debian.org/pkg/maven-debian-helper [4] https://tracker.debian.org/pkg/maven-repo-helper - To
