Re: The state of Maven
Le 18/09/2017 à 15:04, Emmanuel Bourg a écrit :
> Quick update from the Maven front:
> - The removal of Maven 2 is almost complete. I'm waiting for
> aether-ant-tasks in the NEW queue to remove maven-ant-tasks, the last
> package holding maven2-core.
> - Some packages still fail to build due to missing junit/junit4
> dependencies. I'm fixing them as they are reported by the reproducible
> builds notifications.
> - The maven dependencies have been further improved, we are now down to
> 37 packages downloaded (11MB) for 15MB on disk. There is still room to
> shave ~2MB more by splitting libwagon2-java.
Hi all,
maven2-core is now gone and the removal of the duplicated dependencies
is almost done. I'm still wondering what to do with the plexus-utils and
plexus-utils2 packages. Ideally like the other plexus packages I'd like
to rename plexus-utils2 to plexus-utils, but plexus-utils has an epoch
and it hurts my sense of aesthetic. So I'm leaning toward keeping
plexus-utils2 and removing plexus-utils. Maybe in a few years once
plexus-utils is long forgotten we'll be able to rename
plexus-utils2->plexus-utils without the epoch.
I've uploaded the last optimizations to the Maven dependencies. Now
installing maven pulls only 32 packages (8.9MB) for 11.7MB on disk,
which is very close to the size of the upstream distribution. At this
point it's difficult to do better. We might be able to reduce the size
of the Maven metadata (/usr/share/maven-repo) which account for ~2MB of
the install size. For example we could use the relocation mechanism to
redirect the versioned poms to the 'debian' ones, this may reduce the
size of the metadata by ~40%. Alternatively we could simply remove the
versioned poms and keep only the generics ones ('debian', 1.x, etc), I
don't think they are really useful.
Emmanuel Bourg
Re: The state of Maven
Le 3/08/2017 à 10:31, Emmanuel Bourg a écrit : > Our maven package was rather fat a few releases > ago, there was even a Launchpad bug rightfully complaining about it. > Here is the result of 'apt-get install maven' since Wheezy (with the JRE > already installed): > wheezy: 129 packages, 95.3MB archives, 164MB on disk > jessie: 118 packages, 59.5MB archives, 92MB on disk > stretch: 82 packages, 29.8MB archives, 41MB on disk > unstable: 56 packages, 23.0MB archives, 30MB on disk > > The situation is improving but we can do better. The upstream binary for > Maven 3.5.0 weighs 8.2MB compressed, 10MB installed, so even with the > package overhead I expect to halve the current numbers. Hi all, Quick update from the Maven front: - The removal of Maven 2 is almost complete. I'm waiting for aether-ant-tasks in the NEW queue to remove maven-ant-tasks, the last package holding maven2-core. - Some packages still fail to build due to missing junit/junit4 dependencies. I'm fixing them as they are reported by the reproducible builds notifications. - The maven dependencies have been further improved, we are now down to 37 packages downloaded (11MB) for 15MB on disk. There is still room to shave ~2MB more by splitting libwagon2-java. Emmanuel Bourg
Re: The state of Maven
On Thu, 31 Aug 2017, Emmanuel Bourg wrote: > Le 5/08/2017 à 23:39, Thorsten Glaser a écrit : > > > The C function isatty() on the output filedescriptor, but in a > > build helper like that I’d expect it to always be disabled. > > I tried istty() but it's disabled when using pbuilder even if the > terminal supports colors. That's the idea: pbuilder's output is a logfile, NOT just the terminal, so it SHOULD NOT show the colours. bye, //mirabilos -- tarent solutions GmbH Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/ Tel: +49 228 54881-393 • Fax: +49 228 54881-235 HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941 Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg * **Besuchen Sie uns auf der dmexco!** 13. und 14. September 2017, Messe Köln,** Halle 7.1, Stand E042** Digital Marketing Exposition & Conference [www.tarent.de/dmexco](http://www.tarent.de/dmexco) Wir empfehlen unseren Vortrag "Wettbewerbsanalyse im Handel: Preisvergleich online und offline" am 13. September, 18:00 Uhr im Speaker's Forum / Broadway * **Visit us at dmexco!** September 13th and 14th, 2017, Messe Köln,** Hall 7.1, Booth E042** Digital Marketing Exposition & Conference [www.tarent.de/dmexco](http://www.tarent.de/dmexco) We recommend our presentation "Competitor analysis in retail: price comparison online and offline" on September 13th, at 6 pm at the Speaker's Corner / Broadway *
Re: The state of Maven
Le 5/08/2017 à 23:39, Thorsten Glaser a écrit : > The C function isatty() on the output filedescriptor, but in a > build helper like that I’d expect it to always be disabled. I tried istty() but it's disabled when using pbuilder even if the terminal supports colors. I used 'tput colors' in maven-debian-helper 2.2.3 to detect if the colors are supported, but the escape sequences still appear in the reproducible build logs. I'd like to avoid disabling the colors completely, if someone has a good idea I'm interested. Emmanuel Bourg
Re: The state of Maven
Le 3/08/2017 à 10:31, Emmanuel Bourg a écrit : > This caused some disruptions, not all are resolved yet, but > fortunately I've managed to avoid the trap of breaking Maven completely > and being unable to fix it. I take that claim back :) As I switched more packages from the older dependencies to the new ones I've hit a point where some packages expecting an older dependency but not declaring it directly now fail to run. Fortunately this affects rarely used packages like... maven-debian-helper :) I suspected this could happen but it was difficult to guess where it would strike. I'm rushing to upload a fix tonight, sorry for the disruption. Emmanuel Bourg
Re: The state of Maven
On Sat, 5 Aug 2017, Emmanuel Bourg wrote: > Le 3/08/2017 à 14:03, Thorsten Glaser a écrit : > > > I’d have preferred it doing something like tty detection, to behave > > nice with other OSS citizens, but oh well and it’s upstream’s issue. > > Interesting, do you know how to do that in Perl? We could perform the > detection in maven-debian-helper and pass the -B parameter if necessary. The C function isatty() on the output filedescriptor, but in a build helper like that I’d expect it to always be disabled. bye, //mirabilos -- tarent solutions GmbH Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/ Tel: +49 228 54881-393 • Fax: +49 228 54881-235 HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941 Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
Re: The state of Maven
Le 5/08/2017 à 13:15, Markus Koschany a écrit : > My question: What exactly can I/we do to help? Excellent question, thanks :) Some help migrating away from the older packages I mentioned would be nice (for example libplexus-utils-java which still has 50 reverse dependencies). Emmanuel Bourg
Re: The state of Maven
Le 3/08/2017 à 14:03, Thorsten Glaser a écrit : > I’d have preferred it doing something like tty detection, to behave > nice with other OSS citizens, but oh well and it’s upstream’s issue. Interesting, do you know how to do that in Perl? We could perform the detection in maven-debian-helper and pass the -B parameter if necessary. > My own desktop and the Jenkins are running unstable, so I always get > this kind of updates. Is there a way I can check the list of maven > plugins used/preferred by Debian, so I can enter them in the POM > (where it always warns when no version is given anyway) so that > others (e.g. my coworkers using *buntu) get the same versions but > I can use the packaged versions instead of it having to download > stuff from Central? Sorry I don't see an easy way to track the Maven plugins packaged in Debian. Someday it would be nice to index all the Maven artifacts packaged and have a search engine like http://www.mvnrepository.com to inspect them. That would be the right place to look for plugins in Debian. Emmanuel Bourg
Re: The state of Maven
On 03/08/17 04:31, Emmanuel Bourg wrote: Hi all, As you may have noticed I have updated many Maven related packages recently, and I'd like to summarize quickly what I'm doing and where we're heading. Hi, first of all thank you very much for updating this core build tool for Java. Maven in testing/unstable is currently at the latest upstream release (3.5.0). The most visible change since Maven 3.3.9 is the colored console output which is really nice (except for the builder logs, they are now cluttered with ANSI escape code and I don't know if there is a way to avoid that). If time allows I'll try to backport this version to Stretch. I like the new color output but the ANSI escape code makes it difficult to copy&paste log information. I would be great if we could avoid it. [...] Thanks for explaining the most important changes. My question: What exactly can I/we do to help? Once all of this is over I'd like to invest some time on maven-debian-helper to better support multi module projects. I have 3 improvements in mind: Sounds good to me. Markus
Re: The state of Maven
On Thu, 3 Aug 2017, Emmanuel Bourg wrote: > Hi all, > > As you may have noticed I have updated many Maven related packages > recently, and I'd like to summarize quickly what I'm doing and where > we're heading. > > Maven in testing/unstable is currently at the latest upstream release > (3.5.0). The most visible change since Maven 3.3.9 is the colored > console output which is really nice (except for the builder logs, they > are now cluttered with ANSI escape code and I don't know if there is a > way to avoid that). If time allows I'll try to backport this version to > Stretch. > > [...] > > The situation is improving but we can do better. The upstream binary for > Maven 3.5.0 weighs 8.2MB compressed, 10MB installed, so even with the > package overhead I expect to halve the current numbers. Hi Emmanuel, Thanks for your work on this area, the improvements are clearly notable. Cheers, Miguel. -- Miguel Landaeta, nomadium at debian.org secure email with PGP 0x6E608B637D8967E9 available at http://miguel.cc/key. "Faith means not wanting to know what is true." -- Nietzsche signature.asc Description: Digital signature
Re: The state of Maven
On Thu, 3 Aug 2017, Emmanuel Bourg wrote:
> Maven in testing/unstable is currently at the latest upstream release
> (3.5.0). The most visible change since Maven 3.3.9 is the colored
> console output which is really nice (except for the builder logs, they
> are now cluttered with ANSI escape code and I don't know if there is a
> way to avoid that). If time allows I'll try to backport this version to
It also *BREAKS* software, such as Jenkins (I manually had to patch
/var/lib/jenkins/plugins/maven-plugin/WEB-INF/lib/maven32-interceptor-1.6.jar
in order to get it to disable coloured output) and anything that, for
example, analyses the output of mvn dependency:{list,tree} — we use
that to compare the dependencies against ① a list of packages with
known security issues (alpha status) and ② a list of known dependencies
in the package, so that, when one changes, we know to manually adjust
the whole-WAR licence documentation (and check for compatibility, etc).
You can only disable it by adding -B on the mvn commandline, or by
calling a new method when not using the commandline (e.g. Jenkins).
I’d have preferred it doing something like tty detection, to behave
nice with other OSS citizens, but oh well and it’s upstream’s issue.
> redirects the version 3.1 to 3.6.1). This means we can leave the default
> versions unchanged in our maven package, and still be able to build our
> packages with the latest versions of the plugins. This change will
> probably go live with the next update of the maven package.
Oh, interesting.
Another integrator’s PoV question:
My own desktop and the Jenkins are running unstable, so I always get
this kind of updates. Is there a way I can check the list of maven
plugins used/preferred by Debian, so I can enter them in the POM
(where it always warns when no version is given anyway) so that
others (e.g. my coworkers using *buntu) get the same versions but
I can use the packaged versions instead of it having to download
stuff from Central?
> This update is also the opportunity to transition away from Maven 2.
Oh, nice. I’m not aware of an inhouse project still requiring
Maven 2 (and even the old ones build with Maven 3, sometimes
with little changes we do anyway to e.g. insert our new Nexus
repository or something).
> Once all of this is over I'd like to invest some time on
> maven-debian-helper to better support multi module projects. I have 3
Oh cool! That would help the (somewhat stalled) efforts I had
trying to get some of the software we use/write packaged for
Debian.
> With these changes I hope we'll able to better optimize the dependencies
Congratulations!
bye,
//mirabilos
--
tarent solutions GmbH
Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/
Tel: +49 228 54881-393 • Fax: +49 228 54881-235
HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941
Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg
The state of Maven
Hi all,
As you may have noticed I have updated many Maven related packages
recently, and I'd like to summarize quickly what I'm doing and where
we're heading.
Maven in testing/unstable is currently at the latest upstream release
(3.5.0). The most visible change since Maven 3.3.9 is the colored
console output which is really nice (except for the builder logs, they
are now cluttered with ANSI escape code and I don't know if there is a
way to avoid that). If time allows I'll try to backport this version to
Stretch.
Our version of Maven is almost on par with upstream. The only difference
is the default version of the plugins used since we've aligned them with
the versions in Debian to avoid build failures. For example someone
using our package in unstable to build its projects uses the
maven-compiler-plugin 3.6.1 by default instead of the version 3.1 with
the upstream binary, therefore the build may behave slightly
differently. I've figured out a simple solution to this problem: our
maven plugin packages now relocate the older version used by Maven to
the latest one (for example our package maven-compiler-plugin/3.6.1-2
redirects the version 3.1 to 3.6.1). This means we can leave the default
versions unchanged in our maven package, and still be able to build our
packages with the latest versions of the plugins. This change will
probably go live with the next update of the maven package.
After investigating a couple of Maven related libraries/plugins
responsible for build failures under Java 9 I realized that several
packages were lagging behind upstream, and some had Java 9 fixes ready
(such as maven-compiler-plugin 3.6.1). All Java 9 issues haven't been
addressed upstream yet, but we'll be better prepared to integrate the
Java 9 friendly versions if our packages are already up to date. For
this reason I've undertaken an update of all Maven/Plexus libraries and
plugins. This caused some disruptions, not all are resolved yet, but
fortunately I've managed to avoid the trap of breaking Maven completely
and being unable to fix it. That's the risk when playing with toolchain
packages that build depend on themselves ;) I'm currently waiting for
several packages in the NEW queue to move forward (the new
maven-reporting-api package will unlock ~10 plugin updates).
This update is also the opportunity to transition away from Maven 2.
Maven 2 is still in the archive in library form as libmaven2-core-java
and many plugins are still depending on it. This transition will allow
us to remove several libraries that were duplicated to support both
Maven 2 and Maven 3 (wagon vs wagon2, plexus-utils vs plexus-utils2,
plexus-classworlds vs plexus-classworlds2, plexus-containers vs
plexus-containers1.5, etc). I've already filed some removal requests and
there are more to come.
Once all of this is over I'd like to invest some time on
maven-debian-helper to better support multi module projects. I have 3
improvements in mind:
- Enable/disable the unit tests per module and not globally. I'm
thinking about an additional parameter in the debian/*.poms file that
would trigger the injection of a surefire configuration enabling or
disabling the tests.
- Build some modules without installing them. Sometimes modules contain
test helper classes used by other modules, and installing them pulls
many test libraries in the package dependencies.
- Resolve the dependencies per package, ${maven:Depends} would be
adapted to each binary package produced. This will allows us to easily
split several packages and have finer-grained dependencies.
With these changes I hope we'll able to better optimize the dependencies
of our applications. Our maven package was rather fat a few releases
ago, there was even a Launchpad bug rightfully complaining about it.
Here is the result of 'apt-get install maven' since Wheezy (with the JRE
already installed):
wheezy: 129 packages, 95.3MB archives, 164MB on disk
jessie: 118 packages, 59.5MB archives, 92MB on disk
stretch: 82 packages, 29.8MB archives, 41MB on disk
unstable: 56 packages, 23.0MB archives, 30MB on disk
The situation is improving but we can do better. The upstream binary for
Maven 3.5.0 weighs 8.2MB compressed, 10MB installed, so even with the
package overhead I expect to halve the current numbers.
Emmanuel Bourg
