What was the third run? It was one thread with GraalVM? $ time mvnd install -DskipTests -Dinvoker.skip=true real 0m11,295s user 0m0,354s sys 0m0,176s
On Wed, Apr 7, 2021 at 7:09 PM Romain Manni-Bucau <rmannibu...@gmail.com> wrote: > Le mer. 7 avr. 2021 à 17:08, Tibor Digana <tibordig...@apache.org> a > écrit : > > > Romain, our builds are always downloading the artifacts. > > The I/O cannot be 0 time. We invest in safety builds rather than > > performance and so we rather download fresh artifacts. > > Other teams may cach the artifacts which may not be always so safe - > other > > preferences. > > > > Got it, note that redownloading a release is normally wrong and caching a > snapshot is also but this is your policy and a docker solution will not > solve that since you will need to update the docker base image as often as > you want updates (likely daily for snapshot) or still use snapshots so at > the end docker is not what helps *for this factor*. > > > > > > If you think that the Cache or AppCDS or GraalVM would have an effect, > feel > > free to make a prototype and try to measure the performance. > > Publish the benchmark test result and we will see how big percentage > > improvement it would be in categories (asciidoc, normal build, specific > > builds, whatever...) > > > > We already discussed it multiple times but graalvm boost or cds boost can > be siginificative but without rewriting *all* maven design gain is 0 (both > only work with a flat classpath - even a bit more constraints) so it means > no more isolated plugins, no more plexus-classworld etc, not sure we want > to do that anytime soon since it is a big feature of maven. > > > > > > As I can see the following article, such a benchmark test can be made in > a > > spare time: > > > > > https://medium.com/@toparvion/appcds-for-spring-boot-applications-first-contact-6216db6a4194 > > > Figures already had been shared on the list and it is not rare to have > around 30% boost on real life projects. > If you want something closer, maven-surefire figures are: > > $ time mvn install -DskipTests -Dinvoker.skip=true > real 0m29,606s > user 2m10,142s > sys 0m3,160s > > $ time mvn install -DskipTests -Dinvoker.skip=true -T15 # I have 16 core, > just to match mvnd defaults > real 0m19,494s > user 2m47,651s > sys 0m4,098s > > $ time mvnd install -DskipTests -Dinvoker.skip=true > real 0m11,295s > user 0m0,354s > sys 0m0,176s > > > 42% faster (I skipped tests to only include project pipeline but it is the > kind of boost the caching + long running daemon provides you) > > > > > > > > > > > Cheers > > T > > > > > > On Wed, Apr 7, 2021 at 3:52 PM Romain Manni-Bucau <rmannibu...@gmail.com > > > > wrote: > > > > > Hi Tibor, > > > > > > I see what you mean but I think this topic is actually unrelated to IO > > > since this remote latency is actually 0 in the case we are discussing > and > > > this case is generally solved by caching on all CI (jenkins, gh > actions, > > > travis for the ones I can speak about out of my head) - and locally by > > your > > > local m2 as you mentionned. > > > So overall the download time is always skipped in this daemon topic > since > > > it is a pay once cost that devs rarely face. > > > > > > What we care about is to a have sane defaults and the capacity to stop > > > creating and recreating ClassRealm with potentially slow init in these > > > realms (use asciidoctor to see something slow to create ;)). > > > A daemon enables to add a ton of caches which save a lot of time in > > > practise when rebuilding the same project. > > > Indeed part of these enhancements can be backported to maven core - and > > > thanks mvnd team to have done part of it - but not all of them since by > > > design the biggest slow part of a JVM is the classloading (it is one > part > > > GraalVM speeds up a lot by bypassing it), in particular with plexus > > > classrealm hierarchy and impl. > > > At least this feature can justify a daemon for me but also having an in > > > memory cache to not resolving and resolving is the second big feature > > maven > > > benefits a lot from what I saw (it is crazy how we loose time there). > > > Indeed, as soon as instances can be reused plugins could cache more > > without > > > breaking the "not daemon" cache and provide way more boosts but current > > > behavior is already impressive (I expected the boost to be important > but > > > less when I started the thread 4 years ago to be honest). > > > > > > Le mer. 7 avr. 2021 à 15:40, Tibor Digana <tibordig...@apache.org> a > > > écrit : > > > > > > > I think two years ago we were talking about Maven dockerization. > > > > We had the work in progress and I think I will be able to find it > > again. > > > > The Docker image included local repo. > > > > I think the biggest latencies are when you are downloading artifacts. > > > > > > Of course, you have one local repo, but that's suitable for those CI > > > > systems which do not want to override the local repo or share the > > > artifacts > > > > with other projects. It is our case in Apache. > > > > > > > > T > > > > > > > > On Wed, Apr 7, 2021 at 8:34 AM Guillaume Nodet <gno...@apache.org> > > > wrote: > > > > > > > > > Fwiw, Peter and I would be happy to donate mvnd to the Apache Maven > > > > > project, should you choose to accept it. > > > > > > > > > > Cheers, > > > > > Guillaume > > > > > > > > > > Le lun. 29 mars 2021 à 12:21, Romain Manni-Bucau < > > > rmannibu...@gmail.com> > > > > a > > > > > écrit : > > > > > > > > > > > Up 4 years later ;) > > > > > > > > > > > > Now mvnd exists and proves it is very interesting to have such a > > > > feature, > > > > > > should it be something which can fit maven standard delivery? > > > > > > If overall yes we can start by asking mvnd if it can be > contributed > > > > with > > > > > > main codebase and if not we can either decide to do our own (hope > > not > > > > ;)) > > > > > > or that maven does not care about caching/optimizations in its > > "core" > > > > and > > > > > > that it is only done in extensions (I know 3 "main" ones as of > > > today). > > > > > > > > > > > > Wdyt? > > > > > > > > > > > > Romain Manni-Bucau > > > > > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > > > > > <https://rmannibucau.metawerx.net/> | Old Blog > > > > > > <http://rmannibucau.wordpress.com> | Github < > > > > > > https://github.com/rmannibucau> | > > > > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > > > > > > < > > > > > > > > > > > > > > > > > > > > > https://www.packtpub.com/application-development/java-ee-8-high-performance > > > > > > > > > > > > > > > > > > > > > > > > > Le mer. 9 mars 2016 à 23:58, Jeff Jensen < > > > > > > jeffjen...@upstairstechnology.com> > > > > > > a écrit : > > > > > > > > > > > > > Thanks! It's running just fine. :-) It's very cool. Really > > nice > > > > > > > directory traversal and completion, colors, and > > commands/features I > > > > > don't > > > > > > > know yet! > > > > > > > > > > > > > > Very interesting timing diffs (for each casual test, I ran the > > > > command > > > > > > for > > > > > > > each multiple times to seed infra caches): > > > > > > > * on some asciidoc gen with "mvn generate-resources", it was > > about > > > > the > > > > > > > same duration as CLI but after running each about 10 times, > > > mvnshell > > > > > > saved > > > > > > > about 20% consistently (possibly due to JIT? besides directory > > > > > traversal, > > > > > > > this was the first things I did). This was my key use case for > > > > > wanting a > > > > > > > "mvn server" - handling situations like this with repeated runs > > > > > > (asciidoc, > > > > > > > site, etc.). > > > > > > > > > > > > > > * on a simple "mvn clean", mvnshell was about 2x faster than > > CLI. > > > > > > > > > > > > > > * on a small module build, "mvn install" was about 20% faster > > over > > > > CLI > > > > > > > (after a mvn clean for each). > > > > > > > > > > > > > > I look forward to trying more things. > > > > > > > > > > > > > > Nice to have, thank you Jason! > > > > > > > > > > > > > > > > > > > > > On Wed, Mar 9, 2016 at 4:17 PM, Jason Dillon < > > > jason.dil...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Jason, if you have a built version, do you mind adding it as > a > > > > > download > > > > > > > to > > > > > > > > > > > > > > > > the release files? > > > > > > > > > > > > > > > > I can make a binary of this, though I do plan on fixing it up > > so > > > > that > > > > > > > > folks can build it in the near future. > > > > > > > > > > > > > > > > Build up here for the moment: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.dropbox.com/sh/yvt1g43r2pr2scj/AADqM__VhJaFf_x57OiUEZXva?dl=0 > > > > > > > > > > > > > > > > gshell:master should be buildable with just central now, > > dangling > > > > ref > > > > > > to > > > > > > > > older version of jline for classifer=tests which was unused > and > > > > > > polluting > > > > > > > > the build dependencies. > > > > > > > > > > > > > > > > —jason > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > ------------------------ > > > > > Guillaume Nodet > > > > > > > > > > > > > > >
