Thanks Jesse, Tim. I've modified https://issues.apache.org/jira/browse/NETBEANS-4746 with links to your posts - I assume both are part of the problem: cold JVM for small projects, a lot of I/O for large project structures.
-jt Dne pátek 4. září 2020 5:26:21 CEST, Tim Boudreau napsal(a): > For larger multi-module projects, I'd tend to suspect that a substantial > chunk of the time is I/O-bound, parsing > XML - build one project with a substantial dependency tree. How many POM > files both in the project and > under ~/.m2 need to parsed? It would be interesting to instrument Maven > and get some timings out of that. > > My fairly sprawling Antlr-IDE-support + > build-ide-plugins-with-a-compiled-antlr-grammar-and-some-annotations > framework clocks in at 68 pom files (probably 20 of these are projects for > tests to run against and test-support > libraries). Certainly there are other in-the-ide performance issues > (change the version of one thing in the > dependency management section of the parent POM and plan not to use the IDE > for a few minutes while > scanning the universe brings the IDE to its knees). > > It would be worth doing some measurements of. I'd imagine if that proves > to be a root cause, some sort of > fast and queriable binary cache not unlike those the IDE uses for layer > contents and such in its cache > directory could be very helpful. The trick would be identifying those > changes to POM files that do *not* require > invalidating the cached POM of every project that depends on it. > > The 1.5 second improvement of a warm VM would be helpful on small projects, > but in any large tree of projects > where a build can take several minutes, it's not going to be significant. > > I don't know if small monolithic single-pom projects are the common case, > or multi-module projects. If multi-module > projects are the more common case, the warm-VM approach may not be worth > the engineering effort, given the > things that can go wrong and the effort required to defend against them. > You could have it internally cache and > keep hot information about dependencies to avoid I/O on subsequent builds, > but that could start to have a > substantial memory footprint. > > My experience running with, say, -T 8 can have a substantial impact on > build times, but it appears not to take > into account test dependencies when sorting the projects into a build > order, so if you run tests, at some point > you wind up running against something that is in the process of being built > and it crashes. > > -Tim > > On Thu, Sep 3, 2020 at 4:10 PM Jesse Glick <typr...@gmail.com> wrote: > > On Thu, Aug 27, 2020 at 1:30 AM Jaroslav Tulach > > > > <jaroslav.tul...@gmail.com> wrote: > > > Hypothesis: Probably different Ant and Maven integration into the > > > > NetBeans IDE. > > > > > While Ant is executed in the same JVM as the IDE in an isolated > > > > classloader, > > > > > Maven is probably always started as a separate process. > > > > Yes this likely accounts for a lot of the difference. > > > > For a regular edit-test-edit-test cycle you can use the misleadingly > > named “compile on save” feature to save time, but in the case of Maven > > projects it still does launch `mvn`, just with a minimal set of mojos. > > At one point Maven CoS used the IDE’s built-in test runner > > (incidentally implemented with in-JVM Ant), just passing along the > > classpath introspected from the POM, but we switched to launching > > actual Maven to get better compatibility with projects with subtler > > configuration (IIRC). > > > > At any rate, I like the idea of reusing a JVM in the style of `mvnsh` > > for repeated Maven invocations in routine developer workflows. It > > would be important to keep not only Maven core loaded, but mojo class > > loaders, since these are often large and expensive to initialize. You > > would have to tune it in various ways, for example > > > > · Shut down automatically after some idle period. > > · Avoid sharing an instance between projects. > > · Run a regular forked `mvn` for executions that smell production-y, > > such as anything using the `clean` or `install` phases. > > > > It is possible that given a warm JVM, we could dispense with any > > special CoS tricks and still get reasonable performance from the stock > > lifecycle up to the `test` phase. It depends in part on the project > > config: if your POM is requesting a ton of plugins run in earlier > > phases (CheckStyle, etc.), there will still be lots of overhead unless > > the IDE passes a bunch of `-DskipSomething` properties. > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org > > For additional commands, e-mail: dev-h...@netbeans.apache.org > > > > For further information about the NetBeans mailing lists, visit: > > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@netbeans.apache.org For additional commands, e-mail: dev-h...@netbeans.apache.org For further information about the NetBeans mailing lists, visit: https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists
