Le jeu. 10 févr. 2022 à 12:33, Otavio Rodolfo Piske <angusyo...@gmail.com> a écrit :
> This sounds great! Looking forward to trying it out. > I'll try to set up something in the coming days. > > Thanks for the tip about the " -Dmvnd.buildTime"! > > Looking at the build time report from maven daemon: do I understand > correctly that the last part of the report is the totals for each plugin? > Yes > > Looking at what we have, it seems camel-package-maven-plugin seems to be > one of the problematic ones. > For this plugin, it should be possible to look at the timestamp of the input files and compare it with the generated one. The difficult part may be to somehow extract all the input files, but this looks like a good possible improvement at first glance. What I did optimize is the fact that the output is not overwritten if it has not changed, it at least allows the compiler to skip rebuilding all files, so it did reduce the build time a lot. However, I'm not sure if it can be optimized from a clean build. A more in depth analysis with a profiler may help... Guillaume > > Kind regards > > > On Thu, Feb 10, 2022 at 11:24 AM Guillaume Nodet <gno...@apache.org> > wrote: > > > Fwiw, i did a lot of work on the build a while ago, but my main goal was > to > > optimize subsequent `mvnd -DskipTests` run. > > Ideall, in such cases, no jars should be changed at all, but that's not > the > > case completely, especially for maven plugins (try running `ls -ltr > > **/*.jar` after a build). There are pending changes in maven to fix that > > and I plan to push some changes in camel (mostly plugins updates) once > > those are released. > > > > In case you don't know the option, you can use the `mvnd > -Dmvnd.buildTime` > > option to gather statistics on the mojo execution time. > > > > In addition, I've been working a few weeks ago on the > > maven-build-cache-extension [1], which is a powerful extension that > > provides a full per-module build cache. The cache can be configured > > locally and remotely (where the output of a module will be downloaded > from > > the remote cache instead of being built locally). It currently requires > > some changes in maven-core that will only be released in maven 3.9.0. > > However, I can try to set up an experimental mvnd branch that would embed > > this extension if that's of any interest. > > > > Guillaume > > > > [1] https://github.com/apache/maven-build-cache-extension > > > > Le jeu. 10 févr. 2022 à 09:25, Otavio Rodolfo Piske < > angusyo...@gmail.com> > > a écrit : > > > > > Thanks for looking into this. > > > > > > On a related note, I think one area where we could investigate > potential > > > improvements to the compilation is by looking at the performance of our > > > maven plugins and code generators. > > > > > > For example the camel-endpointdsl module takes a long time to generate > > > sources and build (that one takes almost 2 minutes to run on GH). The > > > camel-componentdsl one takes about 1 minute. I suspect that if we could > > > trace why they are so slow, there may be a route for optimizing the > build > > > time (and if we are lucky, that could happen across the whole build). > > > > > > A secondary branch of investigation could be looking at whether we can > > > adjust our build so that we can have it running with "process-classes" > > > target, to avoid the costly process of compressing and installing the > > jars. > > > > > > Kind regards > > > > > > On Wed, Feb 9, 2022 at 2:41 PM Nicolas Filotto <nfilo...@talend.com> > > > wrote: > > > > > > > Hi again, > > > > > > > > Just to let you know that it appears that there are only 2 cores on > the > > > > target server which is obviously not enough to have an impact on the > > > build > > > > time. > > > > > > > > Regards, > > > > Nicolas > > > > ________________________________ > > > > From: Nicolas Filotto <nfilo...@talend.com> > > > > Sent: Wednesday, February 9, 2022 11:33 > > > > To: dev@camel.apache.org <dev@camel.apache.org> > > > > Subject: Re: Parallel Build in Camel > > > > > > > > !-------------------------------------------------------------------| > > > > This Message Is From an External Sender > > > > This message came from outside your organization. > > > > Exercise caution when opening attachments or clicking any > > > > links. > > > > |-------------------------------------------------------------------! > > > > > > > > Hi, > > > > > > > > Thank you very much for your feedbacks. Here is the related PR > > > > > > > > > > https://urldefense.com/v3/__https://github.com/apache/camel/pull/6907__;!!CiXD_PY!C-yfZjEVUOAkwpdj1K9URzQDT9hlsdcr4PMcsL2byb6MCaomvtqUJvVUoIKNkHE$ > > > > . > > > > > > > > Regards, > > > > Nicolas > > > > > > > > ________________________________ > > > > From: Otavio Rodolfo Piske <angusyo...@gmail.com> > > > > Sent: Wednesday, February 9, 2022 10:26 > > > > To: dev@camel.apache.org <dev@camel.apache.org> > > > > Subject: Re: Parallel Build in Camel > > > > > > > > !-------------------------------------------------------------------| > > > > This Message Is From an External Sender > > > > This message came from outside your organization. > > > > Exercise caution when opening attachments or clicking any > > > > links. > > > > |-------------------------------------------------------------------! > > > > > > > > Hi, > > > > > > > > I think it would be OK to give it a try. Please, feel free to open a > PR > > > > with this suggestion. > > > > > > > > Kind regards > > > > > > > > On Wed, Feb 9, 2022 at 10:22 AM Nicolas Filotto <nfilo...@talend.com > > > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > If the parallel build is something that can be used in the build > > > > pipeline, > > > > > why not simply starting by adding the -T option of maven to the > > > commands > > > > > launched by the build ( > > > > > > > > > > > > > > > https://urldefense.com/v3/__https://github.com/apache/camel/blob/main/.github/workflows/master-pr-build.yml*L39__;Iw!!CiXD_PY!EMmp3maGNIUrNLCyD1a__sEjX6IOh6rg2uTID5to5gtnpo19lPAOuN33k5dXjH0$ > > > > ) > > > > > as short/middle term solution before eventually considering the > maven > > > > > daemon? What do you think of it? > > > > > > > > > > Regards, > > > > > Nicolas > > > > > > > > > > ________________________________ > > > > > From: Otavio Rodolfo Piske <angusyo...@gmail.com> > > > > > Sent: Wednesday, February 9, 2022 10:03 > > > > > To: dev@camel.apache.org <dev@camel.apache.org> > > > > > Subject: Re: Parallel Build in Camel > > > > > > > > > > > !-------------------------------------------------------------------| > > > > > This Message Is From an External Sender > > > > > This message came from outside your organization. > > > > > Exercise caution when opening attachments or clicking any > > > > > links. > > > > > > |-------------------------------------------------------------------! > > > > > > > > > > Hi, > > > > > > > > > > Yeah, that helps a bit, although it's still quite a lot. One day > I'd > > > like > > > > > to take a closer look at some of the plugins we have on the build > to > > > see > > > > if > > > > > there's some easy speed up we could do. > > > > > > > > > > As for the Github, I think we depend on 2 things: > > > > > 1. I believe we need a Github action that contains and runs maven > > > daemon > > > > > 2. I believe this action needs to be approved somehow by the ASF. I > > am > > > > not > > > > > sure about this, though ... maybe others in the community know more > > > about > > > > > it. > > > > > > > > > > Kind regards > > > > > > > > > > > > > > > > > > > > As a recipient of an email from Talend, your contact personal data > > will > > > > be > > > > > on our systems. Please see our privacy notice (updated August 2020) > > at > > > > > Talend, Inc. <https://www.talend.com/contacts-privacy-policy/> > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Otavio R. Piske > > > > > > > > > > > > > > https://urldefense.com/v3/__http://orpiske.net__;!!CiXD_PY!EMmp3maGNIUrNLCyD1a__sEjX6IOh6rg2uTID5to5gtnpo19lPAOuN33LLnzx4c$ > > > > > > > > As a recipient of an email from Talend, your contact personal data > will > > > be > > > > on our systems. Please see our privacy notice (updated August 2020) > at > > > > Talend, Inc. <https://www.talend.com/contacts-privacy-policy/> > > > > > > > > > > > > > > > > As a recipient of an email from Talend, your contact personal data > will > > > be > > > > on our systems. Please see our privacy notice (updated August 2020) > at > > > > Talend, Inc. <https://www.talend.com/contacts-privacy-policy/> > > > > > > > > > > > > > > > > > > -- > > > Otavio R. Piske > > > http://orpiske.net > > > > > > > > > -- > > ------------------------ > > Guillaume Nodet > > > > > -- > Otavio R. Piske > http://orpiske.net > -- ------------------------ Guillaume Nodet
