Re: improve Maven performance
Hi, You can have a look to this plugin https://maven-incremental-build.dev.java.net/ I know it doesn't solve completely the problem but now after a modification or an update from scm, you can compile without a clean and be sure that all the children of a modified module will be recompiled. Now, our builds are safe and their times is more often than 5 minutes rather than 11 or 12, depends if modifications are in hight level modules or not. I still search the way to block the build process for modules with no modifications on her dependencies so if someone has more informations on how achieve it, it would be fine. Vincent On Wed, Sep 17, 2008 at 10:56 PM, Yann <[EMAIL PROTECTED]> wrote: > Hi, > > In our company lot of people complains that our maven 2 builds are too > long. > We use multi modules and most of the time we only need to build 1 or 2 > modules not all of them. > > So a solution could be to create a Maven 2 plugin that will check if a > source file has changed in the module. If no file has changed then it will > skip all other phases of the module (from generate-sources to packaging...). > > So I know how to bind a plugin to the first maven phase (validate). > But how to skip all others phases without interrupted the multi-module > build ? > > Thanks > Yann > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
Re: improve Maven performance
On Fri, Sep 19, 2008 at 5:40 AM, Yann <[EMAIL PROTECTED]> wrote: > Yes we break ours projects into small one. Actually, we follow a multi layer > architecture so for 1 EAR we can have up to 10 modules. > > It is difficult to break this application but I understood your point of > view... > > The issue is that we often need to test our application that means create > the full EAR (a full multi module build) and deploy it to the application > server. > And unfortunately all the tests can not be done out of the container... > So how do you mange this long lifecycle ? > Write tests that don't rely on the container. You use inversion of control and then create mocks for the dependencies. Your application can also benefit from this by using spring to manage the dependencies in the deployed application too. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: improve Maven performance
Yes we break ours projects into small one. Actually, we follow a multi layer architecture so for 1 EAR we can have up to 10 modules. It is difficult to break this application but I understood your point of view... The issue is that we often need to test our application that means create the full EAR (a full multi module build) and deploy it to the application server. And unfortunately all the tests can not be done out of the container... So how do you mange this long lifecycle ? Thanks Yann. Graham Leggett a écrit : Yann wrote: In our company lot of people complains that our maven 2 builds are too long. We use multi modules and most of the time we only need to build 1 or 2 modules not all of them. The ideal longer term solution is to break the project up into smaller sections, either individual artifacts with their own life cycle, or small multi module builds. Try and separate out code that tends to change often, and code that does not, and then put the code that doesn't change often into its own release cycle. You will also find you will have less problems this way, because each release is a release of less code, and therefore there is less opportunity for stuff to go wrong. Regards, Graham -- - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: improve Maven performance
I wasn't suggesting speeding up downloading things that are hosted on repo1 or wherever else, I'm saying install your companies artifacts as built by your CI server into archiva. Set the CI server to build/deploy/install on change. That way, if you have module 1 - 20 and a developer just wants to work on module 17 and 18, all the dependencies for that module say, are modules 1 - 10 will be in archiva, and they'll just be pulled down. -Original Message- From: Yeagle1 [mailto:[EMAIL PROTECTED] Sent: Thursday, September 18, 2008 4:05 AM To: users@maven.apache.org Subject: RE: improve Maven performance Hi, Brett: For us a generic way to skip things could be to check if all files in the modules has been updated (through the file timestamp) and compare it to a local cache Some plugins already do it (xmlbeans). So why not apply this logic to the full module ? The logic could be similar to the maven option "--fail-never" except it is not an error... Kalle: Yes, it is better to run only modules that need to, but most of the time in Eclipse we update lot of classes located in several modules. and when it is time to make a maven build we are never sure which module to run... So lot of our guys run all modules to be sure... EJ Ciramella-3: We already have our internal repository with Archiva. So we improved the artefact download phase... Thanks to all for your responses Yann. EJ Ciramella-3 wrote: > > Yeah, this is truly the solution - why not branch/aggregate the bits > you need? Set up nexus or archiva and have your own internal repository? > > -Original Message- > From: Kalle Korhonen [mailto:[EMAIL PROTECTED] > Sent: Wednesday, September 17, 2008 5:09 PM > To: Maven Users List > Subject: Re: improve Maven performance > > Also, if only few modules see activity, wouldn't it be easy to just > release and version the stable parts separately, rather than always > building everything? > > Kalle > > > On Wed, Sep 17, 2008 at 1:59 PM, Brett Porter > <[EMAIL PROTECTED]>wrote: > >> This really needs the support of other plugins to check if they need >> to run - you can't know a generic way to skip things. For that reason >> we haven't exposed a programmatic way to do that. >> >> I would recommend focusing on what the performance bottlenecks are >> and > >> improving them as much as possible. >> >> - Brett >> >> 2008/9/18 Yann <[EMAIL PROTECTED]>: >> > Hi, >> > >> > In our company lot of people complains that our maven 2 builds are >> > too >> long. >> > We use multi modules and most of the time we only need to build 1 >> > or > >> > 2 modules not all of them. >> > >> > So a solution could be to create a Maven 2 plugin that will check >> > if > >> > a source file has changed in the module. If no file has changed >> > then > >> > it >> will >> > skip all other phases of the module (from generate-sources to >> packaging...). >> > >> > So I know how to bind a plugin to the first maven phase (validate). >> > But how to skip all others phases without interrupted the >> > multi-module >> build >> > ? >> > >> > Thanks >> > Yann >> > >> > >> > --- >> > - >> > - To unsubscribe, e-mail: [EMAIL PROTECTED] >> > For additional commands, e-mail: [EMAIL PROTECTED] >> > >> > >> >> >> >> -- >> Brett Porter >> Blog: http://blogs.exist.com/bporter/ >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/improve-Maven-performance-tp19540953p19547988.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: improve Maven performance
Yann wrote: In our company lot of people complains that our maven 2 builds are too long. We use multi modules and most of the time we only need to build 1 or 2 modules not all of them. The ideal longer term solution is to break the project up into smaller sections, either individual artifacts with their own life cycle, or small multi module builds. Try and separate out code that tends to change often, and code that does not, and then put the code that doesn't change often into its own release cycle. You will also find you will have less problems this way, because each release is a release of less code, and therefore there is less opportunity for stuff to go wrong. Regards, Graham -- smime.p7s Description: S/MIME Cryptographic Signature
RE: improve Maven performance
Hi, Brett: For us a generic way to skip things could be to check if all files in the modules has been updated (through the file timestamp) and compare it to a local cache Some plugins already do it (xmlbeans). So why not apply this logic to the full module ? The logic could be similar to the maven option "--fail-never" except it is not an error... Kalle: Yes, it is better to run only modules that need to, but most of the time in Eclipse we update lot of classes located in several modules. and when it is time to make a maven build we are never sure which module to run... So lot of our guys run all modules to be sure... EJ Ciramella-3: We already have our internal repository with Archiva. So we improved the artefact download phase... Thanks to all for your responses Yann. EJ Ciramella-3 wrote: > > Yeah, this is truly the solution - why not branch/aggregate the bits you > need? Set up nexus or archiva and have your own internal repository? > > -Original Message- > From: Kalle Korhonen [mailto:[EMAIL PROTECTED] > Sent: Wednesday, September 17, 2008 5:09 PM > To: Maven Users List > Subject: Re: improve Maven performance > > Also, if only few modules see activity, wouldn't it be easy to just > release and version the stable parts separately, rather than always > building everything? > > Kalle > > > On Wed, Sep 17, 2008 at 1:59 PM, Brett Porter > <[EMAIL PROTECTED]>wrote: > >> This really needs the support of other plugins to check if they need >> to run - you can't know a generic way to skip things. For that reason >> we haven't exposed a programmatic way to do that. >> >> I would recommend focusing on what the performance bottlenecks are and > >> improving them as much as possible. >> >> - Brett >> >> 2008/9/18 Yann <[EMAIL PROTECTED]>: >> > Hi, >> > >> > In our company lot of people complains that our maven 2 builds are >> > too >> long. >> > We use multi modules and most of the time we only need to build 1 or > >> > 2 modules not all of them. >> > >> > So a solution could be to create a Maven 2 plugin that will check if > >> > a source file has changed in the module. If no file has changed then > >> > it >> will >> > skip all other phases of the module (from generate-sources to >> packaging...). >> > >> > So I know how to bind a plugin to the first maven phase (validate). >> > But how to skip all others phases without interrupted the >> > multi-module >> build >> > ? >> > >> > Thanks >> > Yann >> > >> > >> > >> > - To unsubscribe, e-mail: [EMAIL PROTECTED] >> > For additional commands, e-mail: [EMAIL PROTECTED] >> > >> > >> >> >> >> -- >> Brett Porter >> Blog: http://blogs.exist.com/bporter/ >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/improve-Maven-performance-tp19540953p19547988.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: improve Maven performance
Yeah, this is truly the solution - why not branch/aggregate the bits you need? Set up nexus or archiva and have your own internal repository? -Original Message- From: Kalle Korhonen [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 17, 2008 5:09 PM To: Maven Users List Subject: Re: improve Maven performance Also, if only few modules see activity, wouldn't it be easy to just release and version the stable parts separately, rather than always building everything? Kalle On Wed, Sep 17, 2008 at 1:59 PM, Brett Porter <[EMAIL PROTECTED]>wrote: > This really needs the support of other plugins to check if they need > to run - you can't know a generic way to skip things. For that reason > we haven't exposed a programmatic way to do that. > > I would recommend focusing on what the performance bottlenecks are and > improving them as much as possible. > > - Brett > > 2008/9/18 Yann <[EMAIL PROTECTED]>: > > Hi, > > > > In our company lot of people complains that our maven 2 builds are > > too > long. > > We use multi modules and most of the time we only need to build 1 or > > 2 modules not all of them. > > > > So a solution could be to create a Maven 2 plugin that will check if > > a source file has changed in the module. If no file has changed then > > it > will > > skip all other phases of the module (from generate-sources to > packaging...). > > > > So I know how to bind a plugin to the first maven phase (validate). > > But how to skip all others phases without interrupted the > > multi-module > build > > ? > > > > Thanks > > Yann > > > > > > > > - To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > -- > Brett Porter > Blog: http://blogs.exist.com/bporter/ > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: improve Maven performance
On Thu, Sep 18, 2008 at 6:38 AM, Kalle Korhonen <[EMAIL PROTECTED]> wrote: > Also, if only few modules see activity, wouldn't it be easy to just release > and version the stable parts separately, rather than always building > everything? Your development process should be: * Build in your IDE mostly - very quick * When ready to smoke test locally run the build - no greater than 10 minutes (http://martinfowler.com/articles/continuousIntegration.html) * Have your CI system build and email breakages We have a large Eclipse PDE client/server build with EJBs etc that takes 13 minutes. 7 of those is the crappy way eclipse PDE builds jar files by appending to it and the jar file being built 700Mb. And we build from the root of the project. Maven isn't the bottle neck here. Look at what you are building and see if you can fix the bottle necks. Or alternatively only build the modules that you need and rely on CI to pull in newer versions of the other artifacts. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: improve Maven performance
Also, if only few modules see activity, wouldn't it be easy to just release and version the stable parts separately, rather than always building everything? Kalle On Wed, Sep 17, 2008 at 1:59 PM, Brett Porter <[EMAIL PROTECTED]>wrote: > This really needs the support of other plugins to check if they need > to run - you can't know a generic way to skip things. For that reason > we haven't exposed a programmatic way to do that. > > I would recommend focusing on what the performance bottlenecks are and > improving them as much as possible. > > - Brett > > 2008/9/18 Yann <[EMAIL PROTECTED]>: > > Hi, > > > > In our company lot of people complains that our maven 2 builds are too > long. > > We use multi modules and most of the time we only need to build 1 or 2 > > modules not all of them. > > > > So a solution could be to create a Maven 2 plugin that will check if a > > source file has changed in the module. If no file has changed then it > will > > skip all other phases of the module (from generate-sources to > packaging...). > > > > So I know how to bind a plugin to the first maven phase (validate). > > But how to skip all others phases without interrupted the multi-module > build > > ? > > > > Thanks > > Yann > > > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > -- > Brett Porter > Blog: http://blogs.exist.com/bporter/ > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
Re: improve Maven performance
This really needs the support of other plugins to check if they need to run - you can't know a generic way to skip things. For that reason we haven't exposed a programmatic way to do that. I would recommend focusing on what the performance bottlenecks are and improving them as much as possible. - Brett 2008/9/18 Yann <[EMAIL PROTECTED]>: > Hi, > > In our company lot of people complains that our maven 2 builds are too long. > We use multi modules and most of the time we only need to build 1 or 2 > modules not all of them. > > So a solution could be to create a Maven 2 plugin that will check if a > source file has changed in the module. If no file has changed then it will > skip all other phases of the module (from generate-sources to packaging...). > > So I know how to bind a plugin to the first maven phase (validate). > But how to skip all others phases without interrupted the multi-module build > ? > > Thanks > Yann > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Brett Porter Blog: http://blogs.exist.com/bporter/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
improve Maven performance
Hi, In our company lot of people complains that our maven 2 builds are too long. We use multi modules and most of the time we only need to build 1 or 2 modules not all of them. So a solution could be to create a Maven 2 plugin that will check if a source file has changed in the module. If no file has changed then it will skip all other phases of the module (from generate-sources to packaging...). So I know how to bind a plugin to the first maven phase (validate). But how to skip all others phases without interrupted the multi-module build ? Thanks Yann - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
