AW: Independent module release strategies
Well after quite a lot of digging in the code of the release plugin, I think that I will use a different strategy: - in the release.properties or using the commandline, I am able to set individual versions of my modules and the release plugin will use them http://maven.apache.org/plugins/maven-release-plugin/examples/non-interactive-release.html which is basically what I wanted in the first place. - In order to release only individual projects, using the advanced reactor otions of maven seems ideal: http://www.sonatype.com/people/2009/10/maven-tips-and-tricks-advanced-reactor-options/ So in order to have exactly what I wanted, all I need to do, is create a release.properties file to tell the plugin the versions and then limit the maven reactor to only the modules I want to release and use the normal release:prepare and release:perform targets. As creating the properties and commandline arguments I a little uncomfortable, I'm currently wokring on a jenking plugin for my client to select which modules to release and to provide versions for them ... hopefully this way releasing a new version is just a number of clicks and providing version numbers. What do you think about this solution? Any Kittens in danger this time? ;-) Chris Von: Stephen Connolly [stephen.alan.conno...@gmail.com] Gesendet: Dienstag, 16. Oktober 2012 11:06 An: Maven Users List Cc: rwhee...@artifact-software.com Betreff: Re: Independent module release strategies On 16 October 2012 07:45, christofer.d...@c-ware.de christofer.d...@c-ware.de wrote: As described in my other response. Simply keeping the versions in sync is not an option for us due to the donwtime this would mean for our clients and the load this would generate on the central servers. Well currently the approach to release a new version was to have all modules defined in the master pom modules-section as well as a dependencyManagement-section that defines the versions of all the modules. Now if a new build was to be made that updates only some of the modules, the other modules (the ones that should stay the same) were commented out of the master poms modules-section and then the releaseplugin was used to release the desired artifacts. After the release was finished the versions hat do be manually updated. This process really sucked and caused a lot of problems. Now my approach was not to use the release plugin at all and to define all of the versions used throughout the entire project in the master pom using properties. So all I had to do was to increase the versions in the release profile to the versions I want and commit that. Now in jenkins I was able to define some jobs to run mvn deploy for individual projects with turned on release profile. well first off, in my experience, the use of profiles to modify the dependencies is bad karma. many kittens will die if you follow that use of profiles. no matter how clean you think it is, with the current versions of Maven and their current behaviour, attempts to follow this path will result in many dead kittens underfoot. To me this seems a lot cleaner than all other approaches, but as I don't want to kill too many kittens (As stephen on the dev-list called it). So I'm open for other suggestions or explanationy WHY this is bad. Stephen claimed that if I re-define my properties in child modules, I would have really big trouble, but we are developing the entire project and this is a thing we could fefinitely rule out because it should be really easy to enforce such a constraint (Versions are defines solely in the master pom). And as I mentioned, to me it looks more like a highly restricted feature than a bug in maven that I was able to use a variable in the version. Not quite, I fear it is loosing something in translation for you. In an ideal world, before deploying the pom into the local cache (i.e. install:install) or remote repository (i.e. deploy:deploy) Maven would compute a resolved effective pom. Such a pom would strip out a lot of the stuff that is in a pom at present, e.g. it would probably only consist of /project/groupId /project/artifactId /project/version /project/packaging /project/dependencies/* and maybe /project/build/extensions/* (but it gets tricky deciding exactly how much to prune out) Such a pom would be capturing the dependency tree of the built artifact after inheritance from the parent, and any active profiles, etc had been applied. [Let's ignore the problem of “magic” profiles that alter the dependencies when building on JDK1.5 vs JDK1.6 vs JDK1.7 vs JDK1.8] *IF* we did something like that *THEN* it could be valid to try what you want to do (Note: I say ‘could be’ not ‘would be’) This is because anyone depending on the artifact would have a consistent classpath. Now for packagingpom/packaging projects we would have to deploy the raw pom to the repository so that inheritance would continue to work
AW: Independent module release strategies
Well as soon as a major release is out the door, in that branch only bugfixes are added that have agreat effect on the application. So I am sure that mostly 2-3 Modules change. I know that in general updating a client with about 20mb doesn't sound that bad in our days, but imagine the load on the servers if hundreds of clients all over the worls start updating simultaneously. And there are quite some uses with analog modems ... updating them every time would mean a downtime of hours for them (I was told). So the company tries to keep the number of updated modules to a minimum. Chris Von: Wayne Fay [wayne...@gmail.com] Gesendet: Dienstag, 16. Oktober 2012 01:26 An: Maven Users List Betreff: Re: Independent module release strategies The project consists of about 50 Maven artifacts. A lot of people are using this project all over the world. The client is distributed by some web-start similar solution. The problem is whenever a bugfix-release is done, we don’t want to release all modules in a new version because then all of these would have to be downloaded by the clients. Have you performed any analysis of previous changes to give you an indication of the relative stability of one module vs the rest? Is it true that most changes are happening in one or two modules while the rest is stable -- or are changes found in most any module with no obvious pattern? I would expect that suggested approaches for solving your problem may depend a bit on the answers to these questions. Wayne - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Independent module release strategies
-l 1 | grep [maven-release-plugin] then mvn release:prepare release:perform -B else echo no changes, nothing to do fi we actually had a Maven plugin that did the above using forked processes and the SCM shared utils (unfortunately one that was not developed “while travelling between home and the office” and therefore was developed while I was “engaged in the employ of my employers” and hence not one I could open source [got to love the Health and Safety exception in the Irish statute book... my contract stated that anything I developed while “engaged in the employ of ___” belonged to them, and the Irish statute book only mentions “engaged in the employ” in HS where it states that you are *specifically not engaged in the employ of your employers while commuting between home and the office*]) Since I don't have that problem with my current employers, where we are a cloud based PaaS and basically, barring exceptions, always ensure that the head release is either failing to build or safe to push to production (a.k.a. continuous deployment - because if it is failing to build then we fix that damn fast) And since I do not have any free time at present, I am not invested in re-implementing my magic-release plugin... though it would be super handy for lots of people... in part because it identified the dependency chain and evaluated the release roots in the order in which they were required to be evaluated, thereby ensuring that only the minimum required release set was released. Side-note: of course the reason we didn't do the big bang release everything was because some of our customers were on dial-up modems... never mind that some Architect decided that we were going to switch to delivering the software as VM images so that even a 1 byte change ment the customer re-downloading 15GB of VM images :rolleyes: Chris Von: Ron Wheeler [rwhee...@artifact-software.com] Gesendet: Dienstag, 16. Oktober 2012 00:21 An: users@maven.apache.org Betreff: Re: Independent module release strategies Our project was about 50% larger in terms of modules and we did what you are suggesting and got rid of the idea of increasing versions on artifacts that did not change. I am not sure why this is causing you more problems rather than less. We had a master spreadsheet listing all of our modules and their versions. Every time we issued a new release, we went through the list at looked at what was going to change and what was going to carry through as is with the version number that it had. We also looked at third party libraries that we wanted to upgrade at the same time. We fixed up each pom to update the versions of anything that was going to change to a x.x-SNAPSHOT and moved on. It took about an hour to do the whole job since our system was service oriented so there were not a lot of dependencies. Once a module was tested and released we updated the dependencies to the released module. We still ended up with a lot of releases at the very end of the upgrade process but that is partly human nature since deleting a release is a bad thing even if it is one of yours and could only be done by me so it got a lot of visibility if someone made a mistake. It is hard to get everyone confident that their module's specification will not change due to someone else making a mistake in their design which only gets detected late in the process. They solved this by staying at the SNAPSHOT after they had it fully tested and ready for release, if someone who depended on it was not yet done. It was not a big problem and I never took any steps to fix it. We aggregated a lot of library-like dependencies into larger packages that were provided. This gave a dependency on our utilities package that actually was an aggregation of several projects so each war project did not have dozens of dependencies on modules that were shared by most modules. We did this with third party software as well so a lot of really useful Apache libraries were aggregated into 1 jar that all projects depended on. This reduced the number of dependencies in the POM files and made them a lot easier to maintain. By using a lot of Provided jars, we really sped up the builds and reduced the size of the war files from megabytes to kilobytes since they only contained the code that was in the source files rather that a few Kb of code output and megabytes of libraries. I am not sure that this is the best way to tackle the problem but it eliminated the work that we were doing when we changed the version on everything that made up the application. It also got us thinking about our own packages in the same way that we looked at Apache libraries. There was an incentive to think about interfaces and SOA in a more considered way. I hope that this helps. You are on the right track and your project is still pretty small at 50 modules. Ron On 15/10/2012 5:32 PM, christofer.d...@c
Re: Independent module release strategies
On Tue, Oct 16, 2012 at 7:36 PM, Stephen Connolly stephen.alan.conno...@gmail.com wrote: [del] See what happens when your thread reaches critical mass! Kittens die. p.s. Thanks for an in-depth email that will be very useful for the archives. The Maven Way for a release process isn't really well documented anywhere yet, and I fear a few more kittens will die over that debate as well. Most people bump into release:process and blindly follow that path and there a limitations with that option. SCM commit/tags and mvn deploy are just as good. Its really important to understand what is being automated before automating it. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Independent module release strategies
For what its worth: 1) We do not use the Maven release plug-in and follow what Barrie described below - SCM tags, etc. 2) We do not use profiles. 3) By getting the libraries out of our code, we make the wars very small which might help more in this case where a lot of people are downloading bug fixes over slow lines. We generally do not change the provided libraries within a minor update so they are very stable and there is no need to download 20Mb to get 20Kb of code that changed. Things like CXF are huge and are used everywhere in our application so it makes a big difference in the builds if you have to include it in every war file. Ron On 16/10/2012 5:56 AM, Barrie Treloar wrote: On Tue, Oct 16, 2012 at 7:36 PM, Stephen Connolly stephen.alan.conno...@gmail.com wrote: [del] See what happens when your thread reaches critical mass! Kittens die. p.s. Thanks for an in-depth email that will be very useful for the archives. The Maven Way for a release process isn't really well documented anywhere yet, and I fear a few more kittens will die over that debate as well. Most people bump into release:process and blindly follow that path and there a limitations with that option. SCM commit/tags and mvn deploy are just as good. Its really important to understand what is being automated before automating it. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org -- Ron Wheeler President Artifact Software Inc email: rwhee...@artifact-software.com skype: ronaldmwheeler phone: 866-970-2435, ext 102 - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Independent module release strategies
Hi,
I am currently working on finetuning the workflows on a large application that
was migrated from an Ant based build to one based upon Maven.
The build itself is running smoothly but, what I'm currently working on is
getting the release workflow optimized.
The project consists of about 50 Maven artifacts. A lot of people are using
this project all over the world. The client is distributed by some web-start
similar solution.
The problem is whenever a bugfix-release is done, we don't want to release all
modules in a new version because then all of these would have to be downloaded
by the clients.
So we have a project with a lot of modules and a parent pom that configures the
plugins.
Using the regular maven-release-plugin involves a lot of manual adjusting of
version numbers and I would like to eliminate this.
That's why I setup the master pom to have two profiles develop (active by
default) and release activated during a release. In both profiles a lot of
properties are configured to be used for setting the artifact versions.
No comes the part where I was told on the dev-list that I was tempted by the
dark side of the force ;-)
In my master pom, I defined one major dependencyManagement section fixing the
version of each artifact to the versions in the properties. Ok ... so this is
normal and this is not dark side magic. But in order to have the parent
version automatically configured the right way I wanted to have the version of
the parent link configured by these properties too. Ok so maven doesn't allow
this. But it seems that this is not entirely true:
If I configre the version of the artifact I want to use as parent with the same
variable as I am using in the parent definition of the child module. Maven
seems to work fine with that. The only thing that I was not quite satisfied
with, was that the install plugin installed the raw poms into my local repo.
The directory it was installed to contained the correct version so the
resolution must have worked. That's why I thought this was a bug in the deploy
plugin and that's why I filed an issue (which was immediately closed because I
was doing bad magic) http://jira.codehaus.org/browse/MNG-5358
I attached an example project containing an example configuration demonstrating
what I was doing.
What I find particularly strange is that Maven claims not to resolve properties
in project.version and project.parent.version and in 90% of the cases this is
true:
Let me illustrate thsi a little. Assuming I have only two projects ... one
master and one module.
If I define two properties in my master pom: my.cool.master.version and
my.cool.alternate.master.version and set both to the same value of
1.2-SNAPSHOT.
In szenario 1: I hard code the version of the master to 1.2-SNAPSHOT but use
the property to reference the parent from the moule ... when running a build,
maven tries to download
de/mycompany/test/${my.cool.master.version}/mycoolmaster-${my.cool.master.version}.pom
-- Failure, because the property is not resolved
In szenario 2: I use the same variable for defining the masters version. This
time the maven build runs fine and the parent version is correctly resolved.
In szenario 3: I use the first property to define the version of the master and
the second one for referncing the parent from the module ... when running a
build, maven tries to download
de/mycompany/test/${my.cool.alternate.master.version}/mycoolmaster-${my.cool.alternate.master.version}.pom
-- Failure, because the property is not resolved
So to me it looks as if there was some sort of intention behind everything and
not a bug in the system as I was told on the dev-list. To me it looks like one
teeniewienie loophole allowing properies in versions while closing the usage
range so much that possible harm is reduced to it's absolute minimum. So it
seems that my usecase seems to be the onlly one allowed. After all ... this is
a problem users are begging for maven to provide a solution since maven 2.0
(When looking at the forums).
Ok ... and now to finish the loop back to my topic:
If I am doing bad sourcery ... how would I setup one maven build to allow
simple releases of individual modules with individual versions?
Chris
[ C h r i s t o f e r D u t z ]
C-Ware IT-Service
Inhaber
Dipl. Inf. Christofer Dutz
Karlstraße. 104, 64285 Darmstadt
[cid:image001.gif@01CDAB2D.00AD5550]http://www.benchpark.com/788335/kundenzufriedenheit.htm
IT- und Systemhäuserhttp://www.benchpark.com/it_und_systemhaeuser.htm
fon: 0 61 51 / 27315 - 61
fax: 0 61 51 / 27315 - 64
mobil: 0171 / 7 444 2 33
email: christofer.d...@c-ware.demailto:christofer.d...@c-ware.de
http://www.c-ware.dehttp://www.c-ware.de/
UStId-Nr. DE195700962
Re: Independent module release strategies
Our project was about 50% larger in terms of modules and we did what you are suggesting and got rid of the idea of increasing versions on artifacts that did not change. I am not sure why this is causing you more problems rather than less. We had a master spreadsheet listing all of our modules and their versions. Every time we issued a new release, we went through the list at looked at what was going to change and what was going to carry through as is with the version number that it had. We also looked at third party libraries that we wanted to upgrade at the same time. We fixed up each pom to update the versions of anything that was going to change to a x.x-SNAPSHOT and moved on. It took about an hour to do the whole job since our system was service oriented so there were not a lot of dependencies. Once a module was tested and released we updated the dependencies to the released module. We still ended up with a lot of releases at the very end of the upgrade process but that is partly human nature since deleting a release is a bad thing even if it is one of yours and could only be done by me so it got a lot of visibility if someone made a mistake. It is hard to get everyone confident that their module's specification will not change due to someone else making a mistake in their design which only gets detected late in the process. They solved this by staying at the SNAPSHOT after they had it fully tested and ready for release, if someone who depended on it was not yet done. It was not a big problem and I never took any steps to fix it. We aggregated a lot of library-like dependencies into larger packages that were provided. This gave a dependency on our utilities package that actually was an aggregation of several projects so each war project did not have dozens of dependencies on modules that were shared by most modules. We did this with third party software as well so a lot of really useful Apache libraries were aggregated into 1 jar that all projects depended on. This reduced the number of dependencies in the POM files and made them a lot easier to maintain. By using a lot of Provided jars, we really sped up the builds and reduced the size of the war files from megabytes to kilobytes since they only contained the code that was in the source files rather that a few Kb of code output and megabytes of libraries. I am not sure that this is the best way to tackle the problem but it eliminated the work that we were doing when we changed the version on everything that made up the application. It also got us thinking about our own packages in the same way that we looked at Apache libraries. There was an incentive to think about interfaces and SOA in a more considered way. I hope that this helps. You are on the right track and your project is still pretty small at 50 modules. Ron On 15/10/2012 5:32 PM, christofer.d...@c-ware.de wrote: Hi, I am currently working on finetuning the workflows on a large application that was migrated from an Ant based build to one based upon Maven. The build itself is running smoothly but, what I'm currently working on is getting the release workflow optimized. The project consists of about 50 Maven artifacts. A lot of people are using this project all over the world. The client is distributed by some web-start similar solution. The problem is whenever a bugfix-release is done, we don't want to release all modules in a new version because then all of these would have to be downloaded by the clients. So we have a project with a lot of modules and a parent pom that configures the plugins. Using the regular maven-release-plugin involves a lot of manual adjusting of version numbers and I would like to eliminate this. That's why I setup the master pom to have two profiles develop (active by default) and release activated during a release. In both profiles a lot of properties are configured to be used for setting the artifact versions. No comes the part where I was told on the dev-list that I was tempted by the dark side of the force ;-) In my master pom, I defined one major dependencyManagement section fixing the version of each artifact to the versions in the properties. Ok ... so this is normal and this is not dark side magic. But in order to have the parent version automatically configured the right way I wanted to have the version of the parent link configured by these properties too. Ok so maven doesn't allow this. But it seems that this is not entirely true: If I configre the version of the artifact I want to use as parent with the same variable as I am using in the parent definition of the child module. Maven seems to work fine with that. The only thing that I was not quite satisfied with, was that the install plugin installed the raw poms into my local repo. The directory it was installed to contained the correct version so the resolution must have worked. That's why I thought this
Re: Independent module release strategies
The project consists of about 50 Maven artifacts. A lot of people are using this project all over the world. The client is distributed by some web-start similar solution. The problem is whenever a bugfix-release is done, we don’t want to release all modules in a new version because then all of these would have to be downloaded by the clients. Have you performed any analysis of previous changes to give you an indication of the relative stability of one module vs the rest? Is it true that most changes are happening in one or two modules while the rest is stable -- or are changes found in most any module with no obvious pattern? I would expect that suggested approaches for solving your problem may depend a bit on the answers to these questions. Wayne - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Independent module release strategies
On Tue, Oct 16, 2012 at 9:56 AM, Wayne Fay wayne...@gmail.com wrote: I would expect that suggested approaches for solving your problem may depend a bit on the answers to these questions. Which come about because your software has a development lifecycle: Developing Released Maintained End of Life Obviously each stage has different areas that will get changed. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Independent module release strategies
On 15/10/2012 7:39 PM, Barrie Treloar wrote: On Tue, Oct 16, 2012 at 9:56 AM, Wayne Fay wayne...@gmail.com wrote: I would expect that suggested approaches for solving your problem may depend a bit on the answers to these questions. Which come about because your software has a development lifecycle: Developing Released Maintained End of Life Obviously each stage has different areas that will get changed. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org or Developing Released Maintained Maintained Maintained Maintained ... End of Life Developing Released Maintained Maintained and Developed Released And Maintained Maintained ... End of Life I think that a lot of newcomers to the forum are so tied up in the initial development that they are not thinking about what happens when you need to support a package that has a user base for multiple releases while fixing bugs and developing the next release. Maven's ability to identify all of the right dependencies to reliably reproduce a release done months ago is very helpful once you produce something useful that gets into production. Careful management of tags and branches in our SCM is a priority for us. This is perhaps more germane to the discussion on version ranges or profiles which I am pretty convinced are concepts directly related to pure evil or the road to hell, in a real software project. Ron -- Ron Wheeler President Artifact Software Inc email: rwhee...@artifact-software.com skype: ronaldmwheeler phone: 866-970-2435, ext 102 - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
