Re: Reasonable use of profiles?
On 14/12/2010 4:10 PM, Brian Topping wrote: These replies have been incredibly helpful, thanks especially to Ron and Stephen for your investment in them! Very accidentally, I have fallen into the "MV-SOA" pattern. I am using Mule for the services container with the VM connector for speed and ease of iterative development, knowing that I can trade out the connector once the site scales into the datacenter and needs it. At the webapp level, what I've done is focus all command dispatches to a small class that either a) knows how to reach Mule or b) returns some mock objects. I've found developers who are good with both backend architecture and user interfaces are very rare. In "agile" environments, UI folks often are at the front line of working with business stakeholders though, and if they can't see the results of their work without also wiring the backend, they are quickly stuck. In small team situations, stakeholders respond to this by insisting that the architect is smoking something bad and being given an ultimatum to get the UI folks productive again or be replaced. As a response to this, I've found that making the command messages to the SOA core as a separate shared project between these realms as step one. Next, the web build is set up with a default implementation of the dispatcher that returns these command messages with mocked results. Practically, this is managed as a named Spring bean. Things change when a developer includes a single library that has the alternate implementation of the command dispatcher. When this library is loaded, it's alternate definition of the dispatcher overrides the mock dispatcher and is injected into the webapp accordingly. Now, when commands are dispatched, they transparently go to the SOA core instead of returning mocks. This could also be done with JNDI registrations and solves the same problem. This works at an organizational level because UI devs can create mock implementations in their own sandbox, then throw them over the fence to the core developers with very little documentation. A picture speaks a thousand words. All these artifacts are automatically deployed to Nexus by CI with sources after every checkin, so a developer that needs to trace source in the complimentary project need not have privileges on both projects. More restrictive environments are obviously possible. Along with Nexus, my svn repo is also slaved to LDAP, so authorization is easy to manage. One of the last things to note here: Spring or JNDI is not necessary for smaller projects, just use the classpath, whose namespace is naturally the sum of all loaded artifacts. If a build includes a new artifact and that artifact includes additional configuration, use it. Using the same name for the configuration file in all artifacts helps here, when doing a getResources() for a single filename, you'll be given a list of all classpath files that exist across your various jars. We were so lucky to get Spring early by hiring a strong architect to help get us started. I would recommend that managers of new projects look at Spring. It takes a bit of getting used to if you have a history of development with traditional tools but it is really a great approach. JNDI is so simple, I would recommend getting site specific stuff there as quickly as possible. We discovered the real use of JNDI late in the project. We wasted a lot of time and energy developing alternative solutions to simple configuration problems that JNDI and Spring solved rather effortlessly. We read the JNDI documentation and spent a lot of time wondering why we could not find the complex details when, in fact, there weren't any. Ron Brian - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
These replies have been incredibly helpful, thanks especially to Ron and Stephen for your investment in them! Very accidentally, I have fallen into the "MV-SOA" pattern. I am using Mule for the services container with the VM connector for speed and ease of iterative development, knowing that I can trade out the connector once the site scales into the datacenter and needs it. At the webapp level, what I've done is focus all command dispatches to a small class that either a) knows how to reach Mule or b) returns some mock objects. I've found developers who are good with both backend architecture and user interfaces are very rare. In "agile" environments, UI folks often are at the front line of working with business stakeholders though, and if they can't see the results of their work without also wiring the backend, they are quickly stuck. In small team situations, stakeholders respond to this by insisting that the architect is smoking something bad and being given an ultimatum to get the UI folks productive again or be replaced. As a response to this, I've found that making the command messages to the SOA core as a separate shared project between these realms as step one. Next, the web build is set up with a default implementation of the dispatcher that returns these command messages with mocked results. Practically, this is managed as a named Spring bean. Things change when a developer includes a single library that has the alternate implementation of the command dispatcher. When this library is loaded, it's alternate definition of the dispatcher overrides the mock dispatcher and is injected into the webapp accordingly. Now, when commands are dispatched, they transparently go to the SOA core instead of returning mocks. This could also be done with JNDI registrations and solves the same problem. This works at an organizational level because UI devs can create mock implementations in their own sandbox, then throw them over the fence to the core developers with very little documentation. A picture speaks a thousand words. All these artifacts are automatically deployed to Nexus by CI with sources after every checkin, so a developer that needs to trace source in the complimentary project need not have privileges on both projects. More restrictive environments are obviously possible. Along with Nexus, my svn repo is also slaved to LDAP, so authorization is easy to manage. One of the last things to note here: Spring or JNDI is not necessary for smaller projects, just use the classpath, whose namespace is naturally the sum of all loaded artifacts. If a build includes a new artifact and that artifact includes additional configuration, use it. Using the same name for the configuration file in all artifacts helps here, when doing a getResources() for a single filename, you'll be given a list of all classpath files that exist across your various jars. Brian - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
Ok.. Thanks!
FYI all questions I ask are not posed to doubt your answers and
observations. Merely so I'd understand it better.
stephenconnolly wrote:
>
>
> But did you ever ask why it was OK to rebuild the artifact in those
> environments?
>
Actually, yes. The answer to that usually has something to do with building
on the target platform. I've worked on one project where we had a
professional 'build manager'. Yet he worked with unit tests and integration
tests using Teamcity with build-test-fail/succes principles.
stephenconnolly wrote:
>
>
> How do you know that other SCM changes have not crept in? The contents
> of that EAR have different checksums too, so now we have to do a diff
> on the wars and jars in the ear... oh and the jars in the wars are
> also different... so now we have to do a diff of all the .class files
> to show that they are the same and you have code that references
> File.separator and you have optimization turned on in your compiler
> and this build was on windows while that build was on linux ps!
> these artifacts behave differently... the one on windows won't work on
> linux but the one on linux works on windows except when it launches
> the bat file that it writes... eeek!!! if only we had a repeatable
> consistant build that we just passed through all the environments, we
> would have caught this earlier and either turned off the optimization
> in javac, or used System.getProperty("file.separator").
>
I thought that was the whole point of pulling a tag and rebuilding that tag
using profiles on a machine with a contineous build tool such as Hudson or
Teamcity. At any rate, that way we are absolutely sure that other SCM
changes have not crept in, if the dependencies in the local/remote repo are
ok.
Though I do have to admit that I've seen charset problems rather often.
That might have something to do with this kind of building strategy, right?
stephenconnolly wrote:
>
> Correction, you have yet to work on a project where you were aware of
> the work that went into showing why rebuilding was OK, or where a
> quality team understood the risk
>
Fair enough, although I do doubt the amount of work that's being put in on
that account.
--
View this message in context:
http://maven.40175.n5.nabble.com/Reasonable-use-of-profiles-tp3300650p3304419.html
Sent from the Maven - Users mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 14 December 2010 09:45, fhomasp wrote:
>
> Your rant makes sense Stephen and I'm already glad I asked :-)
>
> I honestly have never been in a development environment where rebuilding an
> artifact was a problem.
But did you ever ask why it was OK to rebuild the artifact in those
environments?
Perhaps Joe was working for ABC Ltd, developing software.
ABC Ltd hires some Quality people because customers want a quality
product... they quality people say "Hey you can't keep rebuilding the
artifact for each customer and saying all you did was change the
branding and therefore we don't have to retest"... there are many many
meetings eventually it becomes clear that Joe has not got the time
to refactor the build process so that ABC Ltd can release one artifact
and re-use for each customer, and anyway Joe has some
technical/architectural perspective where he believes that this is not
possible for the type of software that ABC Ltd develops.
The Quality people run off in a sulk back to their offices... after a
while they come back and say "OK, you can rebuild the artifact for
each customer, but only if you add additional testing that shows that
there is no risk in rebuilding this artifact as a result of your build
process and the nature of the changes between builds"
Joe does a small amount of work to help devise the tests, and the
Testing people have more work (but less than if they had to test every
build for every customer)...
Everything is going swimmingly...
Joe has a bright idea... he's decides to leave ABC Ltd to form a
start-up DEF Ltd... It's his idea, he knows the architecture, he sets
up things so that the build has low risk building for different
environments (after all the Quality fight in ABC Ltd didn't really
affect him... he just took on board the lesson that you need to add
testing to show that rebuilding is low risk)...
Joe hires Fred to develop software...
Fred sees Joe rebuilding artifacts all the time... Fred sees the
Quality people Joe hired being ok with this... When Fred moves on,
what will he think? How will he feel about rebuilding artifacts?
Fred is missing the context of Joe's experience... he only has the
context of working for a company where rebuilding was OK (because they
had already validated rebuilding for their context before Fred
started)
> But ok, I see the idea of keeping the build. But
> if everything is the same, except for property files, what's the problem
> with rebuilding?
There may not be any problem... but how can you be sure? Show me the
evidence that for your use case rebuilding is OK... if you have the
evidence then fine... you are in the clear... but in the absence of
evidence that rebuilding is OK then from a quality perspective you
have to assume that it is not ok.
> Ok, the SHA1 and MD5 are indeed different after a rebuild,
> but I never thought that kind of thing to be a source of possible failures.
How do you know that other SCM changes have not crept in? The contents
of that EAR have different checksums too, so now we have to do a diff
on the wars and jars in the ear... oh and the jars in the wars are
also different... so now we have to do a diff of all the .class files
to show that they are the same and you have code that references
File.separator and you have optimization turned on in your compiler
and this build was on windows while that build was on linux ps!
these artifacts behave differently... the one on windows won't work on
linux but the one on linux works on windows except when it launches
the bat file that it writes... eeek!!! if only we had a repeatable
consistant build that we just passed through all the environments, we
would have caught this earlier and either turned off the optimization
in javac, or used System.getProperty("file.separator").
> Although I have to admit that I'm far from a specialst on that matter.
> Perhaps someone or something should 'taint' me ? Though that sounds more
> awkward than intended ^^
>
> I have yet to find a project where rebuilding an artifact is considered a
> problem.
Correction, you have yet to work on a project where you were aware of
the work that went into showing why rebuilding was OK, or where a
quality team understood the risk
> Usually the idea of the testing is a combination of test
> frameworks applied on the codebase, not the release. Hence the idea of
> having unit tests as part of the "success vs. fail" of a build.
> Then a lot of project managers would want me to rebuild the artifact on a
> machine with the same kind of environment as the target machine.
Sounds like you've worked for a lot of Fred's
> Which is
> why some of my clients used Hudson, to be able to do just that and choose
> the properties, with a profile.
>
> Quality should always be a "thing", imho. But at this point I don't really
> see a reason why rebuilding an artifact would break the quality, but of
> course as I said, I'm not a specialist. I've been a pro developer for 3
> years .
I'm at th
Re: Reasonable use of profiles?
Your rant makes sense Stephen and I'm already glad I asked :-) I honestly have never been in a development environment where rebuilding an artifact was a problem. But ok, I see the idea of keeping the build. But if everything is the same, except for property files, what's the problem with rebuilding? Ok, the SHA1 and MD5 are indeed different after a rebuild, but I never thought that kind of thing to be a source of possible failures. Although I have to admit that I'm far from a specialst on that matter. Perhaps someone or something should 'taint' me ? Though that sounds more awkward than intended ^^ I have yet to find a project where rebuilding an artifact is considered a problem. Usually the idea of the testing is a combination of test frameworks applied on the codebase, not the release. Hence the idea of having unit tests as part of the "success vs. fail" of a build. Then a lot of project managers would want me to rebuild the artifact on a machine with the same kind of environment as the target machine. Which is why some of my clients used Hudson, to be able to do just that and choose the properties, with a profile. Quality should always be a "thing", imho. But at this point I don't really see a reason why rebuilding an artifact would break the quality, but of course as I said, I'm not a specialist. I've been a pro developer for 3 years . -- View this message in context: http://maven.40175.n5.nabble.com/Reasonable-use-of-profiles-tp3300650p3304312.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
One other issue with rebuilding the artifacts is that you're going to have problem if you want to do it the Maven way by deploying to a repo. As there can only be one flavor of a specific version of an artifact, you can't rebuild without bumping version. (Yes, you should never ever replace/remove/alter a released artifact.) /Anders On Tue, Dec 14, 2010 at 10:11, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > On 14 December 2010 08:50, fhomasp wrote: > > > > > > > > I didn't mean huge changes for the different platform. The usual changes > > for the specific environments switched with profiles are usually those > > property files you're talking about. > > So the codebase remains the same, but assuming I build my ear using > property > > files for the validation platform, then the property files for the jar > > dependencies in that ear would be automatically customized for the > > validation platform as well. This means that some language properties > > change, also the validation software needs to connect to another server, > > also on the validation platform, so that url and its properties need to > be > > specific as well. And then also the property files for JUnit and for the > > different databases. This way there is almost no human error possible, > the > > artifact is the same for all platforms, but with different config > settings, > > etc. > > > > Or am I missing something? > > OK, I used to do consulting for the pharma sector. The FDA would hang > you for a short-cut if they heard you were rebuilding the artifact and > not doing a full regression test on the modified artifact ;-) > > [OK, so there are ways and means you could make it work and not be > hung by the FDA, but that's a lot of process you'd be putting in place > and a lot of extra validation and a lot of extra consultant hours to > spend... (why did I leave the pharma sector... oh yeah I hate the > paperwork!)] > > You are REBUILDING your artifact for each use case... that is not the > "Maven Way"... > > Just to be clear, if you want to rebuild the artifact for each > environment, then profiles is the solution I would choose... but I > would NEVER want to rebuild the artifact for each environment... > > The "Maven Way" is to find a way of building, packaging and deploying > your application such that you _never_ need to rebuild your artifact > to run it in a different environment... that way you can build the > artifact, test it in your test environment, qualify it for release in > your qualification environment, and deploy it in your customer > environment, and the whole time the SHA1 and MD5 of your artifact is > the exact same, each step of the process. > > If you rebuild at any point, then a strict quality process would > mandate one of two things: > 1. Restart testing back at the start > or > 2. Do a diff of the two artifacts, identify all changes, make a risk > assessment of the impact of each change, and then on the basis of the > assessed risk do strategic testing in the environment you are > deploying to... and also put in place a process of review for the risk > assessments to ensure that they are being performed correctly and that > the additional testing is not missing issues that would have been > caught by just Restarting testing back at the start > > Now if quality is not your "thing" I can see why you might think that > "REBUILDING for each environment" might be "ok"... but I happen to > have been 'tainted' by working in heavily regulated industries and > trained in GMP, GLP, cGCP... in short I have had quality drilled into > me, at what some suspect is a cellular level, and every fiber of my > being screams... never ever rebuild an artifact just so you can deploy > it in a different environment... rebuild to fix bugs... rebuild to add > features rebuild because the artifact will/should be better > > [/rant] > > -Stephen > > > > > > > > > stephenconnolly wrote: > >> > >> On 14 December 2010 08:06, fhomasp > wrote: > >>> > >>> After reading a bit of the debate I wonder a few things. I read "stay > >>> away > >>> from profiles" a lot but I do find them to be very useful. > >>> > >>> So what's the alternative on profiles? Assuming there is a modular > >>> project > >>> with several jars, several wars and several ears. Each of those > >>> artifacts > >>> can be built for a different environment (development, test (1,2,3), > >>> staging, validation,...) > >> > >> > >> Here is your issue. > >> > >> The Maven Way is to build one artifact that works in any environment. > >> You don't go building environment specific artifacts on the Maven Way. > >> > >> You use things like JNDI or properties files added to the classpath to > >> provide the environment customisations > >> > >> In situations like branding, you produce a brand artifact for each of > >> the customer specific brands and you would load that into your > >> application, by e.g. loading from the classpath, or by installing into > >> the
Re: Reasonable use of profiles?
On 14 December 2010 08:50, fhomasp wrote: > > > > I didn't mean huge changes for the different platform. The usual changes > for the specific environments switched with profiles are usually those > property files you're talking about. > So the codebase remains the same, but assuming I build my ear using property > files for the validation platform, then the property files for the jar > dependencies in that ear would be automatically customized for the > validation platform as well. This means that some language properties > change, also the validation software needs to connect to another server, > also on the validation platform, so that url and its properties need to be > specific as well. And then also the property files for JUnit and for the > different databases. This way there is almost no human error possible, the > artifact is the same for all platforms, but with different config settings, > etc. > > Or am I missing something? OK, I used to do consulting for the pharma sector. The FDA would hang you for a short-cut if they heard you were rebuilding the artifact and not doing a full regression test on the modified artifact ;-) [OK, so there are ways and means you could make it work and not be hung by the FDA, but that's a lot of process you'd be putting in place and a lot of extra validation and a lot of extra consultant hours to spend... (why did I leave the pharma sector... oh yeah I hate the paperwork!)] You are REBUILDING your artifact for each use case... that is not the "Maven Way"... Just to be clear, if you want to rebuild the artifact for each environment, then profiles is the solution I would choose... but I would NEVER want to rebuild the artifact for each environment... The "Maven Way" is to find a way of building, packaging and deploying your application such that you _never_ need to rebuild your artifact to run it in a different environment... that way you can build the artifact, test it in your test environment, qualify it for release in your qualification environment, and deploy it in your customer environment, and the whole time the SHA1 and MD5 of your artifact is the exact same, each step of the process. If you rebuild at any point, then a strict quality process would mandate one of two things: 1. Restart testing back at the start or 2. Do a diff of the two artifacts, identify all changes, make a risk assessment of the impact of each change, and then on the basis of the assessed risk do strategic testing in the environment you are deploying to... and also put in place a process of review for the risk assessments to ensure that they are being performed correctly and that the additional testing is not missing issues that would have been caught by just Restarting testing back at the start Now if quality is not your "thing" I can see why you might think that "REBUILDING for each environment" might be "ok"... but I happen to have been 'tainted' by working in heavily regulated industries and trained in GMP, GLP, cGCP... in short I have had quality drilled into me, at what some suspect is a cellular level, and every fiber of my being screams... never ever rebuild an artifact just so you can deploy it in a different environment... rebuild to fix bugs... rebuild to add features rebuild because the artifact will/should be better [/rant] -Stephen > > > > stephenconnolly wrote: >> >> On 14 December 2010 08:06, fhomasp wrote: >>> >>> After reading a bit of the debate I wonder a few things. I read "stay >>> away >>> from profiles" a lot but I do find them to be very useful. >>> >>> So what's the alternative on profiles? Assuming there is a modular >>> project >>> with several jars, several wars and several ears. Each of those >>> artifacts >>> can be built for a different environment (development, test (1,2,3), >>> staging, validation,...) >> >> >> Here is your issue. >> >> The Maven Way is to build one artifact that works in any environment. >> You don't go building environment specific artifacts on the Maven Way. >> >> You use things like JNDI or properties files added to the classpath to >> provide the environment customisations >> >> In situations like branding, you produce a brand artifact for each of >> the customer specific brands and you would load that into your >> application, by e.g. loading from the classpath, or by installing into >> the deployed application. >> >> If you have an existing application that is not designed this way, >> then I can see that you might find it hard to avoid profiles but >> you will have a better application if it is designed for the kind of >> pluggable customizations I describe. >> >> The Maven Way is about best practices and one of the best >> practices there is is ensuring that you only build an artifact once >> and re-use that tested artifact... it reduces the scope of testing >> (i.e. you only have to test the JNDI names exist, or you only have to >> test that the branding is correctly applied, rather than have to
Re: Reasonable use of profiles?
I didn't mean huge changes for the different platform. The usual changes for the specific environments switched with profiles are usually those property files you're talking about. So the codebase remains the same, but assuming I build my ear using property files for the validation platform, then the property files for the jar dependencies in that ear would be automatically customized for the validation platform as well. This means that some language properties change, also the validation software needs to connect to another server, also on the validation platform, so that url and its properties need to be specific as well. And then also the property files for JUnit and for the different databases. This way there is almost no human error possible, the artifact is the same for all platforms, but with different config settings, etc. Or am I missing something? stephenconnolly wrote: > > On 14 December 2010 08:06, fhomasp wrote: >> >> After reading a bit of the debate I wonder a few things. I read "stay >> away >> from profiles" a lot but I do find them to be very useful. >> >> So what's the alternative on profiles? Assuming there is a modular >> project >> with several jars, several wars and several ears. Each of those >> artifacts >> can be built for a different environment (development, test (1,2,3), >> staging, validation,...) > > > Here is your issue. > > The Maven Way is to build one artifact that works in any environment. > You don't go building environment specific artifacts on the Maven Way. > > You use things like JNDI or properties files added to the classpath to > provide the environment customisations > > In situations like branding, you produce a brand artifact for each of > the customer specific brands and you would load that into your > application, by e.g. loading from the classpath, or by installing into > the deployed application. > > If you have an existing application that is not designed this way, > then I can see that you might find it hard to avoid profiles but > you will have a better application if it is designed for the kind of > pluggable customizations I describe. > > The Maven Way is about best practices and one of the best > practices there is is ensuring that you only build an artifact once > and re-use that tested artifact... it reduces the scope of testing > (i.e. you only have to test the JNDI names exist, or you only have to > test that the branding is correctly applied, rather than have to > retest the entire application because you have rebuilt it with the > alternate profile. > > -Stephen > >> >> Then an ear/war can be deployed using Maven to those different >> environments, >> be it from a local machine or Hudson or some other contineous integration >> tool. >> >> How would one automate such situations without profiles and without a >> huge >> amount of redundant maven xml? >> >> >> -- >> View this message in context: >> http://maven.40175.n5.nabble.com/Reasonable-use-of-profiles-tp3300650p3304188.html >> Sent from the Maven - Users mailing list archive at Nabble.com. >> >> - >> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org >> For additional commands, e-mail: users-h...@maven.apache.org >> >> > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > > > -- View this message in context: http://maven.40175.n5.nabble.com/Reasonable-use-of-profiles-tp3300650p3304241.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 14/12/2010 3:06 AM, fhomasp wrote: After reading a bit of the debate I wonder a few things. I read "stay away from profiles" a lot but I do find them to be very useful. So what's the alternative on profiles? Assuming there is a modular project with several jars, several wars and several ears. Each of those artifacts can be built for a different environment (development, test (1,2,3), staging, validation,...) Separate operations (deployment configurations) from development (code, JAR and WARs). Use JNDI or other configuration methods to set up your environments rather than code. We use a project to hold our configurations but there is no Maven build required since we use JNDI in a single file for each of the environments that we support and have not achieved the level of complexity where a build would be helpful. When we get to automating their construction, I wonder if Ant with XSLT might be a better tool for assembling our environments than Maven. The other 70+ projects are environment neutral and each one builds one or 2 artifacts (JARS or WARs) or contains a POM with no code. We are building a portal that runs on Tomcat with Web services, portals and standalone batch jobs. We support 2 client versions with different functionality and numerous test and development environments. We are maintaining production versions at the same time as developing one or more new versions simultaneously. We have a small team that has ranged from 3 to 5 people. I have never used profiles but I see a lot of people get into really complex situations with lots of frustration over profiles and I have a sense that they are really easy to misuse and lead people away from simple sensible solutions to their problems. I also see conversations from people whose opinions I respect saying that they can be useful. I have concluded that they are not a good thing to start with but may be helpful later, once the Maven environment is up and running, to optimize some functions. Ron Then an ear/war can be deployed using Maven to those different environments, be it from a local machine or Hudson or some other contineous integration tool. How would one automate such situations without profiles and without a huge amount of redundant maven xml? - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 14 December 2010 08:06, fhomasp wrote: > > After reading a bit of the debate I wonder a few things. I read "stay away > from profiles" a lot but I do find them to be very useful. > > So what's the alternative on profiles? Assuming there is a modular project > with several jars, several wars and several ears. Each of those artifacts > can be built for a different environment (development, test (1,2,3), > staging, validation,...) Here is your issue. The Maven Way is to build one artifact that works in any environment. You don't go building environment specific artifacts on the Maven Way. You use things like JNDI or properties files added to the classpath to provide the environment customisations In situations like branding, you produce a brand artifact for each of the customer specific brands and you would load that into your application, by e.g. loading from the classpath, or by installing into the deployed application. If you have an existing application that is not designed this way, then I can see that you might find it hard to avoid profiles but you will have a better application if it is designed for the kind of pluggable customizations I describe. The Maven Way is about best practices and one of the best practices there is is ensuring that you only build an artifact once and re-use that tested artifact... it reduces the scope of testing (i.e. you only have to test the JNDI names exist, or you only have to test that the branding is correctly applied, rather than have to retest the entire application because you have rebuilt it with the alternate profile. -Stephen > > Then an ear/war can be deployed using Maven to those different environments, > be it from a local machine or Hudson or some other contineous integration > tool. > > How would one automate such situations without profiles and without a huge > amount of redundant maven xml? > > > -- > View this message in context: > http://maven.40175.n5.nabble.com/Reasonable-use-of-profiles-tp3300650p3304188.html > Sent from the Maven - Users mailing list archive at Nabble.com. > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
After reading a bit of the debate I wonder a few things. I read "stay away from profiles" a lot but I do find them to be very useful. So what's the alternative on profiles? Assuming there is a modular project with several jars, several wars and several ears. Each of those artifacts can be built for a different environment (development, test (1,2,3), staging, validation,...) Then an ear/war can be deployed using Maven to those different environments, be it from a local machine or Hudson or some other contineous integration tool. How would one automate such situations without profiles and without a huge amount of redundant maven xml? -- View this message in context: http://maven.40175.n5.nabble.com/Reasonable-use-of-profiles-tp3300650p3304188.html Sent from the Maven - Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 11/12/2010 3:15 PM, Brian Topping wrote: On Dec 11, 2010, at 3:59 AM, Stephen Connolly wrote: My issue with using orofiles for adding modules is that you can end up with version numbers out of sync of course that's why I created versions-maven-plugin, but that is just a band-aid. Is this because the release plugin doesn't see some modules when doing a release on a module subset or am I missing something else? Initially, I didn't imagine someone doing a release on a profile that contained less than a full set of modules, but see the error in that logic now (it would appear to be a very clean way to release a subset of functionality), and as I put this together with the functionality in the versions-maven-plugin, can better imagine the kinds of builds you are working with. Using profiles to pull in modules can just as easily result in broken builds... what I ended up doing is using profiles to turn off the system specific parts of a build when on a system without those system specific components (e.g. missing flashplayer, rpmbuild), but also to fail a release if the system specific components are missing. So are you also of the camp that having multiple "developer POMs" is the best practical solution? It seems like these approaches are essential for projects that have grown to the size of the death star, and what I am asking myself now is whether there is a breakeven to starting a project with the "wrong" patterns (saving time during early agile development due to things like IDE integration) versus avoiding profiles wherever possible from the start and skipping the pit stop to migrate everything down the road. In other words, are these "good problems to have" (the company and it's IP got out of the incubator and now this as a problem) versus solving them too early being counterproductive to getting smaller projects off the ground. This is a very constructive way to look at the problem. Having gone through this process, we wish that we had done it better from the start but we never got mixed up in profiles. Our main lessons learned are: 1) get your repo up early. 2) RTFM and RTFB 3) move to SOA as early as you can. MVC is great and MV-SOA is better. Look at every process that you build to see if it can be better designed as a service using other services. 4) break the project into libraries. They are likely to be more stable and easier to change. We are still looking for opportunities to extract out common functions that are shared 5) stay away from exotic or custom plug-ins and profiles until you get the basics working. Optimization at the start is bound to be hard and likely to lock you into bad practices before you know what you are doing. Ron Thanks, Brian - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On Dec 11, 2010, at 3:59 AM, Stephen Connolly wrote: > My issue with using orofiles for adding modules is that you can end up with > version numbers out of sync of course that's why I created > versions-maven-plugin, but that is just a band-aid. Is this because the release plugin doesn't see some modules when doing a release on a module subset or am I missing something else? Initially, I didn't imagine someone doing a release on a profile that contained less than a full set of modules, but see the error in that logic now (it would appear to be a very clean way to release a subset of functionality), and as I put this together with the functionality in the versions-maven-plugin, can better imagine the kinds of builds you are working with. > Using profiles to pull in modules can just as easily result in broken > builds... what I ended up doing is using profiles to turn off the system > specific parts of a build when on a system without those system specific > components (e.g. missing flashplayer, rpmbuild), but also to fail a release > if the system specific components are missing. So are you also of the camp that having multiple "developer POMs" is the best practical solution? It seems like these approaches are essential for projects that have grown to the size of the death star, and what I am asking myself now is whether there is a breakeven to starting a project with the "wrong" patterns (saving time during early agile development due to things like IDE integration) versus avoiding profiles wherever possible from the start and skipping the pit stop to migrate everything down the road. In other words, are these "good problems to have" (the company and it's IP got out of the incubator and now this as a problem) versus solving them too early being counterproductive to getting smaller projects off the ground. Thanks, Brian - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 11 December 2010 06:26, Brian Topping wrote: > > On Dec 10, 2010, at 10:03 PM, Ron Wheeler wrote: > > > On 10/12/2010 9:19 PM, Bryan Loofbourrow wrote: > What is the semantic difference between multiple POMs and a single POM > containing the different module lists in profiles? It seems like the former > is harder to support because a change outside the module sets requires > changing every POM.<< > >> Well, one difference is a separation of developer concerns from the > concerns of the consumers of your artifacts. This is especially applicable > if the pom containing the profiles is the parent pom of your project, that > actually gets deployed to a repository and consumed by those who consume > your software. > >> > >> More subtly, I'd argue that the profiles are "magic beans," things for > which you have to introspect your source code to know what's going on, and > not really knowing, without extensive examination, whether these profiles > have any effect on any of the projects in the child tree. In contrast, a > developer-facing pom is a distinct file whose purpose can be made quite > clear at the file system level, and whose purpose is unambiguously contained > entirely within the file itself. > >> > >> Descending down to the final level of abstract mysticism, I'd also say > that the profiles represent a "last resort" in the context of "The Maven > Way," a thing you do not use unless you must. Yes, we use them -- for things > like telling the build to execute integration tests that depend on the > presence of an active database whose location you have defined in a > settings.xml file. But for a trivial purpose like this, they are overkill. > >> > > +1. > > I see where you are coming from here and can see how that would be > attractive in one perspective. > > For the benefit of the original poster (I assume you guys know this > already), module elements are purely metadata for the reactor in the form of > a closure of all the projects that should be built, and don't have anything > to do with module dependencies once installed/deployed. As long as > dependency elements are not inside profiles, the dependency tree is static, > and the process of installing to the local repository "flattens" the tree, > converting the hierarchical file directory tree to a directed graph that > rarely has any resemblance whatsoever to the original project directory > tree. [1] details this with helpful pictures. > > My issue with using orofiles for adding modules is that you can end up with version numbers out of sync of course that's why I created versions-maven-plugin, but that is just a band-aid. Using profiles to pull in modules can just as easily result in broken builds... what I ended up doing is using profiles to turn off the system specific parts of a build when on a system without those system specific components (e.g. missing flashplayer, rpmbuild), but also to fail a release if the system specific components are missing. > Once that's done, whether or not profiles are used to expand or contract a > module selection, what starts as an empty repository is only going to have a > larger or smaller set of artifacts in it in either case (via multiple POMs > or profile-based selection of modules to be built). > > One of the things that I would feel is hard to support with multiple POMs > (at least in IDEA) is how to deal with making the selection of what > artifacts to build. It's easy at the top level, I just import one POM > instead of the other. But what about when there are multiple child POMs > that need to make selections, or there are are permutations? In IDEA, four > distinct profile names can provide up to fifteen compositional permutations > with one POM per project. Getting fifteen builds with separate POMs implies > I have fifteen POMs at the root, and some diminishing number > 1 at most > child levels. I realize this is a degenerate case, but everything in > between is just a matter of degrees. By overloading the profile ID > namespace, I can do this with one POM per project in all cases. > > The other thing that bothers me about multiple POMs is keeping them in > sync. In IDEA, if an import is not on the classpath, an inspector speedmenu > will ask me if I want to add a dependency to my POM. But if IDEA only knows > about one POM, there are others that I have to update manually, and I might > forget about them or not know about them. This leads to broken builds that > might not be hard to fix, but I'll venture that it's more work to keep all > the permutations happy that way than to learn the profiles, keep their use > limited, and maintain a single set of POMs. > > It seems like there is complexity either way. I agree that profiles can be > a bit like C++ multiple inheritance and it's easy to abuse them and end up > with a train wreck, but I've also seen some impossibly elegant designs with > multiple inheritance as well, and the same is true for this kind of use of > profile
Re: Reasonable use of profiles?
On Dec 10, 2010, at 10:03 PM, Ron Wheeler wrote: > On 10/12/2010 9:19 PM, Bryan Loofbourrow wrote: What is the semantic difference between multiple POMs and a single POM containing the different module lists in profiles? It seems like the former is harder to support because a change outside the module sets requires changing every POM.<< >> Well, one difference is a separation of developer concerns from the concerns >> of the consumers of your artifacts. This is especially applicable if the pom >> containing the profiles is the parent pom of your project, that actually >> gets deployed to a repository and consumed by those who consume your >> software. >> >> More subtly, I'd argue that the profiles are "magic beans," things for which >> you have to introspect your source code to know what's going on, and not >> really knowing, without extensive examination, whether these profiles have >> any effect on any of the projects in the child tree. In contrast, a >> developer-facing pom is a distinct file whose purpose can be made quite >> clear at the file system level, and whose purpose is unambiguously contained >> entirely within the file itself. >> >> Descending down to the final level of abstract mysticism, I'd also say that >> the profiles represent a "last resort" in the context of "The Maven Way," a >> thing you do not use unless you must. Yes, we use them -- for things like >> telling the build to execute integration tests that depend on the presence >> of an active database whose location you have defined in a settings.xml >> file. But for a trivial purpose like this, they are overkill. >> > +1. I see where you are coming from here and can see how that would be attractive in one perspective. For the benefit of the original poster (I assume you guys know this already), module elements are purely metadata for the reactor in the form of a closure of all the projects that should be built, and don't have anything to do with module dependencies once installed/deployed. As long as dependency elements are not inside profiles, the dependency tree is static, and the process of installing to the local repository "flattens" the tree, converting the hierarchical file directory tree to a directed graph that rarely has any resemblance whatsoever to the original project directory tree. [1] details this with helpful pictures. Once that's done, whether or not profiles are used to expand or contract a module selection, what starts as an empty repository is only going to have a larger or smaller set of artifacts in it in either case (via multiple POMs or profile-based selection of modules to be built). One of the things that I would feel is hard to support with multiple POMs (at least in IDEA) is how to deal with making the selection of what artifacts to build. It's easy at the top level, I just import one POM instead of the other. But what about when there are multiple child POMs that need to make selections, or there are are permutations? In IDEA, four distinct profile names can provide up to fifteen compositional permutations with one POM per project. Getting fifteen builds with separate POMs implies I have fifteen POMs at the root, and some diminishing number > 1 at most child levels. I realize this is a degenerate case, but everything in between is just a matter of degrees. By overloading the profile ID namespace, I can do this with one POM per project in all cases. The other thing that bothers me about multiple POMs is keeping them in sync. In IDEA, if an import is not on the classpath, an inspector speedmenu will ask me if I want to add a dependency to my POM. But if IDEA only knows about one POM, there are others that I have to update manually, and I might forget about them or not know about them. This leads to broken builds that might not be hard to fix, but I'll venture that it's more work to keep all the permutations happy that way than to learn the profiles, keep their use limited, and maintain a single set of POMs. It seems like there is complexity either way. I agree that profiles can be a bit like C++ multiple inheritance and it's easy to abuse them and end up with a train wreck, but I've also seen some impossibly elegant designs with multiple inheritance as well, and the same is true for this kind of use of profiles. In the end, either technique has pros and cons. Makes for a good horse race... :-) Brian [1] http://www.sonatype.com/books/mvnref-book/reference/pom-relationships-sect-pom-best-practice.html#pom-relationships-sect-multi-vs-inherit - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 10/12/2010 9:19 PM, Bryan Loofbourrow wrote: What is the semantic difference between multiple POMs and a single POM containing the different module lists in profiles? It seems like the former is harder to support because a change outside the module sets requires changing every POM.<< Well, one difference is a separation of developer concerns from the concerns of the consumers of your artifacts. This is especially applicable if the pom containing the profiles is the parent pom of your project, that actually gets deployed to a repository and consumed by those who consume your software. More subtly, I'd argue that the profiles are "magic beans," things for which you have to introspect your source code to know what's going on, and not really knowing, without extensive examination, whether these profiles have any effect on any of the projects in the child tree. In contrast, a developer-facing pom is a distinct file whose purpose can be made quite clear at the file system level, and whose purpose is unambiguously contained entirely within the file itself. Descending down to the final level of abstract mysticism, I'd also say that the profiles represent a "last resort" in the context of "The Maven Way," a thing you do not use unless you must. Yes, we use them -- for things like telling the build to execute integration tests that depend on the presence of an active database whose location you have defined in a settings.xml file. But for a trivial purpose like this, they are overkill. +1. Ron -- Bryan -Original Message- From: Brian Topping [mailto:topp...@codehaus.org] Sent: Friday, December 10, 2010 5:37 PM To: Maven Users List Subject: Re: Reasonable use of profiles? On Dec 10, 2010, at 8:08 PM, Bryan Loofbourrow wrote: You don't need separate projects for this. You just need a bunch of developer-facing pom files with different lists. They can certainly live in the same directory. This is something we definitely take advantage of, for producing interesting developer build sets. What is the semantic difference between multiple POMs and a single POM containing the different module lists in profiles? It seems like the former is harder to support because a change outside the module sets requires changing every POM. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On 10/12/2010 1:24 PM, KARR, DAVID (ATTSI) wrote: -Original Message- From: Kalle Korhonen [mailto:kalle.o.korho...@gmail.com] Sent: Friday, December 10, 2010 9:58 AM To: Maven Users List Subject: Re: Reasonable use of profiles? On Fri, Dec 10, 2010 at 9:20 AM, KARR, DAVID (ATTSI) wrote: Another desire I have is to allow for developers not even building most of the modules, and just letting the "ear" project pull snapshot artifacts from the Nexus repo (built by the release team or continuous integration). I could do this with multiple "build" projects, including different sets of "module" elements. That seems messy, however. I wish I had different options for setting that up, perhaps in a profile, but I don't see how that could work. Overall, I'd say this sounds better than trying to (mis-)use profiles. Nothing you said indicates to me that the EARs are functionally equivalent, and therefore I'd create different modules for different EARs. One of the Maven ground rules is one artifact per module. Typically you deviate from that plan only if you need to build different versions of the same module (packaged differently, for different OSes/JVMs etc.) and often you use profiles for the desired effect. I'd further say using profiles should never be your first option. Reserve profiles for the time you really need them. To clarify the use case I'm talking about here, let's say the EAR requires 20 principal artifacts to construct the EAR (not to mention the numerous third-party dependencies). These have a 1-1 correspondence to modules. Some developers will work on one set of those modules (one or more), and other developers will work on other sets. No developers will work on all of the modules, but all developers need all of the produced artifacts in order to produce an EAR. So, if I only do work on one module, I could produce a build project that just has two modules, the module I work on, and the "EAR" module. I would get the other artifacts from the Nexus repo. Other developers would have to use a different build project, with a different set of modules, including the "EAR" module. Other developers would have a build project with a still different set of modules, including the "EAR" module. > From what I can tell, my only option is to create these numerous very similar "build" projects, all with different permutations of the available modules. Release management would never use these, these are just for developers. I'm simply looking for a better way to manage this. I envision if I somehow had a "list" property, consisting of module names, defined in "settings.xml", a developer could easily add or subtract from this list, and a single "build" project would use that property. I just don't know how to get something like this to work with Maven. Since you are going to re-work the build, I'd mercilessly refactor it into a modular build now. Since Maven is so good at versioning things, your build doesn't have to be monolithic. Snapshots are ok, but it's far better if you can identify common, stable areas of the codebase and simply release them separately. Then your top-level builds are mostly assembling things together rather than compiling/building them. Personally, I'd put my efforts on making the build modular, automating version migration and doing more continuous integration& testing rather than trying to force Maven the same logic as your Ant build. Depending on the size of the project and your team, it would likely be beneficial to pay somebody who really knows Maven to assist you in the migration at this point (if you can get it approved, I know how it is). It would save you from a lot of grief later. What I have the leeway to produce is a structure that is compatible with Maven, but will still work with the Ant build. I can't bring down the entire development process while I single-handedly construct the Maven infrastructure. It was hard enough to get permission to even examine Maven. Each of our individual projects will produce a single artifact, whether from Ant or Maven, and the responsibility for assembling the EAR is now separated into a separate "build" project. That will allow me to integrate Maven with less pain after the slightly transformed Ant build is working. I am still trying to figure out how your development process is so abnormal that the normal Maven way will not work out-of-the-box. We have over 70 projects that build our portal application that includes standalone batch jobs, web services, servlets and portlets. We use no profiles and have very simple POM files. I can not see any reason in what I get from reading your description of your environment that makes me think you are doing anything so far outside normal development project
RE: Reasonable use of profiles?
>> What is the semantic difference between multiple POMs and a single POM >> containing the different module lists in profiles? It seems like the former >> is harder to support because a change outside the module sets requires >> changing every POM. << Well, one difference is a separation of developer concerns from the concerns of the consumers of your artifacts. This is especially applicable if the pom containing the profiles is the parent pom of your project, that actually gets deployed to a repository and consumed by those who consume your software. More subtly, I'd argue that the profiles are "magic beans," things for which you have to introspect your source code to know what's going on, and not really knowing, without extensive examination, whether these profiles have any effect on any of the projects in the child tree. In contrast, a developer-facing pom is a distinct file whose purpose can be made quite clear at the file system level, and whose purpose is unambiguously contained entirely within the file itself. Descending down to the final level of abstract mysticism, I'd also say that the profiles represent a "last resort" in the context of "The Maven Way," a thing you do not use unless you must. Yes, we use them -- for things like telling the build to execute integration tests that depend on the presence of an active database whose location you have defined in a settings.xml file. But for a trivial purpose like this, they are overkill. -- Bryan -Original Message- From: Brian Topping [mailto:topp...@codehaus.org] Sent: Friday, December 10, 2010 5:37 PM To: Maven Users List Subject: Re: Reasonable use of profiles? On Dec 10, 2010, at 8:08 PM, Bryan Loofbourrow wrote: > You don't need separate projects for this. You just need a bunch of > developer-facing pom files with different lists. They can certainly > live in the same directory. This is something we definitely take advantage > of, for producing interesting developer build sets. What is the semantic difference between multiple POMs and a single POM containing the different module lists in profiles? It seems like the former is harder to support because a change outside the module sets requires changing every POM. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On Dec 10, 2010, at 8:08 PM, Bryan Loofbourrow wrote: > You don't need separate projects for this. You just need a bunch of > developer-facing pom files with different lists. They can certainly > live in the same directory. This is something we definitely take advantage > of, for producing interesting developer build sets. What is the semantic difference between multiple POMs and a single POM containing the different module lists in profiles? It seems like the former is harder to support because a change outside the module sets requires changing every POM. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
RE: Reasonable use of profiles?
> > Another desire I have is to allow for developers not even building > most > > of the modules, and just letting the "ear" project pull snapshot > > artifacts from the Nexus repo (built by the release team or > continuous > > integration). I could do this with multiple "build" projects, > including > > different sets of "module" elements. That seems messy, however. You don't need separate projects for this. You just need a bunch of developer-facing pom files with different lists. They can certainly live in the same directory. This is something we definitely take advantage of, for producing interesting developer build sets. This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
Beyond the basics and the core principles, it's difficult to write down the exact instructions for a large scale deployment and use of Maven because the situations are vastly different. Even if one would document the best practices for some particular situations, the issue for the novice is identifying the right case for your situation. Between the company's SCM of choice, preferred branching patterns (release / feature branches), development velocity, maturity of the codebase, the type of software (libraries, service, pre-packaged apps) and programming languages used (plain Java, native libs w/ OS-specific languages, scripting), development methodology etc. there are too many variations to be able to confidently arrive at an optimal solution just by reading about it. For somebody with experience though, it's typically simple to re-organize even larger builds to follow best practices in a matter of few days. However, you often end up spending much much longer in real time, because first, you want to go at it alone and second, you always discount the level of resistance to change. Kalle On Fri, Dec 10, 2010 at 2:19 PM, Peter Schuller wrote: >> I very much second Kalle here. >> Stay away from profiles as much as possible - most often they are used in >> the wrong way. Also, if you're migrating a large system I would very much >> recommend that you get someone with good Maven knowledge to help you get >> things right. I very often come to projects where some senior developer has >> created a Maven build setup which is just completely wrong. Having to >> refactor that costs possibly two-three times more than doing it right from >> the beginning. It often also upsets the developers as they often have to >> change the way they work. > > So on that topic, is there good material that covers doing it right? > I've read the sonatype maven book and while it's good as far as it > goes, it doesn't really cover the intended workflows in various > situations and the intent of various abstractions. Googling is kind of > problematic because there is a lot of cargo cult information floating > around. > > Is there a book of other material that truly goes through the intended > way to deploy Maven in an organization, including not just stuff like > "here's how you install nexus", but things like release management, > the relationship between maven release management and VCS tagging, > what to do about multiple branches of development, etc, etc. > > Suppose you want to build an eco-system of tens or hundreds of > projects in an organization and deploy/build them scalably (in terms > of administration/build costs) using maven in a clean and maintainable > fashion without hacks or non-idiomatic mistakes. What material does > one preferably read? > > -- > / Peter Schuller > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
> I very much second Kalle here. > Stay away from profiles as much as possible - most often they are used in > the wrong way. Also, if you're migrating a large system I would very much > recommend that you get someone with good Maven knowledge to help you get > things right. I very often come to projects where some senior developer has > created a Maven build setup which is just completely wrong. Having to > refactor that costs possibly two-three times more than doing it right from > the beginning. It often also upsets the developers as they often have to > change the way they work. So on that topic, is there good material that covers doing it right? I've read the sonatype maven book and while it's good as far as it goes, it doesn't really cover the intended workflows in various situations and the intent of various abstractions. Googling is kind of problematic because there is a lot of cargo cult information floating around. Is there a book of other material that truly goes through the intended way to deploy Maven in an organization, including not just stuff like "here's how you install nexus", but things like release management, the relationship between maven release management and VCS tagging, what to do about multiple branches of development, etc, etc. Suppose you want to build an eco-system of tens or hundreds of projects in an organization and deploy/build them scalably (in terms of administration/build costs) using maven in a clean and maintainable fashion without hacks or non-idiomatic mistakes. What material does one preferably read? -- / Peter Schuller - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
I very much second Kalle here. Stay away from profiles as much as possible - most often they are used in the wrong way. Also, if you're migrating a large system I would very much recommend that you get someone with good Maven knowledge to help you get things right. I very often come to projects where some senior developer has created a Maven build setup which is just completely wrong. Having to refactor that costs possibly two-three times more than doing it right from the beginning. It often also upsets the developers as they often have to change the way they work. /Anders On Fri, Dec 10, 2010 at 18:57, Kalle Korhonen wrote: > On Fri, Dec 10, 2010 at 9:20 AM, KARR, DAVID (ATTSI) > wrote: > > Another desire I have is to allow for developers not even building most > > of the modules, and just letting the "ear" project pull snapshot > > artifacts from the Nexus repo (built by the release team or continuous > > integration). I could do this with multiple "build" projects, including > > different sets of "module" elements. That seems messy, however. I wish > > I had different options for setting that up, perhaps in a profile, but I > > don't see how that could work. > > Overall, I'd say this sounds better than trying to (mis-)use profiles. > Nothing you said indicates to me that the EARs are functionally > equivalent, and therefore I'd create different modules for different > EARs. One of the Maven ground rules is one artifact per module. > Typically you deviate from that plan only if you need to build > different versions of the same module (packaged differently, for > different OSes/JVMs etc.) and often you use profiles for the desired > effect. I'd further say using profiles should never be your first > option. Reserve profiles for the time you really need them. > > Since you are going to re-work the build, I'd mercilessly refactor it > into a modular build now. Since Maven is so good at versioning things, > your build doesn't have to be monolithic. Snapshots are ok, but it's > far better if you can identify common, stable areas of the codebase > and simply release them separately. Then your top-level builds are > mostly assembling things together rather than compiling/building them. > Personally, I'd put my efforts on making the build modular, automating > version migration and doing more continuous integration & testing > rather than trying to force Maven the same logic as your Ant build. > Depending on the size of the project and your team, it would likely be > beneficial to pay somebody who really knows Maven to assist you in the > migration at this point (if you can get it approved, I know how it > is). It would save you from a lot of grief later. > > Kalle > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > >
Re: Reasonable use of profiles?
On Dec 10, 2010, at 1:30 PM, KARR, DAVID (ATTSI) wrote: > Interesting. This could go into the "build" project POM, but if a > developer needed a custom list of modules, they could define that in > their settings.xml. I guess that could work for what I'm looking for. It's not possible to put the element in settings.xml, if that's what you are thinking. But given that you can specify multiple profiles for a build and individual profile IDs can be used in multiple projects, you might consider making a different profile ID for each root component that made sense to be selectable. IntelliJ lets developers persistently select profiles for a project, so it would not be hard to do an a la carte like this without using settings.xml. I am in the midst of getting Nexus and SVN to play nice from LDAP, so developer access can be granted or revoked from a UI as a part of their master access profile. Nexus privileges can also be set up for "everything but source jars", so CI can publish source to Nexus but not everyone can get it. Nexus isn't hard to set up like this, but getting SVN to work through Atlassian Crowd for fine-grained repository access still needs a little work it seems. Definitely experiment with this dummy projects before jumping in though, SVN authz is not a trivial setup to deploy, get it working and confirm that there aren't leaks. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
RE: Reasonable use of profiles?
An area we've been successful with in using profiles is to distinguish between local builds and CI build to toggle Fat v Skinny WARs Curt Yanko | Continuous Integration Services | UnitedHealth Group IT Making IT Happen, one build at a time, 600 times a day > -Original Message- > From: KARR, DAVID (ATTSI) [mailto:dk0...@att.com] > Sent: Friday, December 10, 2010 12:20 PM > To: Maven Users List > Subject: Reasonable use of profiles? > > I've heard talk about the debatable value of profiles, and I > wanted to describe one use I might have for profiles, > although I'm still in the initial stages of converting a > large Ant environment to Maven, so things may change. I'd > appreciate any comments. > > I think one basic reason that profiles become useful is that > we can't control every aspect of our environment. > > This "environment" as I describe it, consists of a large > enterprise web application, although there are actually 2-3 > different EARS that are built for different but related > purposes. The applications are composed of a bunch of > modules that are divided on functional and team boundaries > (front-end Java, back-end Java, web modules, middleware, et cetera). > > When developers build the present system, they have to check > out all of the modules that contribute to the EAR, and they > have to check them out with their default name, because the > build has hard-coded references to the directory names. > > The work for each release, about every month, is done on a > branch named for that release. If the work for a release > includes changes in a particular module, then that module is > branched so that developers can do the work required for it. > Developers often have to do some work on more than one > branch, in the same module. > > So, in order to do a complete Ant build, you have to have all > the required module names checked out with their default > names. Some modules can do unit and even "integration" unit > testing without building the EAR (back-end and middleware > modules), so that doesn't require checking out the module > with the default name. > > A typical strategy is to create separate Eclipse workspaces, > each of which has all the required modules checked out, but > for different branches. Personally, I find it a pain to > switch workspaces in Eclipse, because I have a lot of > modules, and startup takes quite a while. As a result, I > often end up checking out many modules with a name like > "-", especially if I don't have to build > the EAR with that module, but just be able to inspect the > code, or get unit/integration tests working. > > So, in converting this to Maven, I envision an almost 1-1 > correspondence between the Ant projects and Maven projects, > except the "build" project will just run the build > (previously, we had sort of a "chain" system, where module > builds chained to the next required module), and "ear" > projects will specify just the needed dependencies and the > assembly phase, not to mention a parent POM or two. > > I want to allow for the possibility of not requiring modules > to be checked out with their "default" names, so I want to > have module references that reference property values, > instead of hardcoded names. > It seems to me that if I define a profile in my settings.xml > that corresponds to a branch, I can define the property > values for each module name, and then I can either specify > the profile on the command line, or if I'm working for a few > hours on one branch, I can just change the default profile. > I think this can work. > > Another desire I have is to allow for developers not even > building most of the modules, and just letting the "ear" > project pull snapshot artifacts from the Nexus repo (built by > the release team or continuous integration). I could do this > with multiple "build" projects, including different sets of > "module" elements. That seems messy, however. I wish I had > different options for setting that up, perhaps in a profile, > but I don't see how that could work. > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > > This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For addit
RE: Reasonable use of profiles?
> -Original Message- > From: Brian Topping [mailto:topp...@codehaus.org] > Sent: Friday, December 10, 2010 10:13 AM > To: Maven Users List > Subject: Re: Reasonable use of profiles? > > I'd second the very good points that Kalle made. To add some practical > experience from this end, I just last night learned the power of using > the element within a element... > > > > > > > all > > > > web > > core > > messaging > > security > > > > > > > > web > > > > web > > messaging > > security > > > > > > > > core > > > > core > > messaging > > security > > > > > > Interesting. This could go into the "build" project POM, but if a developer needed a custom list of modules, they could define that in their settings.xml. I guess that could work for what I'm looking for. > On Dec 10, 2010, at 12:57 PM, Kalle Korhonen wrote: > > > On Fri, Dec 10, 2010 at 9:20 AM, KARR, DAVID (ATTSI) > wrote: > >> Another desire I have is to allow for developers not even building > most > >> of the modules, and just letting the "ear" project pull snapshot > >> artifacts from the Nexus repo (built by the release team or > continuous > >> integration). I could do this with multiple "build" projects, > including > >> different sets of "module" elements. That seems messy, however. I > wish > >> I had different options for setting that up, perhaps in a profile, > but I > >> don't see how that could work. > > > > Overall, I'd say this sounds better than trying to (mis-)use > profiles. > > Nothing you said indicates to me that the EARs are functionally > > equivalent, and therefore I'd create different modules for different > > EARs. One of the Maven ground rules is one artifact per module. > > Typically you deviate from that plan only if you need to build > > different versions of the same module (packaged differently, for > > different OSes/JVMs etc.) and often you use profiles for the desired > > effect. I'd further say using profiles should never be your first > > option. Reserve profiles for the time you really need them. > > > > Since you are going to re-work the build, I'd mercilessly refactor it > > into a modular build now. Since Maven is so good at versioning > things, > > your build doesn't have to be monolithic. Snapshots are ok, but it's > > far better if you can identify common, stable areas of the codebase > > and simply release them separately. Then your top-level builds are > > mostly assembling things together rather than compiling/building > them. > > Personally, I'd put my efforts on making the build modular, > automating > > version migration and doing more continuous integration & testing > > rather than trying to force Maven the same logic as your Ant build. > > Depending on the size of the project and your team, it would likely > be > > beneficial to pay somebody who really knows Maven to assist you in > the > > migration at this point (if you can get it approved, I know how it > > is). It would save you from a lot of grief later. > > > > Kalle > > > > - > > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > > For additional commands, e-mail: users-h...@maven.apache.org > > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
RE: Reasonable use of profiles?
> -Original Message- > From: Kalle Korhonen [mailto:kalle.o.korho...@gmail.com] > Sent: Friday, December 10, 2010 9:58 AM > To: Maven Users List > Subject: Re: Reasonable use of profiles? > > On Fri, Dec 10, 2010 at 9:20 AM, KARR, DAVID (ATTSI) > wrote: > > Another desire I have is to allow for developers not even building > most > > of the modules, and just letting the "ear" project pull snapshot > > artifacts from the Nexus repo (built by the release team or > continuous > > integration). I could do this with multiple "build" projects, > including > > different sets of "module" elements. That seems messy, however. I > wish > > I had different options for setting that up, perhaps in a profile, > but I > > don't see how that could work. > > Overall, I'd say this sounds better than trying to (mis-)use profiles. > Nothing you said indicates to me that the EARs are functionally > equivalent, and therefore I'd create different modules for different > EARs. One of the Maven ground rules is one artifact per module. > Typically you deviate from that plan only if you need to build > different versions of the same module (packaged differently, for > different OSes/JVMs etc.) and often you use profiles for the desired > effect. I'd further say using profiles should never be your first > option. Reserve profiles for the time you really need them. To clarify the use case I'm talking about here, let's say the EAR requires 20 principal artifacts to construct the EAR (not to mention the numerous third-party dependencies). These have a 1-1 correspondence to modules. Some developers will work on one set of those modules (one or more), and other developers will work on other sets. No developers will work on all of the modules, but all developers need all of the produced artifacts in order to produce an EAR. So, if I only do work on one module, I could produce a build project that just has two modules, the module I work on, and the "EAR" module. I would get the other artifacts from the Nexus repo. Other developers would have to use a different build project, with a different set of modules, including the "EAR" module. Other developers would have a build project with a still different set of modules, including the "EAR" module. >From what I can tell, my only option is to create these numerous very similar >"build" projects, all with different permutations of the available modules. >Release management would never use these, these are just for developers. I'm simply looking for a better way to manage this. I envision if I somehow had a "list" property, consisting of module names, defined in "settings.xml", a developer could easily add or subtract from this list, and a single "build" project would use that property. I just don't know how to get something like this to work with Maven. > Since you are going to re-work the build, I'd mercilessly refactor it > into a modular build now. Since Maven is so good at versioning things, > your build doesn't have to be monolithic. Snapshots are ok, but it's > far better if you can identify common, stable areas of the codebase > and simply release them separately. Then your top-level builds are > mostly assembling things together rather than compiling/building them. > Personally, I'd put my efforts on making the build modular, automating > version migration and doing more continuous integration & testing > rather than trying to force Maven the same logic as your Ant build. > Depending on the size of the project and your team, it would likely be > beneficial to pay somebody who really knows Maven to assist you in the > migration at this point (if you can get it approved, I know how it > is). It would save you from a lot of grief later. What I have the leeway to produce is a structure that is compatible with Maven, but will still work with the Ant build. I can't bring down the entire development process while I single-handedly construct the Maven infrastructure. It was hard enough to get permission to even examine Maven. Each of our individual projects will produce a single artifact, whether from Ant or Maven, and the responsibility for assembling the EAR is now separated into a separate "build" project. That will allow me to integrate Maven with less pain after the slightly transformed Ant build is working. - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
I'd second the very good points that Kalle made. To add some practical experience from this end, I just last night learned the power of using the element within a element... > > > all > > web > core > messaging > security > > > > web > > web > messaging > security > > > > core > > core > messaging > security > > > I use Spring for the IoC container on my projects, and in an optional project for core, have a stubbed command bean that is defined once in the web project, then with a full implementation again in the core project, but with the "primary" attribute on the bean definition set to true. This transparently overrides the central command message interface with a real implementation, connecting the two projects together at runtime. The messaging and security projects are obviously needed by either project, whether they run separately or together. The messaging project contains the message classes that are sent to the stub or the core depending on what gets loaded, and the core project accepts those messages, thus creating a dependency from both. Brian On Dec 10, 2010, at 12:57 PM, Kalle Korhonen wrote: > On Fri, Dec 10, 2010 at 9:20 AM, KARR, DAVID (ATTSI) wrote: >> Another desire I have is to allow for developers not even building most >> of the modules, and just letting the "ear" project pull snapshot >> artifacts from the Nexus repo (built by the release team or continuous >> integration). I could do this with multiple "build" projects, including >> different sets of "module" elements. That seems messy, however. I wish >> I had different options for setting that up, perhaps in a profile, but I >> don't see how that could work. > > Overall, I'd say this sounds better than trying to (mis-)use profiles. > Nothing you said indicates to me that the EARs are functionally > equivalent, and therefore I'd create different modules for different > EARs. One of the Maven ground rules is one artifact per module. > Typically you deviate from that plan only if you need to build > different versions of the same module (packaged differently, for > different OSes/JVMs etc.) and often you use profiles for the desired > effect. I'd further say using profiles should never be your first > option. Reserve profiles for the time you really need them. > > Since you are going to re-work the build, I'd mercilessly refactor it > into a modular build now. Since Maven is so good at versioning things, > your build doesn't have to be monolithic. Snapshots are ok, but it's > far better if you can identify common, stable areas of the codebase > and simply release them separately. Then your top-level builds are > mostly assembling things together rather than compiling/building them. > Personally, I'd put my efforts on making the build modular, automating > version migration and doing more continuous integration & testing > rather than trying to force Maven the same logic as your Ant build. > Depending on the size of the project and your team, it would likely be > beneficial to pay somebody who really knows Maven to assist you in the > migration at this point (if you can get it approved, I know how it > is). It would save you from a lot of grief later. > > Kalle > > - > To unsubscribe, e-mail: users-unsubscr...@maven.apache.org > For additional commands, e-mail: users-h...@maven.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Reasonable use of profiles?
On Fri, Dec 10, 2010 at 9:20 AM, KARR, DAVID (ATTSI) wrote: > Another desire I have is to allow for developers not even building most > of the modules, and just letting the "ear" project pull snapshot > artifacts from the Nexus repo (built by the release team or continuous > integration). I could do this with multiple "build" projects, including > different sets of "module" elements. That seems messy, however. I wish > I had different options for setting that up, perhaps in a profile, but I > don't see how that could work. Overall, I'd say this sounds better than trying to (mis-)use profiles. Nothing you said indicates to me that the EARs are functionally equivalent, and therefore I'd create different modules for different EARs. One of the Maven ground rules is one artifact per module. Typically you deviate from that plan only if you need to build different versions of the same module (packaged differently, for different OSes/JVMs etc.) and often you use profiles for the desired effect. I'd further say using profiles should never be your first option. Reserve profiles for the time you really need them. Since you are going to re-work the build, I'd mercilessly refactor it into a modular build now. Since Maven is so good at versioning things, your build doesn't have to be monolithic. Snapshots are ok, but it's far better if you can identify common, stable areas of the codebase and simply release them separately. Then your top-level builds are mostly assembling things together rather than compiling/building them. Personally, I'd put my efforts on making the build modular, automating version migration and doing more continuous integration & testing rather than trying to force Maven the same logic as your Ant build. Depending on the size of the project and your team, it would likely be beneficial to pay somebody who really knows Maven to assist you in the migration at this point (if you can get it approved, I know how it is). It would save you from a lot of grief later. Kalle - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
