Re: Continuous Delivery and Maven

[jhumble]

Curt makes a good point - you can do CI without doing CD. The snapshot stuff
works just fine with CI.

It seems to be a tool that is prone to being used foolishly


I think that's probably true of any sufficiently powerful tool or technique.

Jez.

On 15 November 2010 10:23, Yanko, Curtis [via Maven] <
ml-node+3266162-1775935974-143...@n5.nabble.com
> wrote:

> I can't help but feel you are confusing the discussion on CD with CI
> which to me, is mostly a natural progression and maturity of a good
> build system.
>
> The other good news is, you don't have to use it foolishly if you choose
> not too! ;-)
>
> > -Original Message-
> > From: Ron Wheeler [mailto:[hidden 
> > email]]
>
> > Sent: Monday, November 15, 2010 10:28 AM
> > To: [hidden email]
> > Subject: Re: Continuous Delivery and Maven
> >
> > On 15/11/2010 8:18 AM, Yanko, Curtis wrote:
> > > You're happy about NOT using CI
> > >
> > Yes. It seems to be a tool that is prone to being used foolishly.
> >
> > We are a small shop maintaining and developing a large
> > (70+POM files) portal application with portlets, web
> > services, servlets and batch process and do not seem to  have
> > the types of issues that the people, trying to use CI, bring
> > to the table.
> >
> > They seem to get into all kinds of troubles with SNAPSHOTs,
> > build repeatability, source control and architectures that
> > are too interdependent.
> > I can not see how they ever test anything with a continually
> > unstable CI build.
> >
> > Of course, I know that I am only seeing the worst cases in
> > the forum so my mind is not completely closed on the subject.
> > I can hardly wait until we have a "Best Practice" section on
> > the Maven site so that I can see how a CI should be
> > integrated into a Maven environment and perhaps that will
> > make me unhappy that I an not using CI.
> >
> >
> > Ron
> > >> -Original Message-
> > >> From: Ron Wheeler [mailto:[hidden 
> > >> email]]
>
> > >> Sent: Saturday, November 13, 2010 2:05 PM
> > >> To: [hidden email]
> > >> Subject: Re: Continuous Delivery and Maven
> > >>
> > >> I would add the following bits of reality.
> > >> We don't use CI and a lot of the discussion makes me very
> > happy about
> > >> that.
> > >>
> > >
> > > -Curt
> > >
> > > This e-mail, including attachments, may include confidential and/or
> > > proprietary information, and may be used only by the person
> > or entity
> > > to which it is addressed. If the reader of this e-mail is not the
> > > intended recipient or his or her authorized agent, the reader is
> > > hereby notified that any dissemination, distribution or copying of
> > > this e-mail is prohibited. If you have received this e-mail
> > in error,
> > > please notify the sender by replying to this message and
> > delete this e-mail immediately.
> > >
> > >
> > >
> > -
> > > To unsubscribe, e-mail: [hidden 
> > > email]
> > > For additional commands, e-mail: [hidden 
> > > email]
> > >
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: [hidden 
> > email]
> > For additional commands, e-mail: [hidden 
> > email]
> >
> >
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>
>
> -
> To unsubscribe, e-mail: [hidden 
> email]
> For additional commands, e-mail: [hidden 
> email]
>
>
>
> --
>  View message @
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3266162.html
> To unsubscribe from Continuous Delivery and Maven, click 
> here.
>
>
>


-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in conte

Re: Continuous Delivery and Maven

[jhumble]

>
> I would like to see the whole "Best Practice" described. I would like
> to see how Hudson users deal with mock data and incremental development
> and testing of on-line applications where the MC and V teams are
> working together towards a fully functional system or a working bug fix or
> minor enhancement. How does one manage a production environment  with
> released systems functioning while new releases are being developed and
> patches are being applied to the current release?


Without wanting to bang my own drum too much, I just co-authored an entire
book which covers these topics in detail: Continuous Delivery
http://www.amazon.com/gp/product/0321601912?tag=contindelive-20

On 15 November 2010 07:58, Ron Wheeler [via Maven] <
ml-node+3265918-383914585-143...@n5.nabble.com
> wrote:

> On 15/11/2010 10:41 AM, Benson Margulies wrote:
> > Ron,
> >
> > It's not too hard to set up a CI process (e.g. on Hudson) that tests
> > the latest version of everything. Don't publish snapshots to your
> > repo, set up the cascade of jobs to share correctly.
> >
> > If that answers a question that is useful to you, great.
> >
> It is never as hard as people seem to think to set something up so that
> it works correctly but I am amazed at the hacked up development process
> that get described here.
>
> I would like to see the whole "Best Practice" described. I would like to
> see how Hudson users deal with mock data and incremental development and
> testing of on-line applications where the MC and V teams are working
> together towards a fully functional system or a working bug fix or minor
> enhancement.
> How does one manage a production environment  with released systems
> functioning while new releases are being developed and patches are being
> applied to the current release?
>
> > If, rather, you need to somehow model all kinds of combinations of
> > -SNAPSHOT and non-SNAPSHOT dependencies, or you feel compelled to
> > publish snapshots to your local repo, chaos is just around the corner.
> >
> In maintenance and bug-fixing, you do need to mix Releases with
> SNAPSHOTs to build a full system since you might only be releasing 2
> portlets out of 50 to add a new small function or fix a bug.  The
> overhead of rebuilding 70 modules to get 2 fixed is just not something
> that we can support.
>
> We do publish SNAPSHOTS to the internal repo but they come with a
> warranty and some functional spec that the rest of the team can live with.
> This does not cause a problem because we know what we are building and
> know the combination that has to be tested within the scope of each
> active project.
>
>
> Ron
>
> >
> > On Mon, Nov 15, 2010 at 10:27 AM, Ron Wheeler
> > <[hidden email] >
>  wrote:
> >> On 15/11/2010 8:18 AM, Yanko, Curtis wrote:
> >>> You're happy about NOT using CI
> >>>
> >> Yes. It seems to be a tool that is prone to being used foolishly.
> >>
> >> We are a small shop maintaining and developing a large (70+POM files)
> portal
> >> application with portlets, web services, servlets and batch process and
> do
> >> not seem to  have the types of issues that the people, trying to use CI,
>
> >> bring to the table.
> >>
> >> They seem to get into all kinds of troubles with SNAPSHOTs, build
> >> repeatability, source control and architectures that are too
> interdependent.
> >> I can not see how they ever test anything with a continually unstable CI
>
> >> build.
> >>
> >> Of course, I know that I am only seeing the worst cases in the forum so
> my
> >> mind is not completely closed on the subject.
> >> I can hardly wait until we have a "Best Practice" section on the Maven
> site
> >> so that I can see how a CI should be integrated into a Maven environment
> and
> >> perhaps that will make me unhappy that I an not using CI.
> >>
> >>
> >> Ron
>  -Original Message-
>  From: Ron Wheeler [mailto:[hidden 
>  email]]
>
>  Sent: Saturday, November 13, 2010 2:05 PM
>  To: [hidden email]
>  Subject: Re: Continuous Delivery and Maven
> 
>  I would add the following bits of reality.
>  We don't use CI and a lot of the discussion makes me very
>  happy about that.
> 
> >>> -Curt
> >>>
> >>> This e-mail, including attachments, may include confidential and/or
> >>> proprietary information, and may be used only by the person or entity
> >>> to which it is addressed. If the reader of this e-mail is not the
> intended
> >>> recipient or his or her authorized agent, the reader is hereby notified
>
> >>> that any dissemination, distribution or copying of this e-mail is
> >>> prohibited. If you have received this e-mail in error, please notify
> the
> >>> sender by replying to this message and delete this e-mail immediately.
> >>>
> >>>
> >>> -

Re: RE: Continuous Delivery and Maven

[jhumble]

BTW - I also think Stephen's suggestion is a good one - it's more or less
what I suggest in the book.

On 10 November 2010 09:45, Jez Humble  wrote:

> Hey Todd.
>
> My interpretation of what Jason is saying in his comment here -
> http://www.lucasward.net/2010/11/maven-and-continuous-delivery.html - "you
> should be able to tag the code, and pluck the snapshot out and promote it to
> a release repository... Someone just needs to do the two weeks of work to
> add the tooling." is that this is exactly what he's saying. I'm going to
> have a nose around the code and see if I think I can take a shot at it.
>
> Thanks,
>
> Jez.
>
> On 10 November 2010 05:51, Thiessen, Todd (Todd) [via Maven] <
> ml-node+3258691-1180604705-143...@n5.nabble.com
> > wrote:
>
>> Agreed Stephen. I think your proposal is the best so far. I just want to
>> make sure we don't go back into a discussion about promoting snapshots to
>> releases, as I believe that is where this discussion started.  To do that I
>> think would require a significant re-architecture of maven which I don't
>> think would be happening in the near future.
>>
>> > -Original Message-
>> > From: Stephen Connolly [mailto:[hidden 
>> > email]]
>>
>> > Sent: Wednesday, November 10, 2010 8:29 AM
>> > To: Maven Users List
>> > Subject: Re: RE: Continuous Delivery and Maven
>> >
>> > Imho taking a snapshot artefact and renaming as a release artefact and
>> > pushing to the remote repo will never be supported by maven, but my
>> > ship-maven-plugin (not ready for publishing yet) and some extra mojos
>> > added
>> > to versions-maven-plugin should enable CD in a way that is in keeping
>> > with
>> > the "maven way", as long as ranges are used in place of snapshots. (will
>>
>> > also need minor tweaks to the maven-release-plugin, but these would be
>> > backwards compatible tweaks that do not change its current behaviour,
>> > only
>> > provide a hook for extending it)
>> >
>> > - Stephen
>> >
>> > On 10 Nov 2010 13:03, "Thiessen, Todd (Todd)" <[hidden 
>> > email]>
>>
>> > wrote:
>> >
>> > I don't think thats the same thing. The proposal is to take a snapshot
>> > artifact which was built using mvn deploy and promote it to the release
>> > repo. I think what you are referring to here Jason is how the release
>> > plugin
>> > first builds the snapshot, tags it, and then rebuilds the tag.
>> >
>> >
>> > > -Original Message-
>> > > From: Jason van Zyl [mailto:[hidden 
>> > > email]]
>>
>> > > Sent: Wednesday, Nove...
>> >
>> > > Subject: Re: Continuous Delivery and Maven
>> > >
>> > >
>> > > On Nov 9, 2010, at 2:25 PM, Thiessen, Todd (Tod...
>>
>> -
>> To unsubscribe, e-mail: [hidden 
>> email]
>> For additional commands, e-mail: [hidden 
>> email]
>>
>>
>>
>> --
>>  View message @
>> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3258691.html
>> To unsubscribe from Continuous Delivery and Maven, click 
>> here.
>>
>>
>>
>
>
> --
> Jez Humble
> Co-author, *Continuous Delivery *
> http://continuousdelivery.com/
> http://jezhumble.net/
>
>
>


-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3259081.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: RE: Continuous Delivery and Maven

[jhumble]

Hey Todd.

My interpretation of what Jason is saying in his comment here -
http://www.lucasward.net/2010/11/maven-and-continuous-delivery.html - "you
should be able to tag the code, and pluck the snapshot out and promote it to
a release repository... Someone just needs to do the two weeks of work to
add the tooling." is that this is exactly what he's saying. I'm going to
have a nose around the code and see if I think I can take a shot at it.

Thanks,

Jez.

On 10 November 2010 05:51, Thiessen, Todd (Todd) [via Maven] <
ml-node+3258691-1180604705-143...@n5.nabble.com
> wrote:

> Agreed Stephen. I think your proposal is the best so far. I just want to
> make sure we don't go back into a discussion about promoting snapshots to
> releases, as I believe that is where this discussion started.  To do that I
> think would require a significant re-architecture of maven which I don't
> think would be happening in the near future.
>
> > -Original Message-
> > From: Stephen Connolly [mailto:[hidden 
> > email]]
>
> > Sent: Wednesday, November 10, 2010 8:29 AM
> > To: Maven Users List
> > Subject: Re: RE: Continuous Delivery and Maven
> >
> > Imho taking a snapshot artefact and renaming as a release artefact and
> > pushing to the remote repo will never be supported by maven, but my
> > ship-maven-plugin (not ready for publishing yet) and some extra mojos
> > added
> > to versions-maven-plugin should enable CD in a way that is in keeping
> > with
> > the "maven way", as long as ranges are used in place of snapshots. (will
> > also need minor tweaks to the maven-release-plugin, but these would be
> > backwards compatible tweaks that do not change its current behaviour,
> > only
> > provide a hook for extending it)
> >
> > - Stephen
> >
> > On 10 Nov 2010 13:03, "Thiessen, Todd (Todd)" <[hidden 
> > email]>
>
> > wrote:
> >
> > I don't think thats the same thing. The proposal is to take a snapshot
> > artifact which was built using mvn deploy and promote it to the release
> > repo. I think what you are referring to here Jason is how the release
> > plugin
> > first builds the snapshot, tags it, and then rebuilds the tag.
> >
> >
> > > -Original Message-
> > > From: Jason van Zyl [mailto:[hidden 
> > > email]]
>
> > > Sent: Wednesday, Nove...
> >
> > > Subject: Re: Continuous Delivery and Maven
> > >
> > >
> > > On Nov 9, 2010, at 2:25 PM, Thiessen, Todd (Tod...
>
> -
> To unsubscribe, e-mail: [hidden 
> email]
> For additional commands, e-mail: [hidden 
> email]
>
>
>
> --
>  View message @
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3258691.html
> To unsubscribe from Continuous Delivery and Maven, click 
> here.
>
>
>


-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3259078.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Fair enough. I would love to believe that Maven will work just fine with CD
in its present state, and perhaps you have cracked it. It's just that I have
heard enough people I trust tell me otherwise so I wanted to see what people
are doing and throw some ideas around.

Anyway I'm signing off for the day, but thanks to you and the rest of the
group for an interesting discussion :-)

Jez.

On 8 November 2010 18:27, Yanko, Curtis [via Maven] <
ml-node+3256136-495546356-143...@n5.nabble.com
> wrote:

>
> --And I think this is the nub of the problem - you don't have a
> sufficiently frequent release cycle to experience the problems I am
> describing.
>
> I too came to this same conclusion. I get that now, even every two weeks
> isn't enough but I still don't see my process being a pain even if we
> released a few times daily, I really don't.
>
> Also understand that I am committed to understanding how I'm going to
> get my projects to CD some day but my assumption was that I'd be
> building on my existing work and figure it must be my ignorance that
> makes me feel like the rug is being pulled out. I confess I haven't read
> your book yet but it is on my short list so only a matter of time I'm
> sure.
>
> 
>
> Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> Making IT Happen, one build at a time, 600 times a day
>
> -Original Message-
> From: jhumble [mailto:[hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3256136&i=0>]
>
> Sent: Monday, November 08, 2010 6:54 PM
> To: [hidden email] <http://user/SendEmail.jtp?type=node&node=3256136&i=1>
> Subject: Re: Continuous Delivery and Maven
>
>
> >
> > My mantra is this, if you are feeling pain in your process, do it more
>
> > often which should force you to either solve the problem or realize
> > it's upstream.
>
>
> That is one of my 8 principles of CD ("If It Hurts, Do It More
> Frequently, and Bring the Pain Forward" - p26). And I think it applies
> to integrating modules too.
>
>
> > You're trying to solve a two-sided jello view of a system at build
> > time, perhaps what you need is to refactor you architecture (and why
> > SaaS, Cloud and federated systems lend themselves to CD). Maven is
> > your pain point but I say you're just not doing it right.
> >
>
> We can both play that game, but it's not very productive. I respect the
> fact that you have a bunch of experience and that you've created a
> process that works great for you, and I am not suggesting you change it.
> Hard as it may be to believe, I have had many experiences that point me
> in exactly the opposite direction, also working for organizations that
> make "our little
> A->B->C game" seem trivial.
>
> So, after we *find something valuable* we can release it trivially.
> >
>
> And I think this is the nub of the problem - you don't have a
> sufficiently frequent release cycle to experience the problems I am
> describing.
>
> --
> Jez Humble
> Co-author, *Continuous Delivery <http://continuousdelivery.com/>*
> http://continuousdelivery.com/ http://jezhumble.net/
>
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370<http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370?by-user=t>
> p3256015.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the intended
> recipient or his or her authorized agent, the reader is hereby notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify the
> sender by replying to this message and delete this e-mail immediately.
>
>
> -
> To unsubscribe, e-mail: [hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3256136&i=2>
> For additional commands, e-mail: [hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3256136&i=3>
>
>
>
> --
>  View message @
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3256136.html
> To unsubscribe from Continuous Delivery and Maven, click 
> here<http://maven.40175.n5.nabble.com/template/TplServlet.jtp?tpl=unsubscribe_by_code&node=3245370&code=amV6QGplemh1bWJsZS5uZXR8MzI0NTM3MHwtMTg4MjM1NzMyNA==>.
>
>
>


-- 
Jez Humble
Co-author, *Continuous Delivery <http://continuousdelivery.com/>*
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3256153.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

OK this is great - I think we're on the same page now.


> And I am pretty sure that the community in general is not really in favor
> of this kind of massaging of the artifacts from snapshot -> release. It
> invalidates your testing of said snapshot.
>
> I think it is crucial that the release artifact DOES get rebuilt.


I think that exactly the opposite is true - if you recreate the release
artifact, it invalidates your testing of the snapshot, because by definition
you're releasing something different from what you've tested. Worse, because
Maven doesn't currently include that metadata, it might well actually be a
different binary!


> > That would be great, and the obvious place would be in MANIFEST.MF, but
> > the
> > format isn't rich enough to store all the information we'd need.
>
> What would be missing? Your pom says your using artifact 1.0-SNAPSHOT.jar.
> You go and find that jar, open it and find what revision. Do that for all
> snapshot jars you depend on.
>

But I want to know what versions of the upstream jars were that I built my
snapshot against too.


> The second is that maven would have to query this information during a
> build, and then update it own "extra information.xml file" to update its
> transtitive dependency information list and then deploy that list to the
> repository as well. And this would need to be a complete list of all
> transtive dependencies, not just direct ones. So now you can download that
> file, figure out what revision of all your transitive deps are and recreate
> the entire build.
>

Exactly.

Now how would this work if a project has release and snapshot dependencies?
> I think you would only need to do this for snapshot deps.
>

Well for release dependencies you already specify the exact version number
you depend on, so it doesn't matter, right?

Thanks,

Jez.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3256025.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

>
> My mantra is this, if you are feeling pain in your process, do it more
> often which should force you to either solve the problem or realize it's
> upstream.


That is one of my 8 principles of CD ("If It Hurts, Do It More Frequently,
and Bring the Pain Forward" - p26). And I think it applies to integrating
modules too.


> You're trying to solve a two-sided jello view of a system at
> build time, perhaps what you need is to refactor you architecture (and
> why SaaS, Cloud and federated systems lend themselves to CD). Maven is
> your pain point but I say you're just not doing it right.
>

We can both play that game, but it's not very productive. I respect the fact
that you have a bunch of experience and that you've created a process that
works great for you, and I am not suggesting you change it. Hard as it may
be to believe, I have had many experiences that point me in exactly the
opposite direction, also working for organizations that make "our little
A->B->C game" seem trivial.

So, after we *find something valuable* we can release it trivially.
>

And I think this is the nub of the problem - you don't have a sufficiently
frequent release cycle to experience the problems I am describing.

--
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3256015.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

>
>
> And in a community driven environment, if something isn't perceived as
> valuable, it generally doesn't get done. You are welcome to create a plugin
> to do this or perhaps enhance an existing one. If the community feels it is
> valuable, you will get a lot of help to make it happen. If the community
> doesn't, then you won't ;-).


Sure, I get that, that's why I'm polling the maven users list before I break
out IntelliJ :-)

Jez.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255947.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

>
> At the risk of derailing the conversion again though, there is likely a
> good reason for why it has been such an up hill battle for you.  I hope you
> are prepared to consider that CD is not as great of a solution to a lot of
> the industry as you wish it to be.


Well, if I and many others had not seen it make a huge difference to many
organizations of many different types over many years, I would not have
bothered investing 4 years of my personal life writing the book. What I'm
proposing is not something new and magical - it's something that many, many
others have successfully implemented. I am just the messenger here - please
don't shoot me ;-)

More seriously, sure, it is not a silver bullet. Where it really matters is
when you're delivering software that is strategic for business success. If
you're not doing that, it's probably not that important.

Let's assume that it is useful for some set of people. Why the resistance to
Maven supporting this methodology?


> If you don't change the binaries, they will still be a "snapshot" binary;
> which never goes to a customer. For example, when you search a maven
> repository, you will generally look for artifacts in a released reository.
> If your customer artifacts are now in the snapshot repo, everyone is going
> to have to start searching the released artifacts in the snapshot repos.
> This is just confusing.
>

That's fine. What I'm suggesting is simply that Maven creates enough
metadata when building a snapshot that the release plug-in can more or less
just copy it to the released repository when necessary. Crucially though it
won't cause it to be rebuilt.


> I think maybe the part you're missing is the underlying culture of Maven.
> If I need an artifact, you type it in a repository search, hit enter and
> bingo, the dependency appears in your pom. It takes care of downloading all
> of its transitive dependenies. I don't have to worry about getting latest
> version of the dependencies, fixed versions If it's a released artifact,
> I get fixed... if its snapshot, I get the latest. I don't have to look on
> the internet and waste a whole bunch of time. It makes a devs life much
> faster and much easier.
>

Yep, I absolutely undertand that. I am not trying to change anything about
what you just said. In fact, I want to leverage it.

> All I'm doing is adding some more metadata (in the form of
> > changing
> > the names, doing tagging, or whatever)
>
> How does your build know about this extra metadata?  Where are you going to
> store it?
>

I'm proposing to store it in the pom file that gets created along with the
binary (*not* the project's pom file!) - see the example below

A version of 1.0-SNAPSHOT simply means latest 1.0 version. Nice, clean and
> simple. If you want a fixed version you say 1.0-01. Again, nice clean and
> simple.
>

Right. Again, I'm not proposing to change this.

I think the best place to store this information is in the binary itself.
> That will allow you to recreate any build.


That would be great, and the obvious place would be in MANIFEST.MF, but the
format isn't rich enough to store all the information we'd need. Another
possibility would be to create a custom maven XML extension file inside
META-INF.


> Here is an example snapshot folder:
>
>
> https://repository.apache.org/content/repositories/maven-snapshots-sonatype/org/apache/maven/maven-core/3.0-SNAPSHOT/


Right - so you see this file:
https://repository.apache.org/content/repositories/maven-snapshots-sonatype/org/apache/maven/maven-core/3.0-SNAPSHOT/maven-core-3.0-20101004.110147-683.pom

All I'm suggesting is that in addition to the  and 
tags that are attached to each  we *also* store something like
http://svn.apache.org/blah/blah
1423

That's it.

The whole idea is that you are working off the latest, not a specific
> version. If you are storing this information in source control (whether it's
> the pom or in its own file) then you have to somehow constantly change that
> information to always point to the latest. I don't think there is logically
> away around that ;-).
>

So just to reiterate - I'm not suggesting we change the behavior. All I'm
suggesting is that we *record* the information on what version in source
control each snapshot artifact came from and propagate this information down
the dependency chain so that doing a "release" in maven is just a question
of copying the artifact to the release repo and updating some of the
metadata in its associated pom file - say, giving it a "real" identifier.

Thanks,

Jez.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255940.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

>
> Only my stuff can be a SNAPSHOT and it is either all a snapshot or it is
> not. So, if B & C are mine it's not an issue, if they belong to some one
> else's, they can't be SNAPSHOTs, it's that simple. I can even use the
> enforcer to ensure there isn't a SNAPSHOT set anywhere as part of the
> inspection process.
>

This goes back to my original example. In large development teams it is very
common for different groups to work on different modules which must all be
integrated together for the application to work. In those situations it is
common for all of the modules to change quite regularly. As a result, you
want each of those teams to use snapshots because it's too painful to be
always creating new release versions and update the pom files by hand.

But you also don't want to throw away the testing you do with those
snapshots.

This is what motivates the idea of including enough metadata with the
snapshots so that the binaries can ultimately be released *if required.*


>  We release by changing one entry in one POM and all of our stuff gets
> versioned, built and released. Repeat the process and everything is back
> to the next SNAPSHOT so we can resume changing things. It seems it is
> this one discreet activity that CD abhors. You want to defer it to after
> the fact which doesn't seem any different than what Stephen suggested by
> waiting for CI feedback and then trigger a CD build. I don't get what
> event prompts you to wan to go back and reproduce a build to be a
> release?
>

But if you are continuously integrating the modules, which you should be (CI
applies at the module level, not just at the code level), then you'd be
changing the POMs *all the time*.


> The ours vs. theirs problem exist for your plugin scenario too. Sure
> you've created a synthetic POM but have they? If B & C's  development is
> in fact decoupled from yours, why would they? How would they know? If
> they didn't make one and in turn are using SNAPSHOT deps themselves then
> you're hosed and your problem is spiraling in complexity.
>

If creating the extra metadata is built in to the tool, then we ensure that
everybody has the pom with the extra metadata and there isn't a problem.
Anyone who doesn't need it can ignore it.


> If your goal is to change anything anywhere at anytime and still deliver
> any of it, good luck.


No, that's absolutely not my goal. I want to make what I think is a very
simple change to the way Maven works so that people can carry on with their
existing methodology, but incrementally create a full deployment pipeline
too if they find it to be valuable.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255726.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Hi Curtis


> What you're proposing is indeed interesting. But you do seem to pick and
> choose what you want (reproducibility) and what you can dispense with
> (tags and branches).
>

Correct. In my defence, I have extremely good reasons why I want
reproducibility and don't care about tagging, derived from high level
principles about how to do software delivery in a reliable, efficient way.
You may not care about them, and that's fine, but I want any tool I use to
be able to work according to these principles.


> I'm just saying, that as a practical matter, we can today solve your
> problem without the need for a new plugin. Reproducible, and Auditable.
> I can look at anything directly, even in Prod, and make the leap all the
> way to SVN without a single cross reference or lookup because we put all
> that meta-data in the manifest, just not in the POM (in your scenario,
> in ours the POM has everything listed as a version though so we don't
> face the SNAPSHOT issue).
>

Fair enough, but you have to do a bunch of extra work to include that
information in the manifest, and the manifest doesn't include information on
transitive dependencies. I want the tool to support it out of the box by
including the information in the pom that goes along with the artifacts it
creates.

Of course, the pom is in XML which is eXtensible, so existing tools can
ignore this extra information if they like! So I don't see why it should be
a problem.

You want to be able to Audit but eschew documentation like a Site
> reports? Again seems sort of cake-and-eat-it-too to me.
>

Perhaps I expressed myself poorly. I am fine creating documentation like
site reports, but I want to be able to derive the information in the site
report from the metadata associated with my binaries.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255698.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Hi Todd.


> Ok. I will refocus here. Let me just say that this discussion is
> inevitable. It is important as the value of CD will help determine if Maven
> should consider working towards supporting this kind of process.  The basic
> architecture of Maven today does not lend well to it.  So we can table it
> for now. You comments I think took what I am trying to get across very much
> out of context but *snip* them for now and defer the discussion to a later
> date.
>

My apologies - I realize it's somewhat unfair of me to say I don't want to
discuss the value proposition any more and then dissect a bunch of what you
said. I guess I have been explaining the value proposition quite a lot
recently and would like to get back to some more technical stuff, but that
is my problem, not yours. Having said that, I wrote something which you may
find interesting here:
http://www.informit.com/articles/article.aspx?p=1641923 - it has a bunch of
links to other stuff too.


> The final artifacts and all the meta do not represent what is considered a
> released artifact.  The meta data is different and the artifact names are
> different. So the release process would have to "massage" the snapshot
> artifacts to "look" like releases. This will of course invalidate much of
> the testing done by your CI.
>

So long as I don't change the actual binaries, how does this invalidate the
testing? All I'm doing is adding some more metadata (in the form of changing
the names, doing tagging, or whatever)


> When I say meta data here, I am referring to the meta data maven stores
> about the artifact. This information doesn't appear in the pom.
>

I must admit my ignorance here - I thought Maven stored metadata about the
artifact in the form of pom files (e.g. this one:
http://repo2.maven.org/maven2/HTTPClient/HTTPClient/0.3-3/HTTPClient-0.3-3.pom
)


> Also if you do this you will be breaking the idea of CI. If your
> dependencies point to specific time stamp then that build won't be getting
> the latest version of those dependencies. You'd need some process to
> constantly update the pom to always point to the latest version of all your
> dependencies.  This I think would be fairly complex as your pom files would
> be constantly changing, even if you as a developer didn't need to change it.
>
>

So the idea is that the pom files for the build just point to the upstream
snapshot as happens right now.

However when the build artifact is created as part of the CI process, the
pom that is created with it also includes the information about exactly
which versions of these snapshots were used, including the version in
version control they were created from.

You can then re-use the same binaries you created from CI throughout the
rest of your delivery process.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255684.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Right - but I don't want to rely on the build logs or the site report, I
want the bill of materials in an easily accessible and simple format that
Maven itself can understand - for example in the pom file associated with
the snapshot.

In particular, I want to be able to point the Maven release plug-in at the
snapshot's pom and run a command which does all the kind of tagging stuff
that the release plug-in does, but that doesn't need to reproduce the binary
(except to verify that it's identical to the snapshot). I'd also want to be
able to create the site report automatically from the snapshot's pom.

I don't ever need to re-produce it because I have it and am managing the
> binary


Except that I do want to keep the ability to blow away the binaries and know
I can re-create them, especially when I'm running multiple builds per day.

More importantly, it's essential for auditing purposes that we can trace
back from any given binary to the exact versions in version control that it
and all its build-time dependencies came from.

On 8 November 2010 09:34, Yanko, Curtis [via Maven] <
ml-node+3255434-1467695194-143...@n5.nabble.com
> wrote:

> Thanks,
>
> I did manage to find it too. Not sure I follow his logic though.
>
>"One of the things I like about snapshots is it just simply means
> "latest".  Though the thing about timestamped snapshots is that they
> aren't guaranteed to exist (the repository is not typically assumed to
> be reliable), and they aren't 100% reproducible (the timestamp offset
> includes the time it took to build the artifact and all the artifacts
> before it, meaning there's no way to know exactly what point in time the
> build came from).  Even if one could find the correct timestamp to check
> out from to get the same binary, whatever subsystem creates the
> timestamp on upload (wagon?) probably doesn't like being told what to
> call the snapshot."
>
> If I build a SNAPSHOT and deploy it to an internal Maven Repo it will be
> given a unique identifier based on a time stamp (repo configured to do
> this). Why is that not guaranteed to exist or be reliable? And at the
> moment I build A the build log will tell me exactly which SNAPSHOT we
> received. I don't ever need to re-produce it because I have it and am
> managing the binary but... I can crack it open and see exactly which SCC
> revision was used and which path within the SCC it came from (because we
> bake that info into everything we build). So I can reproduce it in a
> pinch. What's also missing from the conversation is the final assembly
> where A,B and C get put together for deployment which is also managed as
> a binary and represents the bill-of-materials. And of course my Maven
> Site report for A would have documented which B & C I had too.
>
> ________
>
> Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> Making IT Happen, one build at a time, 600 times a day
>
> -Original Message-
> From: jhumble [mailto:[hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3255434&i=0>]
>
> Sent: Monday, November 08, 2010 12:11 PM
> To: [hidden email] <http://user/SendEmail.jtp?type=node&node=3255434&i=1>
> Subject: Re: Continuous Delivery and Maven
>
>
> Hi Curt - it was this one:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370<http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370?by-user=t>
> p3254439.html
>
> On 8 November 2010 09:07, Yanko, Curtis [via Maven] <
> [hidden email] 
> <http://user/SendEmail.jtp?type=node&node=3255434&i=2>
> [hidden email] <http://user/SendEmail.jtp?type=node&node=3255434&i=3>>
> > wrote:
>
> >
> > I didn't see a reply from a Brian. What answer did he provided that
> > answered your question?
> >
> > 
> >
> > Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> > Making IT Happen, one build at a time, 600 times a day
> >
> > -Original Message-
> > From: jhumble [mailto:[hidden
> > email]<http://user/SendEmail.jtp?type=node&node=3255382&i=0>]
> >
> > Sent: Monday, November 08, 2010 11:58 AM
> > To: [hidden email]
> > <http://user/SendEmail.jtp?type=node&node=3255382&i=1>
> > Subject: Re: Continuous Delivery and Maven
> >
> >
> > Todd, I have read all of your posts and I have come to the conclusion
> > that you're missing the point of CD. I was really hoping to avoid an
> > argument about process, because I just want to work out what needs to

Re: Continuous Delivery and Maven

[jhumble]

Hi Curt - it was this one:
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3254439.html

On 8 November 2010 09:07, Yanko, Curtis [via Maven] <
ml-node+3255382-175104679-143...@n5.nabble.com
> wrote:

>
> I didn't see a reply from a Brian. What answer did he provided that
> answered your question?
>
> 
>
> Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> Making IT Happen, one build at a time, 600 times a day
>
> -Original Message-
> From: jhumble [mailto:[hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3255382&i=0>]
>
> Sent: Monday, November 08, 2010 11:58 AM
> To: [hidden email] <http://user/SendEmail.jtp?type=node&node=3255382&i=1>
> Subject: Re: Continuous Delivery and Maven
>
>
> Todd, I have read all of your posts and I have come to the conclusion
> that you're missing the point of CD. I was really hoping to avoid an
> argument about process, because I just want to work out what needs to be
> done to Maven to make it support CD, and that's already a big enough
> discussion for one thread. However since the thread has (perhaps
> inevitably) been taken over by a discussion about what continuous
> delivery is, I will add my 1c. In any case I think I have what I need
> from the discussion with Brian.
>
> With CD, the software is *always* production ready, right from the start
> of the project. Any work of any kind that doesn't result in a deployable
> build is waste.
>
> If you are at the start of a release, your product owner will have a
> good
> > idea of how much content needs to get to the customer to fullfill that
>
> > release. Doing CD through the entire lifecycle is largely a waste
> IMHO.
>
>
> Wrong. In fact, it's the opposite - any work that doesn't keep the
> software in a deployable, releasable state is waste, because you can't
> know whether or not the work you have done is actually useful, or even
> whether it keeps the software working. And you can't know whether or not
> the software is working - i.e. whether or not the build can be deployed
> - until it has passed end-to-end acceptance tests under realistic loads
> in a production-like environment.
>
> I am fine with you using the process you describe. If it works for you,
> that's great. But please don't call it continuous delivery - it isn't.
>
> Now, assuming we are working in a cd process, the crucial thing is that
> we don't waste any cycles creating a build that couldn't be released. We
> then take this binary and put it through the rest of the deployment
> pipeline (or build life or whatever you want to call it). But crucially,
> we don't want to recreate the binary later on. If you want more detail
> on the mechanics of how it works, you can read the free chapter from my
> book here:
> http://www.informit.com/articles/article.aspx?p=1621865
>
> *What I want from Maven*
> *===*
>
> We want the simplicity of snapshots with the traceability of proper
> releases. So I think from what Brian said, I'd like the the Maven
> snapshot build process to create enough metadata in the pom file such
> that when you ran the release plugin, it wouldn't be necessary for it to
> rebuild the artifact - it could just do the various bits of tagging and
> metadata creation using the information in the pom associated with the
> snapshot. We might also want the release plugin to try and recreate the
> binary using its process and verify the md5 is the same as the md5 of
> the snapshot.
>
> If anybody has any feedback on this hypothesis, I'd be very grateful.
>
> Thanks,
>
> Jez.
>
> On 8 November 2010 08:49, Thiessen, Todd (Todd) [via Maven] <
> [hidden email] 
> <http://user/SendEmail.jtp?type=node&node=3255382&i=2>
> [hidden email] <http://user/SendEmail.jtp?type=node&node=3255382&i=3>>
> > wrote:
>
> > > I'm thinking tha Ci wouldn't be affected at all, CD still requires
> > > Ci as a quality metric preventing deployment to the customer.
> >
> > I am curious to see that. Or how it would work. How do you put in
> > fixed release numbers into a CD build and then switch back to CI
> > building? And I can only imagine it being quite complex.
> >
> > The only thing I can think of is something like:
> >
> > 1. CI build produces 1.0-SNAPSHOT
> > 2. CD build produces 1.0-01
> > 3. CD build reverts source back to 1.0-SNAPSHOT 4. Repeat
> >
> >
> > -
> > To unsubscri

Re: Continuous Delivery and Maven

[jhumble]

>
>  You may use any number of commits to complete a particular user story.


But you can still keep your software releasable by implementing features in
an incremental way, as described here:
http://martinfowler.com/bliki/FeatureToggle.html

I don't see how delivering an in progress story has any value.


Two reasons: one, the faster you get feedback on the part of the story you
have done so far, the faster you know if any further work is going to be
valuable, and what in fact the next most valuable thing to deliver is.

Second, most of the pain of the software delivery process comes *after* the
software is dev-complete, during testing and deployment (often called the
"last mile"). One of the important points of cd is that by creating
deployable software with every commit, you avoid the pain at the end of the
delivery process.

Jez.

On 8 November 2010 08:52, Thiessen, Todd (Todd) [via Maven] <
ml-node+3255344-939946112-143...@n5.nabble.com
> wrote:

>
> > In my view of CD, you wouldn't come back, ever. You'd just commit code
> > that triggers a build and if it passes each and every quality gate along
> > the way it would get deployed.
>
> I am even struggling with that. A commit of some code, does not imply a
> deliverable feature.  You may use any number of commits to complete a
> particular user story. I don't see how delivering an in progress story has
> any value.
>
> -
> To unsubscribe, e-mail: [hidden 
> email]
> For additional commands, e-mail: [hidden 
> email]
>
>
>
> --
>  View message @
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255344.html
> To unsubscribe from Continuous Delivery and Maven, click 
> here.
>
>
>


-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255375.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Todd, I have read all of your posts and I have come to the conclusion that
you're missing the point of CD. I was really hoping to avoid an argument
about process, because I just want to work out what needs to be done to
Maven to make it support CD, and that's already a big enough discussion for
one thread. However since the thread has (perhaps inevitably) been taken
over by a discussion about what continuous delivery is, I will add my 1c. In
any case I think I have what I need from the discussion with Brian.

With CD, the software is *always* production ready, right from the start of
the project. Any work of any kind that doesn't result in a deployable build
is waste.

If you are at the start of a release, your product owner will have a good
> idea of how much content needs to get to the customer to fullfill that
> release. Doing CD through the entire lifecycle is largely a waste IMHO.


Wrong. In fact, it's the opposite - any work that doesn't keep the software
in a deployable, releasable state is waste, because you can't know whether
or not the work you have done is actually useful, or even whether it keeps
the software working. And you can't know whether or not the software is
working - i.e. whether or not the build can be deployed - until it has
passed end-to-end acceptance tests under realistic loads in a
production-like environment.

I am fine with you using the process you describe. If it works for you,
that's great. But please don't call it continuous delivery - it isn't.

Now, assuming we are working in a cd process, the crucial thing is that we
don't waste any cycles creating a build that couldn't be released. We then
take this binary and put it through the rest of the deployment pipeline (or
build life or whatever you want to call it). But crucially, we don't want to
recreate the binary later on. If you want more detail on the mechanics of
how it works, you can read the free chapter from my book here:
http://www.informit.com/articles/article.aspx?p=1621865

*What I want from Maven*
*===*

We want the simplicity of snapshots with the traceability of proper
releases. So I think from what Brian said, I'd like the the Maven snapshot
build process to create enough metadata in the pom file such that when you
ran the release plugin, it wouldn't be necessary for it to rebuild the
artifact - it could just do the various bits of tagging and metadata
creation using the information in the pom associated with the snapshot. We
might also want the release plugin to try and recreate the binary using its
process and verify the md5 is the same as the md5 of the snapshot.

If anybody has any feedback on this hypothesis, I'd be very grateful.

Thanks,

Jez.

On 8 November 2010 08:49, Thiessen, Todd (Todd) [via Maven] <
ml-node+3255336-1962346083-143...@n5.nabble.com
> wrote:

> > I'm thinking tha Ci wouldn't be affected at all, CD still requires Ci
> > as a quality metric preventing deployment to the customer.
>
> I am curious to see that. Or how it would work. How do you put in fixed
> release numbers into a CD build and then switch back to CI building? And I
> can only imagine it being quite complex.
>
> The only thing I can think of is something like:
>
> 1. CI build produces 1.0-SNAPSHOT
> 2. CD build produces 1.0-01
> 3. CD build reverts source back to 1.0-SNAPSHOT
> 4. Repeat
>
>
> -
> To unsubscribe, e-mail: [hidden 
> email]
> For additional commands, e-mail: [hidden 
> email]
>
>
>
> --
>  View message @
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255336.html
> To unsubscribe from Continuous Delivery and Maven, click 
> here.
>
>
>


-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3255361.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Hi Brian

It just seems like the rev ID is really useful here for identifying
> reproducible builds without creating releases every time, does it fit with
> your ideas?  If so, a hypothetical repository manager plugin could be
> maintaining information about snapshot dependencies based on SCM rev ID,
> thus allowing for reproducibility without modifying Maven or existing
> snapshot mechanics.  Such a plugin might be able to generate a POM that has
> the extra rev ID metadata that the repo manager would recognize, allowing
> for existing SNAPSHOT-style identifiers to keep working for developer
> desktops (avoiding SCM thrash), but also providing reproducibility through
> synthetic POMs.


I think this is a great idea. If the pom for each snapshot contains enough
metadata about each of its upstream snapshots to be able to reproduce an
identical binary, we're good. Since I believe Maven already stores the md5
for each snapshot, this is verifiable.

Perhaps including in the POM for a snapshot the SCM URIs that were used to
create it and the version id (for SCMs that support atomic commits - I can't
see a happy way to do this for CVS, for example, without using tags) and the
same information nested for any of their upstream snapshots?

And then for extra points a command that could look at this pom, regenerate
the whole thing from scratch, and verify it against the md5 for the original
binary.

Jez.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3254548.html
Sent from the Maven - Users mailing list archive at Nabble.com.

Re: Continuous Delivery and Maven

[jhumble]

Hi Curtis

I'm the first to admit I'm no Maven expert.

So please let me just confirm. Let's assume I am working on project A which
depends on projects B and C. For the sake of argument, let's say that the
source code for A, B and C have separate roots in SVN, and are usually
checked out independently. The CI system builds A at version 50 using
snapshot dependencies on B and C, which are fetched from a central snapshot
repository.

Later, there are multiple updates to projects B and C which result in new
versions of them becoming available.

Say I now check out the source code to project A to version 50, because I
want to try and debug some issue that cropped up in a performance testing
environment, and I run a maven build. Will that use the latest versions of
the snapshots from the repo, or the versions that were originally fetched
when it was run on the CI system?

Can I even find out exactly which versions from svn the snapshots of B and C
came from that were used by the CI system to generate the original build of
A?

Thanks,

Jez.

On 7 November 2010 20:10, Yanko, Curtis [via Maven] <
ml-node+3254520-1036569885-143...@n5.nabble.com
> wrote:

> Very interesting discussion. With all due respect to Mr. Humble, and I
> am a big fan of CD, I am going to venture to say that you don't
> understand Maven very well. As a thought experiment, you are correct in
> saying that a build based on snapshots is not reproducible. As a more
> practical matter however, I feel it is.
>
> Dependencies come in two flavors, our and theirs (internal and 3rd
> party). If, all of *our* dependencies are SNAPSHOT (we're doing the
> developing) and all of *theirs* are 'versioned' then the build is in
> fact reproducible assuming you build everything from a particular repo
> version even with the default auto-update setting (in fact, it's
> required).
> 
>
> Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> Making IT Happen, one build at a time, 600 times a day
>
> -Original Message-
> From: jhumble [mailto:[hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=0>]
>
> Sent: Sunday, November 07, 2010 11:15 AM
> To: [hidden email] <http://user/SendEmail.jtp?type=node&node=3254520&i=1>
> Subject: RE: Continuous Delivery and Maven
>
>
> Hey Todd
>
> The whole point of continuous delivery is that every check-in creates a
> potential release candidate.
>
> When you're doing continuous deployment, you could be releasing multiple
> times a day, so you don't bother cutting branches or tagging or any of
> that stuff because of the overhead. I'd rather not get into the
> justification for this process on this thread - but I wrote a book on it
> if you're interested:
> http://www.amazon.com/gp/product/0321601912 and many other people have
> blogged about it.
>
> You're right that creating a concrete release for each commit could
> potentially use up a lot of space - but that's fine, you can just delete
> the older ones. What this *does* mean in turn though is that it is
> essential to be able to recreate any given build given the version in
> source control it came from, and this is where Maven falls down.
> Snapshots just aren't suitable because they aren't reproducible: what
> the snapshot looks like depends not only on what versions of the
> dependencies are available at the time the snapshot is created, but also
> what Maven's configuration and plug-ins happen to be at the time you run
> it (assuming Maven is configured to auto-update - the default). I can't
> revert back to a particular revision in version control, run maven, and
> be sure that the artifact it generates is identical to the one it
> created when the commit was initially triggered.
>
> Ideally what I'd like is for Maven to explicitly support the continuous
> delivery model and provide snapshots that are reproducible. Failing
> that, a guide to configuring Maven so that its binaries are reproducible
> (for example by switching off auto-update, and having sufficient
> metadata stored in pom files and Maven's artifacts repository to know
> what the state of each of the dependencies was at any given time.
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370<http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370?by-user=t>
> p3254090.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: [hidden 
> email]<http://user/SendEmail.jtp?type=node&node=3254520&

Re: Continuous Delivery and Maven

[jhumble]

On 7 November 2010 10:02, Brian Topping wrote:


> Does your book discuss ways to get around these issues?


No, it's a patterns / practices book so we don't go into a lot of detail on
the tools because that information tends to go out of date fast. We discuss
Maven a bit at the end of Chapter 13 ("Managing components and
dependencies"), pp375-378

The advice we give is this: "have your CI server produce canonical versions
of each dependency, using the build label as part of the artifact’s version
number, and store these in your organization’s central artifact repoistory.
You can then use Maven’s version quantifiers in your pom file to specify a
range of acceptable versions to use. If you really need to do some
exploratory work on your local machine, you can always edit your pom
definition to temporarily enable snapshots"

This isn't really ideal, partly for the reason you specify: this leads to a
lot of thrashing with the POM file. So I'm interested to see what could be
done to make Maven work better with the CD paradigm, for instance, stug23's
idea:

we have a so-called base-pom, which all projects inherit from, that locks
> down all of the Maven plugin versions so that the build is repeatable at a
> later time.


I think you identify the problem exactly right:

If I understand the paradigm, it's not that developers would want to reject
> the latest version of any dependencies, just that the snapshot builds should
> be reproducible


In fact, I'd go further and say we want to encourage people taking the
latest version of dependencies - otherwise you're not doing continuous
integration. But of course we want all of those dependencies in the artifact
repo, not built from scratch on the developer machines, because then it
means firstly wasted time and secondly that everybody is working with
potentially different binaries.

One possibility to get repeatable builds without filling up an artifacts
repository too fast could be to make Maven store the fully qualified pom
files in the artifacts repo and an md5 of the binary but not necessarily the
actual binary. I know artifacts repos already store some of this
information.

That way you could make sure sufficient metadata is publicly available such
that they can be reproduced, without using up loads of disk space. You could
also happily delete older binaries, safe in the knowledge that people could
reproduce them from the metadata in the artifacts repo.

As you can probably tell I'm no Maven maven, but I do want to help in
whatever way I can to make it work well with a continuous delivery process.

Jez.

-- 
Jez Humble
Co-author, *Continuous Delivery *
http://continuousdelivery.com/
http://jezhumble.net/

-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3254183.html
Sent from the Maven - Users mailing list archive at Nabble.com.

RE: Continuous Delivery and Maven

[jhumble]

Hey Todd

The whole point of continuous delivery is that every check-in creates a
potential release candidate.

When you're doing continuous deployment, you could be releasing multiple
times a day, so you don't bother cutting branches or tagging or any of that
stuff because of the overhead. I'd rather not get into the justification for
this process on this thread - but I wrote a book on it if you're interested:
http://www.amazon.com/gp/product/0321601912 and many other people have
blogged about it.

You're right that creating a concrete release for each commit could
potentially use up a lot of space - but that's fine, you can just delete the
older ones. What this *does* mean in turn though is that it is essential to
be able to recreate any given build given the version in source control it
came from, and this is where Maven falls down. Snapshots just aren't
suitable because they aren't reproducible: what the snapshot looks like
depends not only on what versions of the dependencies are available at the
time the snapshot is created, but also what Maven's configuration and
plug-ins happen to be at the time you run it (assuming Maven is configured
to auto-update - the default). I can't revert back to a particular revision
in version control, run maven, and be sure that the artifact it generates is
identical to the one it created when the commit was initially triggered.

Ideally what I'd like is for Maven to explicitly support the continuous
delivery model and provide snapshots that are reproducible. Failing that, a
guide to configuring Maven so that its binaries are reproducible (for
example by switching off auto-update, and having sufficient metadata stored
in pom files and Maven's artifacts repository to know what the state of each
of the dependencies was at any given time.
-- 
View this message in context: 
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3254090.html
Sent from the Maven - Users mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
For additional commands, e-mail: users-h...@maven.apache.org