RE: [GUMP] Build Failure - Turbine
Philosophical discussions are always worth a couple of cents of my own :-) On Wed, 23 May 2001, Tim Vernum wrote: > > From: Sam Ruby (23 May 2001 10:02 AM) > > > Jason van Zyl wrote: > > > > > > What is going on here? I fixed this code almost two days ago. > > > > http://jakarta.apache.org/builds/gump/2001-05-21/cvs_index.html > > http://jakarta.apache.org/builds/gump/2001-05-22/cvs_index.html > > > > Compare this to the "good old days": > > > > http://jakarta.apache.org/builds/gump/2001-04-15/cvs_index.html > > > > Daedalus performance (or lack thereof) has basically gotten > > to the point > > where programs like Gump are great in theory, but impractical in > > practice... > > (Disclaimer: I'm not a committer on any project, so my opinion is > objective, but slightly under-informed, and without authority) > > In general, I believe that automated build/test processes are *very* > important, an do work. > We've got some teams using CruiseControl here, and it is doing well. > The reasons I think that gump registers so many failures are: > > 1) The environment in which gump runs is not identical to the > environment the developers run (OS, jdk, jars, etc), and I'm > not sure if it is even well defined. >Now, given that these are supposed to be cross platform > java projects, that shouldn't be an issue, but it has been, > and will continue to be. >Unless each developer replicates the Gump environment on his/her > workstation, they will not find all problems. However doing so > will only fix problems for that environment, so in some ways > leaving it slightly undefined is helpful to keep portability. Most of the failure issues here are based on slow CVS updates, not on environmental differences. But the important point is a little more subtle - the "write once run anywhere" Java mantra is not just a marketing slogan. We as Java developers should be designing to minimize environmental dependencies. GUMP focuses on one kind of dependency (cross project dependency at compile time), which is important, but it's not the only one (for example, code to build file paths that uses hardcoded '/' or '\' delimiters). However, GUMP has served a valuable role in just raising awareness of these issues, above and beyond it's usefulness as a tool. > > 2) Developers can't/don't run the tests before checkin. >In our situation, if you checkin something that breaks the tests, > you're an idiot. >I don't think that happens in apache, for a few reasons. What does "in apache" mean? There is no one single project here -- each Jakarta project (for example) consists of its own constellation of developers. Sure, there is overlap, but nobody at all is involved in everything. >The primary one being that most people are doing this in the spare > time, and a channeling in patches from numerous sources. Forcing > full rebuilds with unit tests would make that channel so small that > work would grind to a virtual stand-still. > Part of the issue is a (good) increase in expectations. It used to be that, if you didn't break the build on your own project, you were doing good. (For amusement, check out the warnings about what to do before checking code in on many of the project README files). Now, if we don't run the appropriate unit test suites, other developers on the same project start frowning at you. GUMP (along with attitudes like yours about "test first before committing") are becoming much more widespread. The other reality is that Moore's law tends to make the "full rebuilds" issue moot. I just upgraded my home PC, and as a result I can build Tomcat (modest-sized but still complex), from scratch, in less than 20 seconds. There is absolutely no excuse not to do that kind of thing on your own project any more. > 3) Developers can't/don't test other projects. >This is particularly relevant to ant, but also to some of the XML > and (soon) commons modules. >A number of build failures have been due to changes in ant. >Ant should continue to be free to change things as needed to make > ant better, and that will continue to break projects that rely > on specific ant features/bugs. >Unless the developers are going to rebuild all the dependant > projects everytime they make a change, then this will keep > happening. > Having to rebuild "foreign" projects is not something that I should really have to worry about. Moreover, I don't know the code of that project well enough to test it anyway. I want to just trust the developers of the projects I depend on. GUMP has raised awareness of the negative impacts that API changes have on other projects (be careful what methods you make public, because people will use them :-). Next step is to be more picky about functionality changes that don't involve API changes a compiler will catch for you. > 4) Gump (I think) only runs once a day. >This means that there's up t
Re: [GUMP] Build Failure - Turbine
Sam Ruby wrote: > > Geir Magnusson Jr. wrote: > > > > There is nothing (other than the sloth of Daedalus) which prevents us > > from adding any 'test' we want - Gump is running the Velocity testbed, > > for example, which is more than interface-related, but functional as > > well. > > That test suite takes 58 seconds to run. I think we can afford that. In > any case, it runs on my machine, not on Daedalus. No - what I meant was adding to the gump cycle time by adding more CVS activity to the mix. I figured that the CVS is the bottleneck - you can always parallelize the gumping once the source is there, although I suspect you won't need to :) > > That's a matter of resources for (4) and time and social engineering for > > (5), I think. With an all volunteer group, I think it has to be worked > > into the culture. While you could force the issue by making it a part > > of being a Jakarta project, I don't think we want to do that. > > This is happening. Six months ago several key projects were in perpetual > alpha, and there was very little code sharing. And Gump was just a > concept. > > Six months later, look at how things have changed. The mere thought that > somebody might not have addressed a problem 48 hours after it was detected > is being raised as an issue... > > Gump and I are happy. It's never clear to me that they are distinct entities ;) geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ "still climbing up to the shoulders..." - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [GUMP] Build Failure - Turbine
> > 3) Developers can't/don't test other projects. > >This is particularly relevant to ant, but also to some of the XML > > and (soon) commons modules. > >A number of build failures have been due to changes in ant. > >Ant should continue to be free to change things as needed to make > > ant better, and that will continue to break projects that rely > > on specific ant features/bugs. > >Unless the developers are going to rebuild all the dependant > > projects everytime they make a change, then this will keep > > happening. > > > > Interesting topic : like any other released package/project/tool with > users, does Ant ensure that it is backwards compatible with itself, at > least on major version boundaries? > > (I think it should be...) > I think the Ant team take backward compatability very seriously. Nevertheless, sometimes a change is introduced, which, although unintended, does cause compatability issues. We track Gump closely to catch these unintended problems and try to resolve them as quickly as practical. Sometimes we make a change which will flush out problems in build files. Is that a backward compatability issue? The recent copying of non-existent files is an example. A number of builds broke because Ant tightened up one of its checks. What level of backward compatibility are we looking for. Should we preserve bugs so that nobody ever has to change a build file. I'd be interested in viewpoints there. Sometimes, even deprecating things in Ant causes complaints :-) As Gump is using CVS snapshots we should expect Gump to fail - this is its benefit. We should be happy when it fails :-) Conor - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [GUMP] Build Failure - Turbine
Geir Magnusson Jr. wrote: > > There is nothing (other than the sloth of Daedalus) which prevents us > from adding any 'test' we want - Gump is running the Velocity testbed, > for example, which is more than interface-related, but functional as > well. That test suite takes 58 seconds to run. I think we can afford that. In any case, it runs on my machine, not on Daedalus. > I once thought it important to setup Gump tests that specify a specific > version set (i.e. the release versions of all dependencies), but I > realize that's pointless. Do the test once, and repeating can add no > new information. ;-) > That's a matter of resources for (4) and time and social engineering for > (5), I think. With an all volunteer group, I think it has to be worked > into the culture. While you could force the issue by making it a part > of being a Jakarta project, I don't think we want to do that. This is happening. Six months ago several key projects were in perpetual alpha, and there was very little code sharing. And Gump was just a concept. Six months later, look at how things have changed. The mere thought that somebody might not have addressed a problem 48 hours after it was detected is being raised as an issue... Gump and I are happy. ;-) - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [GUMP] Build Failure - Turbine
Tim Vernum wrote: > > In general, I believe that automated build/test processes are *very* > important, an do work. > We've got some teams using CruiseControl here, and it is doing well. > The reasons I think that gump registers so many failures are: > > 1) The environment in which gump runs is not identical to the > environment the developers run (OS, jdk, jars, etc), and I'm > not sure if it is even well defined. >Now, given that these are supposed to be cross platform > java projects, that shouldn't be an issue, but it has been, > and will continue to be. When has it been? If an OS issue is found, Gump did its job. The 'cross HEAD CVS', which is how i think of Gump, does something that developers don't do, but should have done for them - which is early detection of interface-ish problems. There is nothing (other than the sloth of Daedalus) which prevents us from adding any 'test' we want - Gump is running the Velocity testbed, for example, which is more than interface-related, but functional as well. I once thought it important to setup Gump tests that specify a specific version set (i.e. the release versions of all dependencies), but I realize that's pointless. Do the test once, and repeating can add no new information. >Unless each developer replicates the Gump environment on his/her > workstation, they will not find all problems. However doing so > will only fix problems for that environment, so in some ways > leaving it slightly undefined is helpful to keep portability. Yes. > 2) Developers can't/don't run the tests before checkin. >In our situation, if you checkin something that breaks the tests, > you're an idiot. >I don't think that happens in apache, for a few reasons. >The primary one being that most people are doing this in the spare > time, and a channeling in patches from numerous sources. Forcing > full rebuilds with unit tests would make that channel so small that > work would grind to a virtual stand-still. Not sure I buy this - Velocity has a unit test suite that has saved us (me) in countless instances, so I know if a change has altered the gross functional envelope that we have been able to define. > > 3) Developers can't/don't test other projects. >This is particularly relevant to ant, but also to some of the XML > and (soon) commons modules. >A number of build failures have been due to changes in ant. >Ant should continue to be free to change things as needed to make > ant better, and that will continue to break projects that rely > on specific ant features/bugs. >Unless the developers are going to rebuild all the dependant > projects everytime they make a change, then this will keep > happening. > Interesting topic : like any other released package/project/tool with users, does Ant ensure that it is backwards compatible with itself, at least on major version boundaries? (I think it should be...) > [SNIP] > > 5) Attitude. >Given the length of time that some projects remain broken, I guess >some projects don't particularly care too much. Which is a pity. >(Granted some projects believe they've fixed it, and that gump is >misbehaving, so you can't blame them for that). > I think if it gets repositioned to fill such a role (or maybe it > already is, and I'm misinformed) then solving (4) will be useful. > Ultimately though, (5) is the kicker. That's a matter of resources for (4) and time and social engineering for (5), I think. With an all volunteer group, I think it has to be worked into the culture. While you could force the issue by making it a part of being a Jakarta project, I don't think we want to do that. geir -- Geir Magnusson Jr. [EMAIL PROTECTED] System and Software Consulting Developing for the web? See http://jakarta.apache.org/velocity/ "still climbing up to the shoulders..." - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [GUMP] Build Failure - Turbine
Resending because in my haste, I left out a few crucial words...added in ALL CAPS. Tim Vernum wrote: > > 1) The environment in which gump runs is not identical to the > environment the developers run (OS, jdk, jars, etc), and I'm > not sure if it is even well defined. It is well defined, and you can even run it yourself. Loosely, it can be described as "the latest CVS of everything possible." > 2) Developers can't/don't run the tests before checkin. >In our situation, if you checkin something that breaks the tests, >you're an idiot. This doesn't apply here...however your next point does. > 3) Developers can't/don't test other projects. >This is particularly relevant to ant, but also to some of the XML >and (soon) commons modules. >Unless the developers are going to rebuild all the dependant >projects everytime they make a change, then this will keep >happening. In practice, it isn't practical for turbine developers to test every change to every dependency that they have the instant that product makes a change. In practice, it isn't practical for velocity developers to test every change they make against every product or potential product which may use their code. That's why Gump is there - as a valuable (though not infallible) cross check. Note: I do fully intend to keep up with the developments of xml-commons as they become available. > 4) Gump (I think) only runs once a day. >This means that there's up to 24hrs before you know that you broke >something. >CruiseControl (generally) runs every 15 minutes, and rebuilds if >the project has had a new checkin. This way you know quite quickly >that you broke something. Given the current performace of the Apache server, it has taken up to twelve hours to get the updates to the latest projects - and that is only after killing off updates that take more than a half an hour! Last month I had days where I could do a COMPLETE UPDATE OF EVERY PROJECT in as little at 35 minutes! > 5) Attitude. >Given the length of time that some projects remain broken, I guess >some projects don't particularly care too much. Which is a pity. >(Granted some projects believe they've fixed it, and that gump is >misbehaving, so you can't blame them for that). This is grossly unfair, and the real reason why I took the time to respond to this note. JVZ promptly addressed the root problem (I have verified the change on my own machine). Unfortunately, this change is not reflected in Gump because for two days straight, I have been unable to successfully do an cvs update for the projects in question in the time allotted. And if I can't successfully update a single project IN THIRTY MINUTES, no tool which runs every 15 minutes would have a chance... - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [GUMP] Build Failure - Turbine
Tim Vernum wrote: > > 1) The environment in which gump runs is not identical to the > environment the developers run (OS, jdk, jars, etc), and I'm > not sure if it is even well defined. It is well defined, and you can even run it yourself. Loosely, it can be described as "the latest CVS of everything possible." > 2) Developers can't/don't run the tests before checkin. >In our situation, if you checkin something that breaks the tests, >you're an idiot. This doesn't apply here...however your next point does. > 3) Developers can't/don't test other projects. >This is particularly relevant to ant, but also to some of the XML >and (soon) commons modules. >Unless the developers are going to rebuild all the dependant >projects everytime they make a change, then this will keep >happening. In practice, it isn't practical for turbine developers to test every change to every dependency that they have the instant that product makes a change. In practice, it isn't practical for velocity developers to test every change they make against every product or potential product which may use their code. That's why Gump is there - as a valuable (though not infallible) cross check. Note: I do fully intend to keep up with the developments of xml-commons as they become available. > 4) Gump (I think) only runs once a day. >This means that there's up to 24hrs before you know that you broke >something. >CruiseControl (generally) runs every 15 minutes, and rebuilds if >the project has had a new checkin. This way you know quite quickly >that you broke something. Given the current performace of the Apache server, it has taken up to twelve hours to get the updates to the latest projects - and that is only after killing off updates that take more than a half an hour! Last month I had days where I could update every project in as little at 35 minutes! > 5) Attitude. >Given the length of time that some projects remain broken, I guess >some projects don't particularly care too much. Which is a pity. >(Granted some projects believe they've fixed it, and that gump is >misbehaving, so you can't blame them for that). This is grossly unfair, and the real reason why I took the time to respond to this note. JVZ promptly addressed the root problem (I have verified the change on my own machine). Unfortunately, this change is not reflected in Gump because for two days straight, I have been unable to successfully do an cvs update for the projects in question in the time allotted. And if I can't successfully update a single project, no tool which runs every 15 minutes would have a chance... - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: [GUMP] Build Failure - Turbine
From: Sam Ruby (23 May 2001 10:02 AM) > Jason van Zyl wrote: > > > > What is going on here? I fixed this code almost two days ago. > > http://jakarta.apache.org/builds/gump/2001-05-21/cvs_index.html > http://jakarta.apache.org/builds/gump/2001-05-22/cvs_index.html > > Compare this to the "good old days": > > http://jakarta.apache.org/builds/gump/2001-04-15/cvs_index.html > > Daedalus performance (or lack thereof) has basically gotten > to the point > where programs like Gump are great in theory, but impractical in > practice... (Disclaimer: I'm not a committer on any project, so my opinion is objective, but slightly under-informed, and without authority) In general, I believe that automated build/test processes are *very* important, an do work. We've got some teams using CruiseControl here, and it is doing well. The reasons I think that gump registers so many failures are: 1) The environment in which gump runs is not identical to the environment the developers run (OS, jdk, jars, etc), and I'm not sure if it is even well defined. Now, given that these are supposed to be cross platform java projects, that shouldn't be an issue, but it has been, and will continue to be. Unless each developer replicates the Gump environment on his/her workstation, they will not find all problems. However doing so will only fix problems for that environment, so in some ways leaving it slightly undefined is helpful to keep portability. 2) Developers can't/don't run the tests before checkin. In our situation, if you checkin something that breaks the tests, you're an idiot. I don't think that happens in apache, for a few reasons. The primary one being that most people are doing this in the spare time, and a channeling in patches from numerous sources. Forcing full rebuilds with unit tests would make that channel so small that work would grind to a virtual stand-still. 3) Developers can't/don't test other projects. This is particularly relevant to ant, but also to some of the XML and (soon) commons modules. A number of build failures have been due to changes in ant. Ant should continue to be free to change things as needed to make ant better, and that will continue to break projects that rely on specific ant features/bugs. Unless the developers are going to rebuild all the dependant projects everytime they make a change, then this will keep happening. 4) Gump (I think) only runs once a day. This means that there's up to 24hrs before you know that you broke something. CruiseControl (generally) runs every 15 minutes, and rebuilds if the project has had a new checkin. This way you know quite quickly that you broke something. Potentially, you could have gump work in a system where if a project fails a build, it moves to nightly builds only, but onces it passes a nightly build it moves to a build every 3(?) hours, until it fails again. That way, you're not going to keep getting "build failed" messages all days, waiting for someone to fix it, but you will know straight away (alomost) when a change breaks a project. 5) Attitude. Given the length of time that some projects remain broken, I guess some projects don't particularly care too much. Which is a pity. (Granted some projects believe they've fixed it, and that gump is misbehaving, so you can't blame them for that). Anyway, the conclusion I can see is that for reasons 1-3, gump is actually replacing build/testing on developers machines, and is generally being the method by which people are alerted that they broke something, rather than them discovering it themselves. I don't think that this is a problem, but it will mean that gump is going to keep failing. I think if it gets repositioned to fill such a role (or maybe it already is, and I'm misinformed) then solving (4) will be useful. Ultimately though, (5) is the kicker. JM2C. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [GUMP] Build Failure - Turbine
Jason van Zyl wrote: > > What is going on here? I fixed this code almost two days ago. http://jakarta.apache.org/builds/gump/2001-05-21/cvs_index.html http://jakarta.apache.org/builds/gump/2001-05-22/cvs_index.html Compare this to the "good old days": http://jakarta.apache.org/builds/gump/2001-04-15/cvs_index.html Daedalus performance (or lack thereof) has basically gotten to the point where programs like Gump are great in theory, but impractical in practice... - Sam Ruby - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
