Re: [Math] Performance bug: Lessons to be drawn?
> > > > Are there lessons to be drawn from the problem discovered just after the > > recent release? Apart from pointing fingers, that is. :-} > > Most important point - don't do that (point fingers) :) > > > > I presume that people who voted for the release did as thorough a review as > > they could. > > And the problem is not primarily one of unit test coverage. Some use cases > > will always escape the scope of unit testing: It is not reasonable to extend > > the tests suite with all imaginable use cases. > > > > One thing that comes to mind would be to announce the release candidates on > > the "user" ML, gently requesting those users who intend to upgrade to test > > the JAR in as many of their applications as possible. ["Let them speak now > > or forever remain silent." ;-)] > > For backwards-compatible releases, that does not seem to be a huge burden on > > the people that use CM. > > > > What do you think? > > As I said in another post, I think the Tomcat community came up with > a pretty good way to deal with this, which was to cut releases and > then after the general user community had an opportunity to work > with them, assess their stability. See [1] and browse tomcat-dev > archives for an idea of how this works in practice. The basic idea > is that it is best to get code into the hands of users ASAP and > assess its stability after you have gotten some feedback, basically > acknowledging that you can't possibly anticipate all of the problems > that a new release might bring and enabling developers - and the > user community - to move *forward* to stable releases as we identify > and address problems in releases that do not get "stable" status. > > So in concrete terms, I suggest that we move to a "release early, > release often" model with releases designated alpha when they are > cut and subsequently voted as beta or stable. We haven't done this > is Commons before; but we have talked about it a few times over the > years. [...] Sure. Lately, the few times this option was raised, it was by me, and each time, I did not get the impression that it was warmly welcome: when I proposed to get a release every 6 months, or after a shorter time span (i.e. when some to-be-defined amount of work had been done), the discussion ended with something like "we'll try to get 2 releases per year"... There are plenty of arguments in favour of "release early, release often", and I didn't understand the reluctance to apply it here. Or did I just get the wrong impression? Can I really start a release process every three months? Of course, I agree with your above suggestion to work like they do in Tomcat. However, I'm not optimistic that the CM's users community will actually be as reactive... Regards, Gilles - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [Math] Performance bug: Lessons to be drawn?
On Mon, Dec 31, 2012 at 2:00 AM, Phil Steitz wrote: > On 12/30/12 4:01 PM, Gilles Sadowski wrote: > > Hello. > > > > Are there lessons to be drawn from the problem discovered just after the > > recent release? Apart from pointing fingers, that is. :-} > > Most important point - don't do that (point fingers) :) > > > > I presume that people who voted for the release did as thorough a review > as > > they could. > > And the problem is not primarily one of unit test coverage. Some use > cases > > will always escape the scope of unit testing: It is not reasonable to > extend > > the tests suite with all imaginable use cases. > > > > One thing that comes to mind would be to announce the release candidates > on > > the "user" ML, gently requesting those users who intend to upgrade to > test > > the JAR in as many of their applications as possible. ["Let them speak > now > > or forever remain silent." ;-)] > > For backwards-compatible releases, that does not seem to be a huge > burden on > > the people that use CM. > > > > What do you think? > > As I said in another post, I think the Tomcat community came up with > a pretty good way to deal with this, which was to cut releases and > then after the general user community had an opportunity to work > with them, assess their stability. See [1] and browse tomcat-dev > archives for an idea of how this works in practice. The basic idea > is that it is best to get code into the hands of users ASAP and > assess its stability after you have gotten some feedback, basically > acknowledging that you can't possibly anticipate all of the problems > that a new release might bring and enabling developers - and the > user community - to move *forward* to stable releases as we identify > and address problems in releases that do not get "stable" status. > > So in concrete terms, I suggest that we move to a "release early, > release often" model with releases designated alpha when they are > cut and subsequently voted as beta or stable. We haven't done this > is Commons before; but we have talked about it a few times over the > years. Some wisdom from markt, mturk or other tomcat devs would be > appreciated here :) > > Phil > > [1] http://tomcat.apache.org/whichversion.html > Hi, I totally agree with Phil here, for a library of considerable size as CM it is very difficult to assess the quality when following the standard commons release process (which mainly focuses on formal things). Even if there is 100% code coverage, there are still areas that may cause problems (performance / numerical stability for different data sets as the ones envisioned by the developers, I had several examples of this). In theory, users can grab the latest snapshots as they tend to be quite stable, and provide feedback, but this happens only very rarely in my experience. Maybe alpha / beta releases would give our user base more incentives to test the code with their applications / use cases? Thomas
Re: [Math] Performance bug: Lessons to be drawn?
Hi Gilles, Gilles Sadowski wrote: > Hello. > > Are there lessons to be drawn from the problem discovered just after the > recent release? Apart from pointing fingers, that is. :-} > > I presume that people who voted for the release did as thorough a review > as they could. > And the problem is not primarily one of unit test coverage. Some use cases > will always escape the scope of unit testing: It is not reasonable to > extend the tests suite with all imaginable use cases. > > One thing that comes to mind would be to announce the release candidates > on the "user" ML, gently requesting those users who intend to upgrade to > test the JAR in as many of their applications as possible. ["Let them > speak now or forever remain silent." ;-)] > For backwards-compatible releases, that does not seem to be a huge burden > on the people that use CM. > > What do you think? So what? Simply release 3.1.1. We as PMC have to rely on the active component committers that they got it right. This time something slipped through, shit happens. No-one can expect that most of the commons PMC can judge a release with regard to contents. But we try to ensure the formal rules. These rules ensure that we should be able to get out a binary compatible 3.1.1 very easy. Cheers, Jörg - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
Re: [Math] Performance bug: Lessons to be drawn?
On 12/30/12 4:01 PM, Gilles Sadowski wrote: > Hello. > > Are there lessons to be drawn from the problem discovered just after the > recent release? Apart from pointing fingers, that is. :-} Most important point - don't do that (point fingers) :) > > I presume that people who voted for the release did as thorough a review as > they could. > And the problem is not primarily one of unit test coverage. Some use cases > will always escape the scope of unit testing: It is not reasonable to extend > the tests suite with all imaginable use cases. > > One thing that comes to mind would be to announce the release candidates on > the "user" ML, gently requesting those users who intend to upgrade to test > the JAR in as many of their applications as possible. ["Let them speak now > or forever remain silent." ;-)] > For backwards-compatible releases, that does not seem to be a huge burden on > the people that use CM. > > What do you think? As I said in another post, I think the Tomcat community came up with a pretty good way to deal with this, which was to cut releases and then after the general user community had an opportunity to work with them, assess their stability. See [1] and browse tomcat-dev archives for an idea of how this works in practice. The basic idea is that it is best to get code into the hands of users ASAP and assess its stability after you have gotten some feedback, basically acknowledging that you can't possibly anticipate all of the problems that a new release might bring and enabling developers - and the user community - to move *forward* to stable releases as we identify and address problems in releases that do not get "stable" status. So in concrete terms, I suggest that we move to a "release early, release often" model with releases designated alpha when they are cut and subsequently voted as beta or stable. We haven't done this is Commons before; but we have talked about it a few times over the years. Some wisdom from markt, mturk or other tomcat devs would be appreciated here :) Phil [1] http://tomcat.apache.org/whichversion.html > > > Regards, > Gilles > > - > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > > - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
[Math] Performance bug: Lessons to be drawn?
Hello. Are there lessons to be drawn from the problem discovered just after the recent release? Apart from pointing fingers, that is. :-} I presume that people who voted for the release did as thorough a review as they could. And the problem is not primarily one of unit test coverage. Some use cases will always escape the scope of unit testing: It is not reasonable to extend the tests suite with all imaginable use cases. One thing that comes to mind would be to announce the release candidates on the "user" ML, gently requesting those users who intend to upgrade to test the JAR in as many of their applications as possible. ["Let them speak now or forever remain silent." ;-)] For backwards-compatible releases, that does not seem to be a huge burden on the people that use CM. What do you think? Regards, Gilles - To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org