On Mon, Dec 31, 2012 at 2:00 AM, Phil Steitz <phil.ste...@gmail.com> wrote:
> On 12/30/12 4:01 PM, Gilles Sadowski wrote: > > Hello. > > > > Are there lessons to be drawn from the problem discovered just after the > > recent release? Apart from pointing fingers, that is. :-} > > Most important point - don't do that (point fingers) :) > > > > I presume that people who voted for the release did as thorough a review > as > > they could. > > And the problem is not primarily one of unit test coverage. Some use > cases > > will always escape the scope of unit testing: It is not reasonable to > extend > > the tests suite with all imaginable use cases. > > > > One thing that comes to mind would be to announce the release candidates > on > > the "user" ML, gently requesting those users who intend to upgrade to > test > > the JAR in as many of their applications as possible. ["Let them speak > now > > or forever remain silent." ;-)] > > For backwards-compatible releases, that does not seem to be a huge > burden on > > the people that use CM. > > > > What do you think? > > As I said in another post, I think the Tomcat community came up with > a pretty good way to deal with this, which was to cut releases and > then after the general user community had an opportunity to work > with them, assess their stability. See [1] and browse tomcat-dev > archives for an idea of how this works in practice. The basic idea > is that it is best to get code into the hands of users ASAP and > assess its stability after you have gotten some feedback, basically > acknowledging that you can't possibly anticipate all of the problems > that a new release might bring and enabling developers - and the > user community - to move *forward* to stable releases as we identify > and address problems in releases that do not get "stable" status. > > So in concrete terms, I suggest that we move to a "release early, > release often" model with releases designated alpha when they are > cut and subsequently voted as beta or stable. We haven't done this > is Commons before; but we have talked about it a few times over the > years. Some wisdom from markt, mturk or other tomcat devs would be > appreciated here :) > > Phil > > [1] http://tomcat.apache.org/whichversion.html > Hi, I totally agree with Phil here, for a library of considerable size as CM it is very difficult to assess the quality when following the standard commons release process (which mainly focuses on formal things). Even if there is 100% code coverage, there are still areas that may cause problems (performance / numerical stability for different data sets as the ones envisioned by the developers, I had several examples of this). In theory, users can grab the latest snapshots as they tend to be quite stable, and provide feedback, but this happens only very rarely in my experience. Maybe alpha / beta releases would give our user base more incentives to test the code with their applications / use cases? Thomas
