Hi, We've now done five releases in the modern era. I think that in general the release process has worked well, but I do feel that there are some problems.
A number of people spent large amounts of time to get 3.0 as good as possible. This included work on the manual, basic functionality testing, going through the bug list, trying to fix obvious bugs and possibly disabling features if they were too buggy. There were of course problems, and maybe the release process took too long (five weeks from freeze to release), but I think that in general this was a nice release that we can be proud of. In my opinion, our latest release, 3.4, was very different. Some of the people who had been involved in the release process were busy with other things, and we kind of forgot about it. For a while it wasn't too clear if the freeze had already started, and we didn't do any RC builds. The result was a very short release process (one week). Quality-wise however, I think (although I don't really have proof, this is more of a general feeling) that this release was not very good, and that this is just the low point in a longer trend, i.e. releases are gradually getting worse. So what's been going on? I think there are two main factors involved: - We've been getting used to doing releases, and working on them is just not as exciting as it once was - More people use the releases, so bugs in relatively rarely used parts of the code don't get noticed immediately anymore Can we reverse the trend? I suspect that we can. The main thing that has been missing recently is testing of the soon-to-be release. This testing does not need to be very in-depth, often a quick check is enough to spot problems. It has to be done on every target though. My proposal is to involve our users here, and to do the following: - Starting from the day of the freeze, we provide RC builds. - We provide a feedback template that helps people figure out what needs testing, and to ensure that the resulting reports are reasonably easy to handle. - Someone collects these reports and consolidates them to flyspray tasks for the actual bugs. If we do this, having a week (or more, although I don't think more is needed) of RC time after the freeze makes sense again. If we have two weeks between the start of the freeze (and the start of the reporting season) and the release, I suspect that we can fix most of the obvious bugs in time, and if not actually know about them so we can mention them in the release notes. For the next release, I can volunteer for providing the RC builds, for proposing the feedback template, and for collecting and analysing the reports. Thoughts? Frank -- "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan
