on 29/1/02 6:20 PM, Scott Raney at [EMAIL PROTECTED] wrote: > While we're on the subject, I'd like to put in my 2 cents about why > generating such release notes is a *bad* idea, and why people who are > asking for such things are doing a disservice to the development team > and to themselves and other users of the product. > > First of all, we've generated a README file for MetaCard with each new > release going on 10 years now that lists major new features and > incompatibilities introduced.
Scott, thanks for mentioning the MetaCard README; as a Revolution user, I hadn't clocked that - I'll download the current MetaCard distribution and check it out. Perhaps the Revolution team could incorporate this document (if necessary with some search + replace to call it 'engine release notes') in the Rev distribution. But I'm guessing that the Geometry Manager is actually a Rev feature, not an MC one? Also from your description (and your other remarks) it sounds like it doesn't describe actual bugs. So I don't think it would satisfy all my requirements. > Secondly, it takes time to generate these things, time that would be > much better spent on development or producing other types of > documentation. For example, would you prefer having a detailed list > of all the bugs reported against the geometry manager along with a > response to each of them, or would you rather just have it work? In > many cases, that's the choice you're forcing. I can't accept that. Your 'choice' is straw-man - there are levels of detail between none and "a detailed list of all the bugs reported against the geometry manager along with a response to each of them". I'd be happy with "some bugs fixed in the Geometry Manager which produced incorrect positions/sizes when the stack was resized under certain conditions", which is the similar to the level of detail in the one example of release notes I'm aware of from RunRev, released on request to accompany Rev beta 4. Nor do I believe any development team is just fixing things and making changes without making notes; nor that they are simply responding to bug reports scrawled on the back of envelopes. Surely both Rev and MC teams must be using some kind of bug tracking database; and must be noting their changes. If I didn't believe that, I'd be really worried. My main current project in Rev is a tool being used by only three people, which I'm developing alone. I record release notes in a field on the help stack - I don't imagine any of my users read these, but adding a sentence for each substantial feature I've added or bug removed takes very little time, and is the minimum form for me to record what I've done. (I also maintain a snagging database on the web for users (and me) to record bugs found or feature requests.) If a developer of an ongoing project is recording changes at all, adding a one-liner for the release notes is not going to 'force the choice' of not having things fixed. If they're not recording changes at all, they damn well should be - and this kind of release note seems like the minimum level, even if it it's only for their own use. > (And as an aside, if you really *don't* know whether it has already been > "substantially improved" as your comment above indicates, you're obviously not > beta testing, which you should be if you care at all about the quality of the > product). I _totally_ reject that. I've so far bought, on behalf of my (small) company, one MetaCard full license (before Rev appeared), three Rev Professional licenses, and three Rev Standard licenses, to support a total of five developers. Admittedly many of these were at various special offer rates, but I've still spent something over $3000 on MC/Rev in the last twelve months. Guess what? I have to spend some time working for _my_ clients, not my suppliers, to earn some money to pay for that outlay. I work far too many hours as it is; and over the last year a lot of hours have gone in non-chargeable time researching Rev issues. When Rev was in beta before 1.0 I put in a great deal of time tracking down issues, simplifying repro cases and making careful, detailed bug reports (and Kevin acknowledged at the time that I had become one of Rev's "best testers"). I justified this to my colleagues as an investment in a tool I believed would become valuable to us, and because I wanted to get certain features to work. And fortunately (?) at that time, work was 'slow' enough (a relative term) so that I could squeeze some more hours out. During this period I spent a quantity of time wrestling with the Geometry manager; and then some time exterminating it from my stacks. I did try to track down the details of its misbehaviour but it was getting sticky, so rather than spend more time on something Rev team might already have known about (that's the value of an accessible bug tracking list, which is a separate discussion) I sent a note which said that I thought it was substantially screwed, and if they thought otherwise I'd put some more work into reporting it. They didn't ask for more info, I left it. Now, your suggestion is that with each new release I should be diligently putting all the Geometry Manager settings back into my stacks to see if it messes up again, and then removing the data if it does, on the off-chance that it has been worked on but that the improvements haven't been mentioned to customers. Did I mention I'm pushed for time? That's _exactly_ why we need release notes; if I saw a mention in release notes that said it had been improved in a particular version, then I know it's worth investing the time to testing whether it had been fixed. But if not, it's not going to happpen. I can't afford to waste my time like that. > Finally, and most importantly, anyone who *doesn't* report a bug > because they read about it in a README, or worse, doesn't actually > verify that a bug they have reported is fixed in a new release > regardless of whether it's listed in some README, is making a huge > mistake, and is almost certainly a causal factor in *decreasing* the > overall quality of the product. We, like all software developers I > know, live or die by the quantity and quality of bug reports that we > get. Not to keep repeating myself, but putting in a good quality bug report takes time. If I can't judge whether that time is wasted, I'm less likely to spend it. The result is reduced quantity and/or quality of bug reports for Rev/MC, not increased. > In order to properly balance the effort spent on fixing bugs to > adding new features, we need to know how often people are running into > problems in particular areas and how serious the consequences are > (i.e., are there workarounds, does a problem just affect development > or also the quality of apps you release, etc.) Anything that > decreases either of these (other than maybe pulling off the miracle of > releasing a defect-free product ;-) is a serious threat to our very > survival, and I for one believe that releasing a complete list of all > bug reports and responses will decrease both. (Again I'm not arguing for a complete list of bug reports and responses.) You charge a serious amount of money for a serious product, a tool which is used by people to do real work, to develop products of their own which affect their reputation (if you want to disagree with that description, then for the purposes of this argument imagine restricting release notes to 'Professional' users). Your product is a tool; I consider release notes one of the features of the tool, that professional users need in order to make the best use of it. Your argument seems to consider your product in isolation, forgetting about the needs of your customers. I have torn my hair out debugging an impossible problem in CodeWarrior, before I turned to the read-me and discovered that there was a known bug in the code-optimiser which could cause my problem (that's one of the things that taught me to make diligent use of read-me's). Conversely some years ago testing found a problem with our installer, shortly before releasing a CD-ROM product. When we finally tracked it down to the combination of a particular (MS) mouse with a particular version of Windows, we contacted the installer company and found that they knew about it but had chosen not to tell their users (by definition, professional users) - in read-me or FAQs, until they asked. I immediately cancelled our licenses with this company and switched to their competitor, who we have used ever since. Not only did we waste our time trying to track down something that they knew about; we found it by chance in testing (that hardware wasn't in our config lab) and could easily have pressed 20,000 CDs with this bug, and without a read-me that would have told users how to avoid it, leading to a substantial potential increase in our support calls, which completely undermines the profitability of that kind of product. CodeWarrior, BareBones, iCab are among the tools that I use, with very different scales of development teams, that include read-me's of this sort. Their products seem to improve substantially with each release. They don't seem to have been forced to cancel all development work in order to concentrate on generating release notes; nor do they seem to have lost customers. The survival of the products and companies has not noticeably been affected. > PS: I also consider even public reporting of bugs to be a bad idea, > but I'm in some disagreement with the folks at RR about this. I agree that the arguments here are different, and I think there are more pros and cons to balance on this issue than (in my opinion) on the release notes question. > The way I see it, public reporting a) decreases everyones perception of the > quality of the product, most importantly that of potential customers True; but weigh against the satisfaction or otherwise, with product and company, reported by actual customers (or users of the starter kit); and experienced by users of products built with your product, which have bugs that the developer didn't know about so couldn't work around, or which have less features than they might because the developer's time was wasted. And notice that in this internet age, if you don't support a forum for bugs, users will discuss them anyway in some other public forum. Arguably the perception for potential users will be worse because they see the same bug reported endlessly than reading it once in a list. > and b) makes it impossible to use reports to prioritize development > because you can't tell the difference between an obscure problem that > only one person is running into from problems that *everyone* is > running into. A lot of your discussion seems to be skewed by this particular requirement, of knowing how many users are encountering a particular bug. But in reality you must already have to weigh up such questions for bugs you discover yourself, or features you consider that haven't been requested yet; and you must have a number of means of evaluating priorities in this case, from knowledge of the platforms your users are developing on and targeting, the aspects of the product that are getting most use (animation facilites v database, for example). There's also nothing to stop people contacting you or a mailing list or news group to say how much they want a fix for a particular bug, given that they know it's been reported. The number of people reporting the bug is also a flawed measure, I'd imagine; some kinds of bugs may be more likely to be found by the kind of people who'd be less likely to report it. So while I accept this has a value, I question whether it should have an overriding one. > Public reporting may, from time to time, save you the 5 > minutes it takes to report a bug by eliminating duplication When I make a serious report of a serious bug, it takes me a _lot_ longer than 5 minutes. That's why I resent wasting that investment. That's why I'd take the time to check the release notes to see if it's known; and why if I can't tell if my effort would be wasted, I may not make the effort. > (but I'd argue that even keeping up with the reports of bugs in QT when you're > not even using that ends up costing you more time in the long run). If you have to read every message on a mailing list to find out what bugs are being reported, that's right; which is why a list of known bugs is a big timesaver for the poor customers. MetaCard, and you personally, have an enviable reputation for customer support and responsiveness; which I think the RunRev team are living up to. I agree there are questions worth asking about running public bug lists, but I really think that proper release notes with each test release are an important feature of the Revolution package. Ben Rubinstein | Email: [EMAIL PROTECTED] Cognitive Applications Ltd | Phone: +44 (0)1273-821600 http://www.cogapp.com | Fax : +44 (0)1273-728866 _______________________________________________ improve-revolution mailing list [EMAIL PROTECTED] http://lists.runrev.com/mailman/listinfo/improve-revolution
