I note that there are many more categories than of software defects than just coding bugs and usability problems.
If you look at the Beizer/Vinter taxonomy, there's a large category of requirements defects.
Commercially, requirements defects, particularly changes, have the largest impact on schedules since
fixing them may require re-designing and re-writing extensive parts of the product.
From the point of view of where in the development cycle they originate, usability problems can originate at multiple places
in the development life cycle and can have multiple causes. Consider the case of a problem found during a usability review of users being provided with
no way to delete a shared document from their workspace. This defect could have arisen during the requirements stage, because the requirements analyst
failed to include this capability as a requirement.It could also have occurred during the software design stage because the designer failed to
include the delete command in the software design or it could have occurred in the coding stage because the developer failed to code the delete command. About the only
thing that is distinctive about a usability problem is the technique used to discover it!
In my organization, all defects, regardless of how they are discovered, are logged in the anomoly tracking system (we're forbidden to call it a defect tracking system).
Every entry is assigned a severity by the person doing the entry. The anomolies are periodically reviewed and prioritized for action. We try hard to rank and prioritize the item
based on user impact, rather than on ease of devleoper repair. This works well for single point usability defects, such as the missing
delete command problem given above. It even works fairly well for problems that map onto a single part of the system, e.g. "interface does not give good feedback on user's current position in the menu structure." Where it works less well is if the defect is more global and/or requires changes to multiple parts of the product at once.
Ruven Brooks
| Effie Law <[EMAIL PROTECTED]>
Sent by: [EMAIL PROTECTED] 12/05/2005 09:30 AM
|
To: Sebastian Jekutsch <[EMAIL PROTECTED]> cc: discuss@ppig.org Subject: Re: PPIG discuss: Typology of software bug/usability problem fixes |
Hi Sebastian,
Thank you for your inputs.
Yes, there are a number of classic as well as more recent taxonomies of software analomies/bugs/defects in the field of software engineering, as cited by you and Bjorn. In the field of HCI, several research studies on developing taxonomies of usability problems (UPs) have been conducted in the recent years, e.g.,
Andre, T.S., Hartson, H.R.., Belz, S.M., & McCreary, F.A. (2001). The user action framework: a reliable foundation for usability engineering support tools.
International Journal of Human-Computer Studies, 54, 107-136.
Hvannberg, E.T., & Law, E. L-C. (2003). The Classification of Usability Problems (CUP) Scheme. In Proc. INTERACT 2003, Sept 5-9, Zurich, Switzerland
As you pointed out, UPs normally are not registered as bugs. There have been debates on the pros and cons of this practice.
Wilson, C. & Coyne, K. P. (2001): The whiteboard: Tracking usability issues: to bug or not to bug? Interactions, 8 (3) p. 15-19.
One obvious drawback of excluding UPs in a bug tracking system is that they tend to be overlooked or ignored, thus remaining unattended and unfixed.
Anyway, my concern is how the distinction between bugs and UPs, if any, can be formalized, i.e. defining a set of criteria for each of them; meeting certain criteria will be classified as bugs, otherwise UPs. Besides, I doubt there are direct links between types of bugs and types of fixes or between types of UPs and types of fixes. There can be various ways to fix a bug or a UP, even if the type per se implies the underlying cause(s). Could the existing taxonomies of software bugs be "translated" into taxonomies of fixes? I am sceptical of such a possibility. Sure, I may be wrong.
Yes, it'll be very interesting to study programmer errors systematically. How many % in general can software bugs be attributed to programmers' mistakes?
Cheers,
Effie
Hi Effie,
I'm not quite sure whether there's a difference for you between "types of fixes" and "types of defects = bugs". The distinction doesn't seem to be obvious.
Bjorn's list on "types of defects" contains some classics but misses the work of Basili, e.g.
* Victor R. Basili, Barry T. Perricone: Software errors and complexity: an empirical investigation, Communications of the ACM, Volume 27, Issue 1 (January 1984), Pages: 42 - 52.
* Michael Fredericks, Victor Basili. Using Defect Tracking and Analysis to Improve Software Quality, DACS State-of-the-Art Report SP0700-98-4000, Data & Analysis Center for Software, 14 November 1998 (http://www.dacs.dtic.mil/techs/defect/)
This may only be interesting to computer scientists, though.
Beizer's extensive taxonomy can be found online here: http://inet.uni2.dk/~vinter/bugtaxst.doc
Interesting (and including a comparative section on defect taxonomies) but maybe out of focus are more general papers on using and designing defect classifications:
* Brian Marick. A survey of software fault surveys. Technical Report UIUCDCS-R-90-1651, University of Illinois at Urbana Champaign, December 1990.
* Bernd Freimut. Developing and Using Defect Classification Schemes. Report Fraunhofer IESE-072.01/E, Sep. 1, 2001. (http://docserver.fhg.de/iese/2001/reports/072.pdf)
I'd like to add that in many cases a taxonomy of programmer (i.e. human) errors are much more interesting from a research point of view compared to studying the results, i.e. the "bugs".
I don't know of any discussion about how general programming errors relate to user interface design errors. I guess there are only few because interface defects are mainly due to wrong guessing and missing knowledge at the UI design level. Usability issues aren't often registered as software defects, since in most cases the software still seem to work - from a pure functional point of view. Of course, that's an ignorant point of view.
Sebastian
--
Software Engineering Working Group
Institut für Informatik
Freie Universitaet Berlin
Takustr. 9, 14195 Berlin, Germany
+49 30 838 75239
http://www.inf.fu-berlin.de/~jekutsch
-----Original Message-----
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
On Behalf Of Bjorn Reese
Sent: Thursday, November 24, 2005 6:56 PM
To: discuss@ppig.org
Subject: Re: PPIG discuss: Typology of software bug/usability
problem fixes
Effie Law wrote:
I'll much appreciate if you could give me some good
references for the
following topics:
(1) the distinction between software bugs and usability problems
(2) typology of fixes for software bugs
There are a whole range of software bug taxonomies. Some examples are:
IEEE Std 1044-1993: Standard Classification for Software Anomalies,
http://standards.ieee.org/reading/ieee/std_public/description/
se/1044-1993_desc.html
Ram Chillarege et al., Orthogonal Defect Classification,
http://www.chillarege.com/odc/odc-papers.html
Boris Beizer, Software Testing Techniques, Van Nostrand
Reinhold Co.,
1990.
Cem Kaner & Jack Falk & Hung Quoc Nguyen, Testing Computer
Software,
Wiley, 1999.
Matt Telles & Yuan Hsieh, The Science of Debugging, Coriolis, 2001.
Robert Metzger, Debugging by Thinking, Elsevier, 2004.
(3) typology of fixes for usability problems
----------------------------------------------------------------------
PPIG Discuss List (discuss@ppig.org)
Discuss admin: http://limitlessmail.net/mailman/listinfo/discuss
Announce admin: http://limitlessmail.net/mailman/listinfo/announce
PPIG Discuss archive: http://www.mail-archive.com/discuss%40ppig.org/
