Re: Fundamental flaw in bug reporting system
* Ian Jackson <[EMAIL PROTECTED]> [060713 22:03]:
> Matt Zimmerman writes ("Re: Fundamental flaw in bug reporting system"):
> > We want users who don't know how to write a good bug report to become users
> > who do, not get discouraged and not contribute at all.
>
> This idea is based on the misapprehension that we are short of bug
> reports,
Here opinions vary. As I'm mostly using production machines running
Debian stable, in my experience the largest problem is in my eyes
the lack of proper testing and filing of bug reports. (And thus hundreds
of annoying little bugs slipping through)
Of course it is not nice to have some bugs reported several dozen times,
but a majority of bugs goes unnoticed, and no writing of test-cases and
qa can ever replace a large group of users testing it with real data.
> and (I'm sorry to say) on the misapprehension that every user
> can be made into a good bug submitter.
If you don't like bug reports, you can still discard them afterwards.
Also note the time of more experienced people is more sparse than
of inexperienced people. Someone having no clue about what is happening
has more time to walk through several hops to report it. (Or he will
switch away totaly). But someone abe to diagnose the problem can
just work around it normaly and not help others improving their code
when they do not want his contribution.
Hochachtungsvoll,
Bernhard R. Link
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, Jul 13, 2006 at 08:44:57PM +0100, Ian Jackson wrote: > I don't think picking a package and comparing bug reports like for > like across two distributions is `anecdotal evidence'. Anecdotal > evidence is statements like `well I tried to submit a bug report and > was discouraged'. Yes, I stand corrected. The data you've given is not anecdotal, it's just too narrow to base any particular conclusions on. (IMO) > What is your opinion based on ? My opinion is that your opinion isn't thoroughly researched. This is partially due to the fact that you are asking the wrong questions. To repeat myself, the questions to ask are, how quickly are (real) bugs being reported, and how quickly are they getting fixed. I agree that intuitively, it seems that if you have more bad bug reports, then developers will waste more time dealing with them. But you need to balance that with the possibility that making it easier to submit bug reports may also cause problems to be reported more quickly. It doesn't really matter how much free time a developer has if he doesn't hear about a problem until 2 weeks after it's started affecting our users. > > The amount of noise in the system is really a secondary concern if it leads > > to bugs getting reported faster and fixed faster. > > And it is a primary concern if it leads to bugs being introduced more > often or fixed more slowly because the developers are too busy dealing > with bad bug reports. I agree. So why not find out if that is really happening, before drawing conclusions? There should be a way to do this empirically. --Adam -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, Jul 13, 2006 at 08:45:27PM +0100, Ian Jackson wrote:
> Matt Zimmerman writes ("Re: Fundamental flaw in bug reporting system"):
> > We want users who don't know how to write a good bug report to become users
> > who do, not get discouraged and not contribute at all.
>
> This idea is based on the misapprehension that we are short of bug
> reports, and (I'm sorry to say) on the misapprehension that every user
> can be made into a good bug submitter.
While I will readily admit that some users are probably not, I think that
any user who is willing to learn can become a good bug submitter.
Furthermore, it is frustrating to find people circulating horrible hacks as
workarounds for bugs which are straightforward to fix, simply because the
bugs don't get reported. There are classes of bugs which are much more
readily discovered and reported by less experienced users, due to
differences in work habits.
--
- mdz
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
Matt Zimmerman writes ("Re: Fundamental flaw in bug reporting system"):
> We want users who don't know how to write a good bug report to become users
> who do, not get discouraged and not contribute at all.
This idea is based on the misapprehension that we are short of bug
reports, and (I'm sorry to say) on the misapprehension that every user
can be made into a good bug submitter.
Ian.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
Adam McKenna writes ("Re: Fundamental flaw in bug reporting system"):
> On Tue, Jul 11, 2006 at 05:16:52PM +0100, Ian Jackson wrote:
> > It works because it acts as a filter. Ability and willingness to
> > write a good bug report are correlated pretty strongly with ability
> > and willingness to read and understand documentation on how to report
> > bugs.
>
> Your statements about correlations seem to be based on your opinions and
> anecdotal evidence, rather than data.
I don't think picking a package and comparing bug reports like for
like across two distributions is `anecdotal evidence'. Anecdotal
evidence is statements like `well I tried to submit a bug report and
was discouraged'.
Ie, what I'm pointing at is some data. It's not very well analysed
and you can complain about the implied methodology.
What is your opinion based on ?
> IMHO, there are two important criteria that should be used to evaluate the
> quality of the bug reporting system
>
> 1. Bugs get reported in a timely fashion
> 2. Bugs get fixed in a timely fashion
>
> The amount of noise in the system is really a secondary concern if it leads
> to bugs getting reported faster and fixed faster.
And it is a primary concern if it leads to bugs being introduced more
often or fixed more slowly because the developers are too busy dealing
with bad bug reports.
Ian.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Tue, 11 Jul 2006, Adam McKenna wrote: > The amount of noise in the system is really a secondary concern if it leads > to bugs getting reported faster and fixed faster. I can state that noise in the BTS is a hindrance to my work, and causes less bugs to be triaged and/or fixed by myself when I hit the noise. Maybe it is different for others. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Tue, Jul 11, 2006 at 05:16:52PM +0100, Ian Jackson wrote: > If you really think this, please go and look at the bug reports filed > against Ubuntu in the Launchpad bugtracker. For example, compare > http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=apt > with > https://launchpad.net/distros/ubuntu/+source/apt/+bugs >(scroll down to the bottom to see the newly-reported bugs) ... > It works because it acts as a filter. Ability and willingness to > write a good bug report are correlated pretty strongly with ability > and willingness to read and understand documentation on how to report > bugs. Your statements about correlations seem to be based on your opinions and anecdotal evidence, rather than data. IMHO, there are two important criteria that should be used to evaluate the quality of the bug reporting system 1. Bugs get reported in a timely fashion 2. Bugs get fixed in a timely fashion The amount of noise in the system is really a secondary concern if it leads to bugs getting reported faster and fixed faster. --Adam -- Adam McKenna <[EMAIL PROTECTED]> <[EMAIL PROTECTED]> -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Tue, 11 Jul 2006, Matt Zimmerman wrote: > We want users who don't know how to write a good bug report to become users > who do, not get discouraged and not contribute at all. Those who get discouraged so fast they won't even send an email directly to the maintainer (yes, users do figure out how to do so quite readly, I help users that contact me directly all the time) or to the mailinglists are not the kind of user that will learn how to write good bug reports in my experience. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Tue, Jul 11, 2006 at 05:16:52PM +0100, Ian Jackson wrote:
> Adam McKenna writes ("Re: Fundamental flaw in bug reporting system"):
> > Frustrating the user could lead to an even poorer quality bug report than
> > would have been given in the first place.
>
> It works because it acts as a filter. Ability and willingness to
> write a good bug report are correlated pretty strongly with ability
> and willingness to read and understand documentation on how to report
> bugs.
>
> The users who don't know how to write a good report, but do care about
> doing so, read the documentation and do file a good report. Those who
> don't know and don't care are discouraged and go elsewhere, to the
> benefit of the developers and of the software quality.
We want users who don't know how to write a good bug report to become users
who do, not get discouraged and not contribute at all.
--
- mdz
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
Adam McKenna writes ("Re: Fundamental flaw in bug reporting system"):
> I don't think this logically follows.. I don't see how "inaccessibility"
> of the BTS would necessarily result in better quality bug reports.
If you really think this, please go and look at the bug reports filed
against Ubuntu in the Launchpad bugtracker. For example, compare
http://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=apt
with
https://launchpad.net/distros/ubuntu/+source/apt/+bugs
(scroll down to the bottom to see the newly-reported bugs)
> Frustrating the user could lead to an even poorer quality bug report than
> would have been given in the first place.
It works because it acts as a filter. Ability and willingness to
write a good bug report are correlated pretty strongly with ability
and willingness to read and understand documentation on how to report
bugs.
The users who don't know how to write a good report, but do care about
doing so, read the documentation and do file a good report. Those who
don't know and don't care are discouraged and go elsewhere, to the
benefit of the developers and of the software quality.
Ian.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, Jun 29, 2006 at 11:30:33AM +0100, Ian Jackson wrote: > The slight inaccessibility in the bug reporting facilies is an > appropriate part of our approach to improving bug report quality. I don't think this logically follows.. I don't see how "inaccessibility" of the BTS would necessarily result in better quality bug reports. Frustrating the user could lead to an even poorer quality bug report than would have been given in the first place. --Adam -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, Aug 28, 2006 at 02:54:09PM -0700, Michael Wheatley wrote: > I would understand a steep learning curve but this is a catch 22 overhang. > > I am a complete newbie. The install went great and I have my command > line. Then I spend hours trying to find info on loading a GUI and the > closest I get is GNOME support telling me to "Click on the session > icon" but all I have is a command line. I decide that this could be > a documentation bug or feature request and I try to submit a bug > report but your system bounces it for poor info or structure. It > looks like it will take me hours to figure out how to properly format > a bug report so I will not try further. > > The fundamental problem is that a newbie lacks the understanding of > the system or the patience that is needed to submit a bug > report. Any bugs that stop newbies from using and learning the > system go unreported and the system retains any flaws that exclude new > users. > > My system is going back into storage for now. > > (I also had trouble with the GNOME bug system. I could not find a > classification for my request/bug report and the "Unclassified" > option just told me all reports are classified. I could of course > not find a classification for that bug either. Just so you know that > either you are not alone or it is just my imagination.) > Hi Michael Wheatley, As someone who reads both debian-user (the debian user mailing list) and this one, I have come accross many users who as 'why was X and/or gnome/kde not installed?' or 'where's X' Those users have had the luck to find the debian-user forum becuase of google and a net connection. Obviously, not all users have these. The only thing of which I would think would be some README/FAQ file or maybe a few files that might be in the /root or / that would get folks going in the right direction. I also thought of having these in a debian doc package and/or adding a tasksel task for something like 'debian beginner FAQ'. But then this assumes those folks can find/use this.You may want to take this to debian-user or debian-project (where there are discussions of interest to the project as a whole). just my 2 euro-centimes Kev - -- | .''`. == Debian GNU/Linux == | my web site: | | : :' : The Universal | debian.home.pipeline.com | | `. `' Operating System| go to counter.li.org and | | `-http://www.debian.org/ |be counted! #238656 | | my keysever: pgp.mit.edu | my NPO: cfsg.org | -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) iD8DBQFEpKOrv8UcC1qRZVMRAruOAKCKhejQEmbd+zbfbAAySwNlJvr4tACeK0tS 2OCI7rm/qaiHJfBukN3oEic= =idf9 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, Jun 29, 2006 at 11:30:33AM +0100, Ian Jackson wrote:
> Michael Wheatley writes ("Fundamental flaw in bug reporting system"):
> > I would understand a steep learning curve but this is a catch 22 overhang.
> >
> > I am a complete newbie. The install went great and I have my command
> > line. Then I spend hours trying to find info on loading a GUI and the
> > closest I get is GNOME support telling me to "Click on the session
> > icon" but all I have is a command line. I decide that this could be
> > a documentation bug or feature request and I try to submit a bug
> > report but your system bounces it for poor info or structure. It
> > looks like it will take me hours to figure out how to properly format
> > a bug report so I will not try further.
>
> I'm going to get flamed to a crisp here, but:
>
> I think that the difficulty of submitting a Debian bug report via the
> BTS (which is after all a fairly minor challenge to anyone who can
> read and understand documentation) provides a very useful barrier
> against poor-quality bug reports.
>
> The purpose of a bug report is to help the developers improve the
> software. There are times when a particular user lacks the capability
> (whether background knowledge or aptitude) to write a bug report that
> can serve that purpose. It is best for those users to seek assistance
> from their friends and colleagues, rather than to file a bug report
> (and perhaps, if there is a bug that could usefully be reported, their
> friends will help them do that, or do it for them).
>
> The slight inaccessibility in the bug reporting facilies is an
> appropriate part of our approach to improving bug report quality.
You raise a fair point, and it's one that I believed myself, but on the
other hand there's a recent example of an X bug that was pretty important
to fix that wasn't reported until a friend/XSF-member bugged me about it on
IRC. Tons of people in #debian had been complaining about this bug, yet no
one had submitted it.
On the other hand, I got a zillion duplicate bugs about things like manpage
sections.
If people can't report trivial bugs that need to be fixed, then it makes it
hard for us to do what we need to do. I'm not so sure that lowering the
barriers is the right answer though, but it needs to be kept in mind.
- David Nusinow
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Mon, Aug 28, 2006 at 02:54:09PM -0700, Michael Wheatley wrote: > The fundamental problem is that a newbie lacks the understanding of > the system or the patience that is needed to submit a bug > report. I hate to break it to you, but becoming an advanced user does not make you more patient :) Cheers, -- Bill, (who once claimed to be "very patient") Imagine a large red swirl here. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, Jun 29, 2006 at 11:30:33AM +0100, Ian Jackson wrote: > The slight inaccessibility in the bug reporting facilies is an > appropriate part of our approach to improving bug report quality. Right, but is "improving bug report quality" the aim of the bug reporting? I believe it rather is "improving software quality". A newbiesh bug report as the one that started this thread is not very helpful, but it definitely has a point: a potential user of our operating system was unable to start the GNOME environment she was looking for. I might agree if you are suggesting we need different channels for that kind of bug reports (and I believe we have some). Cheers. -- Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ If there's any real truth it's that the entire multidimensional infinity of the Universe is almost certainly being run by a bunch of maniacs. -!- signature.asc Description: Digital signature
Re: Fundamental flaw in bug reporting system
"Michael Wheatley" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] I would understand a steep learning curve but this is a catch 22 overhang. I am a complete newbie. The install went great and I have my command line. Then I spend hours trying to find info on loading a GUI and the closest I get is GNOME support telling me to "Click on the session icon" but all I have is a command line. I decide that this could be a documentation bug or feature request and I try to submit a bug report but your system bounces it for poor info or structure. It looks like it will take me hours to figure out how to properly format a bug report so I will not try further. Sorry to hear that. The reccomended way to submit a bug report is by using the 'reportbug' program. That program will guide you through the needed steps. The fundamental problem is that a newbie lacks the understanding of the system or the patience that is needed to submit a bug report. Any bugs that stop newbies from using and learning the system go unreported and the system retains any flaws that exclude new users. My system is going back into storage for now. My guess is that you simply do not have a gui installed. Login using the username 'root' and the password you gave to the root account. Run this: "aptitude update" When that is finished run this: "aptitude install xorg" Once that is finished you have installed a GUI, but X11 by itself is barely usuable. You almost certainly want a Desktop Environment. To install one run: "aptitude install gnome" or "aptitude install kde" The choice is yours. KDE is more heavyweight but is probably slightly more freindly to users coming from Windows or Mac OS X. When you finally get back to the command line run "reboot". If all went well, when it is finished rebooting, you will have a graphical login screen, which is then followed by a GUI. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, 29 Jun 2006, Evan Prodromou wrote:
> On Thu, 2006-29-06 at 11:30 +0100, Ian Jackson wrote:
> > I think that the difficulty of submitting a Debian bug report via the
> > BTS (which is after all a fairly minor challenge to anyone who can
> > read and understand documentation) provides a very useful barrier
> > against poor-quality bug reports.
Agreed. It would be a very different story if we didn't have other forums
for people to ask for help. But we do have them, and they are far more
appropriate for newbie users than the BTS system.
That said, if there is a problem with how newbies are being handled in these
forums when they have doubts and issues to report, then we must address THAT
issue and fix it.
> It does not seem to be the best strategy for software quality to ignore
> the existence of bugs just because the reporting user didn't know the
They are not ignored, but the optimal path to submit those issues is not
directly to the BTS if you are still not sure how to use it as it is right
now.
A very new user ("newbie") is better served by talking to other users and
developers in the mailing-lists, irc(?), etc. AFTER it gets clear a bug
report is in order, he (or someone else with more experience) can submit it.
Nowhere the bug is being ignored. But preliminary help from peers, and
screening/more data gathering by those peers done in mailinglists before a
bug is sent to the BTS is fair and useful for everybody, *including* the
newbie.
Response time for an workaround, for example, is far better in mailinglists
most of the time.
> When users bother to report bugs, it shows respect, loyalty and
> commitment to our project and our software. They think it's worth going
[...]
> Debian has a huge pool of such loyal and responsive users. Squandering
> that resource is a bad idea.
The active developer taskforce is a far more constrained resource in Debian,
so it has to be preserved too. Keeping stuff in the BTS as high-quality as
possible helps this a great deal.
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thu, 2006-29-06 at 11:30 +0100, Ian Jackson wrote: > I think that the difficulty of submitting a Debian bug report via the > BTS (which is after all a fairly minor challenge to anyone who can > read and understand documentation) provides a very useful barrier > against poor-quality bug reports. It does not seem to be the best strategy for software quality to ignore the existence of bugs just because the reporting user didn't know the exact cha-cha-cha steps for making a perfect report. The occasional false report is a brief annoyance for the developer; an unreported bug is a problem for hundreds or thousands of users. When users bother to report bugs, it shows respect, loyalty and commitment to our project and our software. They think it's worth going through the trouble of filing a bug report rather than throwing our software away and using a proprietary or Free alternative. Debian has a huge pool of such loyal and responsive users. Squandering that resource is a bad idea. ~Evan -- Evan Prodromou <[EMAIL PROTECTED]> The Debian Project (http://www.debian.org/) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Fundamental flaw in bug reporting system
On Thursday 29 June 2006 12:30, Ian Jackson wrote: > The purpose of a bug report is to help the developers improve the > software. There are times when a particular user lacks the capability > (whether background knowledge or aptitude) to write a bug report that > can serve that purpose. It is best for those users to seek assistance > from their friends and colleagues, rather than to file a bug report > (and perhaps, if there is a bug that could usefully be reported, their > friends will help them do that, or do it for them). Or on Debian user mailing lists or IRC channels of course; either on the original issue or on how to properly file a bug report. pgp0DG07k8NRF.pgp Description: PGP signature
Re: Fundamental flaw in bug reporting system
Michael Wheatley writes ("Fundamental flaw in bug reporting system"):
> I would understand a steep learning curve but this is a catch 22 overhang.
>
> I am a complete newbie. The install went great and I have my command
> line. Then I spend hours trying to find info on loading a GUI and the
> closest I get is GNOME support telling me to "Click on the session
> icon" but all I have is a command line. I decide that this could be
> a documentation bug or feature request and I try to submit a bug
> report but your system bounces it for poor info or structure. It
> looks like it will take me hours to figure out how to properly format
> a bug report so I will not try further.
I'm going to get flamed to a crisp here, but:
I think that the difficulty of submitting a Debian bug report via the
BTS (which is after all a fairly minor challenge to anyone who can
read and understand documentation) provides a very useful barrier
against poor-quality bug reports.
The purpose of a bug report is to help the developers improve the
software. There are times when a particular user lacks the capability
(whether background knowledge or aptitude) to write a bug report that
can serve that purpose. It is best for those users to seek assistance
from their friends and colleagues, rather than to file a bug report
(and perhaps, if there is a bug that could usefully be reported, their
friends will help them do that, or do it for them).
The slight inaccessibility in the bug reporting facilies is an
appropriate part of our approach to improving bug report quality.
Ian.
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
