Re: [Libreoffice-qa] [tdf-discuss] Intervention
Le 14/07/2014 09:10, jean-christophe manciot a écrit : > Hello everyone, > > I've been aware of the following situation for a long time in the "bug > report forum", but I thought it would improve over time. Since it > hasn't changed at all, I feel I have to blow the whistle on this. > > As I have already written in a specific thread, here's the situation: Could you point to that thread? I didn't see it on the QA list > > 1) Almost all bug reports "checkcers" don't even consider bug reports > when they can't get the original file(s); I don't think they don't consider it, it's just very difficult to reproduce a bug without the original condition of the document producing it. I don't know what amount of triage you're doing, but most of the time, as a triager having the original document let you look into it and try to understand how the user meet it. > > 2) The same people mark bug reports as "Resolved - Invalid / Not A > Bug" even when they are unable to confirm or deny them for whatever > reason: no original file, different environment than the reporter, ... could you give bug numbers please, so we (QA members) can have a look at them. > > 3) It is not always possible for the bug reporters to hand over their > original file(s), for many different reasons that are beyond the scope > of this thread (confidentiality, legal concerns and so on...). there is most of the time a way to remove the confidential data and if the bug is serious enough to provide the doc directly to developers. > > I believe it is always possible for the checkers to trust the > "reporters" and follow the "steps to reproduce" on their own material > first; if they can't do that, they probably should ask > themselves about their true motivations in this forum. I found what you say not fair for our QA members, they are doing their best to triage hundreds or reports each week. It is rare that > the reporters use tricky configurations with tricky files and so on. Each way of work is different, each configuration may have their own settings depending on the company or the work. > Moreover, in case these steps are unclear, the "reporters" are usually > available to clarify their point, otherwise they would not have spent > their time in the first place.*Only then* can some files be uploaded > to check some very uncommon settings or the report be marked as > "invalid" or whatever is the most appropriate. No, it's exactly the contrary, the more the bug is complete and reproducible with the document attached, the less the triager will spend time to reproduce and detail the bug. > > If the checkers continue to become more and more inflexible on this > "file issue", the risks are that some reporters will probably drop > their cases and their involvement in this forum; the related bugs will > remain within LO for a long time... Again, can you please point to bug numbers where you see bugs wrongly marked as invalid. The first work of a triager is to reproduce the bug to confirm it. If he can't confirm or reproduce it, it's because there is not enough information, or the specific document is missing. If his configuration is different (most of the time he has Linux and the bug is Windows or Mac), he requests for another member to triage the bug. Our irc channel is full of these requests, even the QA list. > > Is this what the checkers and LO community really need? I'm ccing the QA list, please follow up on this list if you want to discuss our triaging work. Kind regards Sophie ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [tdf-discuss] Intervention
Hi, just to quickly expand on that: On Mon, Jul 14, 2014 at 02:35:43PM +0200, Sophie wrote: > > I believe it is always possible for the checkers to trust the > > "reporters" and follow the "steps to reproduce" on their own material > > first; if they can't do that, they probably should ask > > themselves about their true motivations in this forum. > > I found what you say not fair for our QA members, they are doing their > best to triage hundreds or reports each week. > It is rare that > > the reporters use tricky configurations with tricky files and so on. > > Each way of work is different, each configuration may have their own > settings depending on the company or the work. Pragmatically, if the bug is not reproducable by a QA triager (that is: someone else than the reporter), it will also not be reproducable by a developer. Without that, the bug cant be fixed really (for a fix also cannot be verified). A bug report that is not fixable is by itself unfortunately not of much value to the project. As such, we need the reporter to find a way to create a confirmed reproduction scenario. In fact, this is the small contribution (by the reporter) that helps enabling a much bigger contribution (the fix, done by a developer). With development resource being limited (they always are), they will -- all other things being equal -- naturally and effectively be used on the most well-triaged issues. Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [tdf-discuss] Intervention
Hi! >From my experience asking for an example file is the best way to triage for following reasons: - saves time - you can download the attachment and check it in different builds right away - important with current backlog in Unconfirmed bugs - reproducible case - sometimes when you follow the STRs and create document from scratch the bug can be gone. Users' files can have their history - be created in different build, envs, corrupted etc. So asking for a file is a best way to receive verified test case. - involve the reporter - some people tend to use Bugzilla as file and forget system. Needinfo stats tell a story... Bug reports with attachments are more interesting than those without them. Some reporters do even screencasts or special STR graphics to help the triagers. IMHO there is no need to panic that most triagers ask for them. Overall I think this is a good policy and reporters should be educated how good bug report should look like. If a reporter cannot spend few minutes to attach a file or make a confidential one into a public document (by search and replace strings - if that makes bug still reproducible), then how can he demand a fix? This cannot be made without a reproducible test case. BTW: Mr Manciot is active in Wireshark Bugzilla, so should be accustomed that good bug report needs attachment. LO needs users' files as much as Wireshark example frame captures... Best regards. P.S. As for bugs closed as Invalid or Worksforme - there are defined QA documents which describe how this process should look like. See https://wiki.documentfoundation.org/QA/BugTriage or https://wiki.documentfoundation.org/QA/BugReport. Most triagers respect them, but those rules are, well, more guidance than a strict policy. LibreOffice is powered by a team of volunteers, every bug is confirmed (triaged) by human beings who mostly give their time for free. Some people see things from different perspective and don't like to "babysit" stagnant issues. -- View this message in context: http://nabble.documentfoundation.org/Re-tdf-discuss-Intervention-tp4115537p4115583.html Sent from the QA mailing list archive at Nabble.com. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [tdf-discuss] Intervention
Hi All, I have found that asking for a document is the best way to get closest to what the user is experiencing and what they are writing the bug for. If they report the bug on windows, i load up windows to confirm it and then also check if its on linux as well. Sometimes the steps to reproduce are easy enough to follow, but not every one of us are experts in the bugs we triage, so having an example file to begin the process of triaging saves quite alot of time. Users i've been dealing with have been quite happy to provide an example file, while a very few have asked that the file be kept confidential. Here is an example bug with steps to reproduce i triaged today [81292]. Problem description: I have a table first column alpha-numeric,crashes when sorting is ask. Steps to reproduce: 1. Load table, 2. select table 3. sort Current behavior: crash Expected behavior: alpha-numeric sorting >From this example, should i waste time that i could be spending triaging other bugs to create a table full of values in order to sort the table. It could be possible that some small feature within the table he is sorting is causing the crash, that i could never reproduce because i dont have his file. In the user's most recent comment, he states that if he deletes the text from the last column, it wont crash. No way i could reproduce such a thing if i created an example file myself. I just submitted a bug today [81351] that crashes calc from as early as 3.6, simply by undo-ing a sort. It is possible that this may not have happened with another file, so i submitted the one i was working on, in order to speed up triaging and hopefully fixing. We have ~1k bugs to still triage and the quicker we are able to triage a bug, the faster we can confirm/close it and move on to the next one. Just my two cents. ;) Regards, Jay Philips On 07/15/2014 01:48 AM, bfoman wrote: > Hi! > From my experience asking for an example file is the best way to triage for > following reasons: > - saves time - you can download the attachment and check it in different > builds right away - important with current backlog in Unconfirmed bugs > - reproducible case - sometimes when you follow the STRs and create document > from scratch the bug can be gone. > Users' files can have their history - be created in different build, envs, > corrupted etc. So asking for a file is a best way to receive verified test > case. > - involve the reporter - some people tend to use Bugzilla as file and forget > system. Needinfo stats tell a story... > Bug reports with attachments are more interesting than those without them. > Some reporters do even screencasts or special STR graphics to help the > triagers. IMHO there is no need to panic that most triagers ask for them. > Overall I think this is a good policy and reporters should be educated how > good bug report should look like. > If a reporter cannot spend few minutes to attach a file or make a > confidential one into a public document (by search and replace strings - if > that makes bug still reproducible), then how can he demand a fix? This > cannot be made without a reproducible test case. > BTW: Mr Manciot is active in Wireshark Bugzilla, so should be accustomed > that good bug report needs attachment. LO needs users' files as much as > Wireshark example frame captures... > Best regards. > P.S. > As for bugs closed as Invalid or Worksforme - there are defined QA documents > which describe how this process should look like. See > https://wiki.documentfoundation.org/QA/BugTriage or > https://wiki.documentfoundation.org/QA/BugReport. Most triagers respect > them, but those rules are, well, more guidance than a strict policy. > LibreOffice is powered by a team of volunteers, every bug is confirmed > (triaged) by human beings who mostly give their time for free. Some people > see things from different perspective and don't like to "babysit" stagnant > issues. > > > > > -- > View this message in context: > http://nabble.documentfoundation.org/Re-tdf-discuss-Intervention-tp4115537p4115583.html > Sent from the QA mailing list archive at Nabble.com. > ___ > List Name: Libreoffice-qa mailing list > Mail address: Libreoffice-qa@lists.freedesktop.org > Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa > Problems? > http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ > Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette > List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ > ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List arch
Re: [Libreoffice-qa] [tdf-discuss] Intervention
I also replied to this off list. We are consistent and that's good ;) Ultimately, we're not going to change our workflow. It's up to a user to report with clear steps and a simple test document - else it's just a waste of our time. Glad we all think alike :-D Best, Joel On 07/14/2014 05:38 PM, Jay Philips wrote: > Hi All, > > I have found that asking for a document is the best way to get closest > to what the user is experiencing and what they are writing the bug for. > If they report the bug on windows, i load up windows to confirm it and > then also check if its on linux as well. Sometimes the steps to > reproduce are easy enough to follow, but not every one of us are experts > in the bugs we triage, so having an example file to begin the process of > triaging saves quite alot of time. Users i've been dealing with have > been quite happy to provide an example file, while a very few have asked > that the file be kept confidential. Here is an example bug with steps to > reproduce i triaged today [81292]. > > > > Problem description: > I have a table first column alpha-numeric,crashes when sorting is ask. > Steps to reproduce: > 1. Load table, > 2. select table > 3. sort > > Current behavior: crash > > Expected behavior: alpha-numeric sorting > > > > From this example, should i waste time that i could be spending triaging > other bugs to create a table full of values in order to sort the table. > It could be possible that some small feature within the table he is > sorting is causing the crash, that i could never reproduce because i > dont have his file. In the user's most recent comment, he states that if > he deletes the text from the last column, it wont crash. No way i could > reproduce such a thing if i created an example file myself. > > I just submitted a bug today [81351] that crashes calc from as early as > 3.6, simply by undo-ing a sort. It is possible that this may not have > happened with another file, so i submitted the one i was working on, in > order to speed up triaging and hopefully fixing. We have ~1k bugs to > still triage and the quicker we are able to triage a bug, the faster we > can confirm/close it and move on to the next one. > > Just my two cents. ;) > > Regards, > Jay Philips > > On 07/15/2014 01:48 AM, bfoman wrote: >> Hi! >> From my experience asking for an example file is the best way to triage for >> following reasons: >> - saves time - you can download the attachment and check it in different >> builds right away - important with current backlog in Unconfirmed bugs >> - reproducible case - sometimes when you follow the STRs and create document >> from scratch the bug can be gone. >> Users' files can have their history - be created in different build, envs, >> corrupted etc. So asking for a file is a best way to receive verified test >> case. >> - involve the reporter - some people tend to use Bugzilla as file and forget >> system. Needinfo stats tell a story... >> Bug reports with attachments are more interesting than those without them. >> Some reporters do even screencasts or special STR graphics to help the >> triagers. IMHO there is no need to panic that most triagers ask for them. >> Overall I think this is a good policy and reporters should be educated how >> good bug report should look like. >> If a reporter cannot spend few minutes to attach a file or make a >> confidential one into a public document (by search and replace strings - if >> that makes bug still reproducible), then how can he demand a fix? This >> cannot be made without a reproducible test case. >> BTW: Mr Manciot is active in Wireshark Bugzilla, so should be accustomed >> that good bug report needs attachment. LO needs users' files as much as >> Wireshark example frame captures... >> Best regards. >> P.S. >> As for bugs closed as Invalid or Worksforme - there are defined QA documents >> which describe how this process should look like. See >> https://wiki.documentfoundation.org/QA/BugTriage or >> https://wiki.documentfoundation.org/QA/BugReport. Most triagers respect >> them, but those rules are, well, more guidance than a strict policy. >> LibreOffice is powered by a team of volunteers, every bug is confirmed >> (triaged) by human beings who mostly give their time for free. Some people >> see things from different perspective and don't like to "babysit" stagnant >> issues. >> >> >> >> >> -- >> View this message in context: >> http://nabble.documentfoundation.org/Re-tdf-discuss-Intervention-tp4115537p4115583.html >> Sent from the QA mailing list archive at Nabble.com. >> ___ >> List Name: Libreoffice-qa mailing list >> Mail address: Libreoffice-qa@lists.freedesktop.org >> Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa >> Problems? >> http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ >> Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette >>
