Re: [Libreoffice] QA-hints ? -> question for devs !
On Sun, Jun 19, 2011 at 3:45 PM, Cor Nouws wrote: > Hi *, Is there a script available, or a combination of git-commands, or ... that > is used to extract certain information for the summaries? > If so, I could use/adapt that to get information on certain weeks, > branches, ... ? > If you have a git repository then you can get info on that repo that is as fresh as your last pull. The lo-commit-stat script [Petr Mladek] in /bin is capable of slicing out whatever you want. The top-dir and --log-suffix arguments are required. >From your local repo root $> cd bin $> lo-commit-stat --help # gives you the help Use the trailing git-args --since (or --after) and --before (or --until) to pick out a range of times: a last argument of --after="2011-05-31" gives bugs/bugnumbers/commits after that date to the present. Use the --bugs arg to get just the commits with associated issue numbers - the default is all commits. So, in any /bin directory, the command $> lo-commit-stat ../ --bugs --log-suffix='test' --after="2011-05-31" gives the list of the commits made so far this month that have an issue # in their summary as a log file in /bin named: bugfixes--test.log Again, the list i only as recent as your latest git pull . Hope that helps, LeMoyne > Cheers, > Cor > ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi *, Cor Nouws wrote (19-06-11 11:51) And thanks to you and others for pointing to the current summaries as a good starting point for that, anyway as far as I am concerned. I understand the limitations now too. Since there is much on the route in our QA-process/work, we can see if there shows up a natural, not too complicated, possibility to enhance it. Is there a script available, or a combination of git-commands, or ... that is used to extract certain information for the summaries? If so, I could use/adapt that to get information on certain weeks, branches, ... ? Cheers, Cor -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi John, LeMoyne Castle wrote (15-06-11 13:01) On Wed, Jun 15, 2011 at 1:02 AM, Cor Nouws mailto:oo...@nouenoff.nl>> wrote: ... What I am more looking for, is sometimes the hints related to (some of) the vast amount of code clean-ups/improvements, that possibly have influence on area A or B. (In the commit-logs for master) Maybe maybe it is possible for the involved devs, when they think it is possibly relevant, to add a few words to the description so that it is indeed able to read that in the summaries. ... Sounds reasonable? It is quite reasonable to try to get the dev work result to mesh with qa testing. Sadly, git requests that the short commit message be <41 characters. Personally, I struggle with that at each commit ;-)... Git does allow more lines in the commit header but those usually describe changes to the code, are often technical and some of the comments about the removed/replaced code do *not* belong on the wiki. Thanks for this explanation. In the wiki of the future, the summary pages' bug numbers will link to the actual bug reports (at least for fdo) so that the summary indirectly provides the intended functional changes in LibreOffice. The bug Also with the present list it is not too difficult to go to the related issue. reports often contain steps to reproduce the bug that make it clear where to test the fix. Perhaps the bug-fix section of the summary could eventually include the bug summary (short desc) next to the link as well. What I am not looking for, is exact steps to reproduce / verify. When I know that some work has been done with function/area X, it gives the opportunity to work a bit around that area, according to my own habits and knowledge of the suite. The addition of a bug number to the short commit message is the most efficient way for the developers to pass testing info to qa. Thanks for raising and clarifying the issue of qa's need for clues about how to find and test the latest changes in any of master, 3.4 or the release branches, And thanks to you and others for pointing to the current summaries as a good starting point for that, anyway as far as I am concerned. I understand the limitations now too. Since there is much on the route in our QA-process/work, we can see if there shows up a natural, not too complicated, possibility to enhance it. Kind regards, Cor -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
On Wed, Jun 15, 2011 at 1:02 AM, Cor Nouws wrote: > Hi John, ... ... > What I am more looking for, is sometimes the hints related to (some of) the > vast amount of code clean-ups/improvements, that possibly have influence on > area A or B. (In the commit-logs for master) > > Maybe maybe it is possible for the involved devs, when they think it is > possibly relevant, to add a few words to the description so that it is > indeed able to read that in the summaries. > > ... > Sounds reasonable? > It is quite reasonable to try to get the dev work result to mesh with qa testing. Sadly, git requests that the short commit message be <41 characters. Personally, I struggle with that at each commit ;-)... Git does allow more lines in the commit header but those usually describe changes to the code, are often technical and some of the comments about the removed/replaced code do *not* belong on the wiki. In the wiki of the future, the summary pages' bug numbers will link to the actual bug reports (at least for fdo) so that the summary indirectly provides the intended functional changes in LibreOffice. The bug reports often contain steps to reproduce the bug that make it clear where to test the fix. Perhaps the bug-fix section of the summary could eventually include the bug summary (short desc) next to the link as well. The addition of a bug number to the short commit message is the most efficient way for the developers to pass testing info to qa. Thanks for raising and clarifying the issue of qa's need for clues about how to find and test the latest changes in any of master, 3.4 or the release branches, LeMoyne ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Cor Nouws píše v St 15. 06. 2011 v 09:08 +0200: > Petr Mladek wrote (13-06-11 11:47) :-( > > You might find some ideas at http://wiki.documentfoundation.org/QA > > I know that, thanks. In my experience (many years OOo) working with the > product random, is a very valuable good way of testing too. sure; I think that it is even must to do when testing new features. > And then sometimes a little hint for example to try working around e.g. > Impress objects, or cursor movement along line (breaks) .. things like that. Hmm, developers could do hints what should get tested. Unfortunately, they sometimes does not know about all consequences. If they know, they would not add bugs ;-) Well, I am not sure if it is good idea to ask developers to provide such information at this stage. We do not enough QA force, so most of the mentioned areas would not be tested. It would just give developers a false feeling that their change would be throughout tested. They would be less careful with their changes. It would take them extra time and we would end with more bugs. > See my reply to John for some more precise explanation of what I was > thinking about. I think thank you could do some testing based on the week commit summary as suggested by John. You could ask him to improve the lo-commit-stat script to filter the commits for you. You could watch for a key words, e.g. implemented, reworked, ... You check for the size of the changes. Well, even one-line change might break many things... Also, you might watch http://wiki.documentfoundation.org/ReleaseNotes/3.5 and monitor new features. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi Yifan, Yifan Jiang wrote (13-06-11 12:47) Great thanks for the message! Sorry I overlooked your mail at first glance, No need to apologise. As you can see, I'm not always fast too ;-) our regression testing is organized here: https://tcm.documentfoundation.org/ Here is the instruction how to contribute on running test on it: http://wiki.documentfoundation.org/Litmus Yes, good to see that growing. I already made a start with translation of that page to Dutch. I am probably going to deliver some important new cases or coverage areas to the mailing list for review/discuss this week, please keep tuned and let me know if you have any ideas about this or about QA. Also please do not hesitate to let us know if any questions. We are looking forward to you :) Thanks, I 'll make sure that I won't forget you :-) And see my reply to John for some more precise explanation of what I was thinking about in this special case. Kind regards, -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi Petr, Petr Mladek wrote (13-06-11 11:47) Quite regular I have the feeling that at least me would be helped a bit by doing QA if there is some hint-list.. It sounds great! We are still looking for more QA people. In the current situation, I do not even make time to file all the bugs that I come along :-\ You might find some ideas at http://wiki.documentfoundation.org/QA I know that, thanks. In my experience (many years OOo) working with the product random, is a very valuable good way of testing too. And then sometimes a little hint for example to try working around e.g. Impress objects, or cursor movement along line (breaks) .. things like that. See my reply to John for some more precise explanation of what I was thinking about. Kind regards, -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi John, John LeMoyne Castle wrote (12-06-11 00:29) I think the Weekly Development Summary might do what u want. Not exactly, but maybe it is closest to. It tends to slide by unnoticed because no one replies to it ( Well there are exceptions ;-) ) as they are all plowing straight ahead. Here is an easy way to find it... [...] + calc + fix for Insert sheets in a protected spreadsheet document (fdo#37772) [Markus Mohrhard] + fixed selection by arrow keys around merged cells. (fdo#34214) [Kohei Yoshida] [...] Indeed lots of useful info. What I am more looking for, is sometimes the hints related to (some of) the vast amount of code clean-ups/improvements, that possibly have influence on area A or B. (In the commit-logs for master) Maybe maybe it is possible for the involved devs, when they think it is possibly relevant, to add a few words to the description so that it is indeed able to read that in de summaries. On the other hand, being honest, in general I only have/make little time to look at those... Hope that helps... It does. Thanks for the hint. So in summary, my request is best picked up if devs, when they think it is relevant, to add a few words to the description of the patch, indicating which user actions might be involved/affected. Sounds reasonable? Cheers, -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi Jean-Baptiste, Thanks for the nice ideas and suggestions! My comments as below. Hi All, This discussion is probably leading the next step we will adjust test case management (Litmus) structure of Libreoffice, ideas and experience sharing are welcome :) And I will summarize where we will go finally. On Mon, Jun 13, 2011 at 01:54:46PM +0200, Jean-Baptiste Faure wrote: > Le 13/06/2011 12:47, Yifan Jiang a écrit : > Hi Yifan, > > I think we should do some simplifications in Litmus. Yes! we need to adjust it to a more explicit way :) > In fact I do not understand well how branches are supposed to work in > Litmus. When you manage testcases and testgroups, you can see that there > is no one in all branches but only in branch 3.3.1 (in Litmus meaning). You are right, the branch information is from an relatively old test case set (3.3.1). What I did for current runs is simply branch from an old test run without touching test cases. So currently we have a simple set of testcases reused everywhere. This actually needs to be improved. > On the other hand when you manage the testruns you can see that each one > contains the three testgroups defined in branch 3.3.1 (EN, DE and FR > testgroups). Ye, a bit confusing :) > It seems to me that it would be more clear and easy to manage if we > defined things as follows: > - branches corresponding to x.y versions of LibreOffice: 3.3, 3.4, 3.5, > and so on. > - the branch x.y+1 would be defined by cloning the x.y branch and next > adding new testcases needed by QA of the new functions implemented in > the x.y+1 version. My thinking is to: 1. have a relatively stable regression test case set which cover most important function areas such as installation, launch, file open, save etc. 2. have a changing feature cases (new functions) with the evolution of Libreoffice builds. These cases can be various among different Libreoffice build. And we can start testing features from beta phase. > - we manage RC and bugfix versions (x.y.1, x.y.2, etc.) only by creating > dedicated testruns: there is no reason to change the testcases set when > we pass from version x.y.0 to rc x.y.1 rc1 to bugfix x.y.1 to ... etc. > So there is no reason to have a new branch for a new rc or bugfix > - we need to define the buildID for each testrun, which does not seem to > be the case at this moment. Great idea! Actually we may not really want to have big change of regression test cases set even between big versions as x.y and x.y+n. A problem is if it is worth to repeatedly run regression cases in diffrent rc builds unders the same big version? My initial thinking about this is to define a rc test run for each minor version build, QA could run the tests always on the corresponding latest rc build they have. This is based on the assumptions: 1. code change between rc release would be safe 2. currently we have not enough resource to repeatedly run all regression tests on each rc build. But I guess it is not a big problem since when can always clone between rc runs, the matter is how to define the run name. How about let us use 'rc' only at very beginning and see if they are going well. Then we can expand the test dedicating to specific rc builds if needed. > According to that, we would have: > - 2 branches 3.3 and 3.4 > - for the branch 3.3 : > + one testrun 3.3.3-rc1 > + one testrun 3.3.3-rc2 > - for the branch 3.4 : > + one testrun 3.4.0 > + one testrun 3.4.1-rc1 > + one testrun 3.4.1-rc2 > + etc. > + one testrun 3.4.2-rc1 > + etc. > - then a branch 3.5 > + etc. Here is my thinking based on your proposal: Take into consideration libreoffice build 3.3 and 3.4: Branches: - 3.3 Regression test branch (assuming it contains stable regression cases written from scratch) + Test run - 3.3.0-rc regression test + Test run - 3.3.1-rc regression test + Test run - 3.3.2-rc regression test ... - 3.3 New Feature test branch + Test run - 3.3 feature test ( new features in 3.3 build ) - 3.4 Regression test branch (containing stable regression cases, could be cloned from 3.3 and did some improvement) + Test run - 3.4.0-rc regression test + Test run - 3.4.1-rc regression test + Test run - 3.4.2-rc regression test ... - 3.4 New Feature test branch + Test run - 3.4 feature test ( new features in 3.4 build ) > I think we need to agree on the general organization before > that the system has become too big and too widely used. > I can do these modifications if we have a general agreement. > As that assume to remove several branches and to rebuild testruns, it > would be prudent to not be several to do changes at the same time. That'd be great and thanks for your offer of help! Meanwhile we wo
Re: [Libreoffice] QA-hints ? -> question for devs !
Le 13/06/2011 12:47, Yifan Jiang a écrit : > On Mon, Jun 13, 2011 at 11:47:18AM +0200, Petr Mladek wrote: >> Also Sophie and Yi Fan are trying to organize testing of beta and rc >> builds. There is used Litmus server to coordinate the work between the >> various testers. I am sure that they are looking for more testers and >> test case writers. > > Hi Cor, > > Great thanks for the message! Sorry I overlooked your mail at first glance, > our regression testing is organized here: > > https://tcm.documentfoundation.org/ > > Here is the instruction how to contribute on running test on it: > > http://wiki.documentfoundation.org/Litmus > > I am probably going to deliver some important new cases or coverage areas to > the mailing list for review/discuss this week, please keep tuned and let me > know if you have any ideas about this or about QA. > > Also please do not hesitate to let us know if any questions. We are looking > forward to you :) > > Best wishes, > Yifan > Hi Yifan, I think we should do some simplifications in Litmus. In fact I do not understand well how branches are supposed to work in Litmus. When you manage testcases and testgroups, you can see that there is no one in all branches but only in branch 3.3.1 (in Litmus meaning). On the other hand when you manage the testruns you can see that each one contains the three testgroups defined in branch 3.3.1 (EN, DE and FR testgroups). It seems to me that it would be more clear and easy to manage if we defined things as follows: - branches corresponding to x.y versions of LibreOffice: 3.3, 3.4, 3.5, and so on. - the branch x.y+1 would be defined by cloning the x.y branch and next adding new testcases needed by QA of the new functions implemented in the x.y+1 version. - we manage RC and bugfix versions (x.y.1, x.y.2, etc.) only by creating dedicated testruns: there is no reason to change the testcases set when we pass from version x.y.0 to rc x.y.1 rc1 to bugfix x.y.1 to ... etc. So there is no reason to have a new branch for a new rc or bugfix - we need to define the buildID for each testrun, which does not seem to be the case at this moment. According to that, we would have: - 2 branches 3.3 and 3.4 - for the branch 3.3 : + one testrun 3.3.3-rc1 + one testrun 3.3.3-rc2 - for the branch 3.4 : + one testrun 3.4.0 + one testrun 3.4.1-rc1 + one testrun 3.4.1-rc2 + etc. + one testrun 3.4.2-rc1 + etc. - then a branch 3.5 + etc. I think we need to agree on the general organization before that the system has become too big and too widely used. I can do these modifications if we have a general agreement. As that assume to remove several branches and to rebuild testruns, it would be prudent to not be several to do changes at the same time. What do think about this proposal? Did I overlook some point which make my beautiful organization completely stupid? Have a nice day and sorry for my bad English. JBF -- Seuls des formats ouverts peuvent assurer la pérennité de vos documents. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
On Mon, Jun 13, 2011 at 11:47:18AM +0200, Petr Mladek wrote: > Also Sophie and Yi Fan are trying to organize testing of beta and rc > builds. There is used Litmus server to coordinate the work between the > various testers. I am sure that they are looking for more testers and > test case writers. Hi Cor, Great thanks for the message! Sorry I overlooked your mail at first glance, our regression testing is organized here: https://tcm.documentfoundation.org/ Here is the instruction how to contribute on running test on it: http://wiki.documentfoundation.org/Litmus I am probably going to deliver some important new cases or coverage areas to the mailing list for review/discuss this week, please keep tuned and let me know if you have any ideas about this or about QA. Also please do not hesitate to let us know if any questions. We are looking forward to you :) Best wishes, Yifan -- Yifan Jiang Libreoffice Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi Cor, Cor Nouws píše v So 11. 06. 2011 v 02:58 +0200: > Hi all, > > Quite regular I have the feeling that at least me would be helped a bit > by doing QA if there is some hint-list.. It sounds great! We are still looking for more QA people. You might find some ideas at http://wiki.documentfoundation.org/QA AFAIK, we still do not have enough people doing the bug triage. There always will be more bug reports than developers are able to fix. This activity helps a lot to find the important bugs in the flood of bug reports and prioritize the work. Rainer is very active here and might give you some useful hints. Also Sophie and Yi Fan are trying to organize testing of beta and rc builds. There is used Litmus server to coordinate the work between the various testers. I am sure that they are looking for more testers and test case writers. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] QA-hints ? -> question for devs !
Hi Cor, I think the Weekly Development Summary might do what u want. It tends to slide by unnoticed because no one replies to it as they are all plowing straight ahead. Here is an easy way to find it... Go to top/thread view of dev list in Nabble -- http://nabble.documentfoundation.org/Dev-f1639786.html Enter 'development summary' (quotes not needed) into Search field (upper right) and click Search. In search result page switch view to 'last week' and click Go>> The one and only result is thus: http://nabble.documentfoundation.org/development-summary-year-2011-week-22-td3029190.html Download the attachment(s) for the build you want to test from Petr's summary. Here are the 3.4 bug fixes for 2011 week 22: + artwork + replace OOo icon to LibO icon (fdo#33229) [Andras Timar] + replace OOo icons to LibO icons in Tools - Options dialog (fdo#33229) [Andras Timar] + calc + fix for Insert sheets in a protected spreadsheet document (fdo#37772) [Markus Mohrhard] + fixed selection by arrow keys around merged cells. (fdo#34214) [Kohei Yoshida] + components + fix for Keyboard navigation broken in tools - options (fdo#37761) [Andre Schnabel] + extras + replace StarOffice icons to LibreOffice icons in Web Wizard (fdo#33229) [Andras Timar] + impress + add sanity check before dereference, (bnc#694119) [Tor Lillqvist] + libs-core + fix for bug (fdo#37590) [Noel Power] + replace "seagull" icons to LibreOffice icons (fdo#37617) [Andras Timar] + libs-extern-sys + update: Brazilian portuguese spelling dictionary & hyphenation (fdo#37671) [Andras Timar] + writer + band-aid for immediate crash in IsAlignPossible (rhbz#710004) [Caolán McNamara] + fix one of the crashes in wizards (fdo#36306) [Jan Holesovsky] Hope that helps... LeMoyne -- View this message in context: http://nabble.documentfoundation.org/QA-hints-question-for-devs-tp3051408p3054159.html Sent from the Dev mailing list archive at Nabble.com. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
[Libreoffice] QA-hints ? -> question for devs !
Hi all, Quite regular I have the feeling that at least me would be helped a bit by doing QA if there is some hint-list.. I mean, it is not often that I make time to install a nightly build and really start using it. And then it feels a bit pointless just to do some often easy work - after all, work has to be done here. Therefore the idea for some sort of collected hints. When one knows there has been done some work in this or that area. it feels much more sense full to do some tests ;-) Yes, I know that for good testing it is necessary to check also the un-expected (so basically anything ;-), but you'll get what I mean. And also I know that just by lurking on the list, I pick up some ideas, possible areas to have a look at, but with some extra hints... What do people think if this? Regards, -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
