Re: [Libreoffice-qa] 'free' testing on Moztrap
Hi Sophie, Ah, I just got your role updating in Moztrap from test manager to admin (Sorry I should have noticed that). So you may probably see more user information from the management interface Manage - Users. Moztrap registration needs email activate before usable for test, so I guess most of the emails should be activated. Does it partly resolve the problem? :) Best wishes, Yifan On Fri, Feb 21, 2014 at 10:45:25AM +0100, Sophie Gautier wrote: Hi Yifan, Le 20/02/2014 04:54, Yifan Jiang a écrit : Hi Sophie, I probablly didn't get the background clear here. Sorry if I disturb your thinking :) In Moztrap, when mark a step failed, there's an option to a bugzilla link, see: http://manual-test.libreoffice.org/results/case/1559/ Does it work for your situation if the reporter pastes the bugzilla link (where the attachment appears) back to Moztrap test case? Yes it works, and several testers use it, that's a great feature when you understand Bugzilla and write in English fluently. But I would like Moztrap to be accessible to all, even for QA beginners. What I miss the most for the moment (out of l10n ;) is a way to contact the tester. MozTrap shows only the user name but no way contact him, so if I need more information to open a bug based on his feedback, it's not easy. Cheers Sophie ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 'free' testing on Moztrap
Hi Sophie, I probablly didn't get the background clear here. Sorry if I disturb your thinking :) In Moztrap, when mark a step failed, there's an option to a bugzilla link, see: http://manual-test.libreoffice.org/results/case/1559/ Does it work for your situation if the reporter pastes the bugzilla link (where the attachment appears) back to Moztrap test case? Best wishes, Yifan On Wed, Feb 19, 2014 at 03:37:14PM +0100, Sophie Gautier wrote: Hi all, I'm still reviewing the test we have in Moztrap and increasing them. Because of the amount of bugs we had on Calc, we discussed about 'free' tests on Moztrap where the tester just test one of his document and report in Moztrap what's work/fail. Unfortunately, it will be impossible for me to track the feedback in case of failures and make a report on BZ based on it. I can't reach the tester and I won't be able to reproduce with no clear steps and expected results. Have you some suggestions on how that testing could be made possible? Cheers Sophie ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [MozTrap] CSRF error after login (error 403)
Hi Thomas,
Did you use username, Persona or Openid to login? The problem was
there for some time because of the server side cache, but I think it
should have been configured properly long before.
I suspect it might be because of the client side. Would you double
check your cookie/plugin settings? Preferrably to launch a brand new
instance of firefox without plugins and make sure cookies is
accepted. For example, close all your firefox window, then launch firefox
-safe-mode.
Then we may define where the problem is. On the other hand, to see if
your account works, you could also try logging using another browser
like Chrome.
Best wishes,
Yifan
On Thu, Dec 26, 2013 at 05:52:10PM +0100, Thomas Hackert wrote:
Hello Sophie, Yifan, *,
after waiting a couple of days for answers to my mails here, I
wanted to continue my MozTrap test. But for whatever reason I get
only
quote
Forbidden (403)
CSRF verification failed. Request aborted.
Help
Reason given for failure:
CSRF token missing or incorrect.
In general, this can occur when there is a genuine Cross Site
Request Forgery, or when Django's CSRF mechanism has not been used
correctly. For POST forms, you need to ensure:
Your browser is accepting cookies.
The view function uses RequestContext for the template, instead
of Context.
In the template, there is a {% csrf_token %} template tag inside
each POST form that targets an internal URL.
If you are not using CsrfViewMiddleware, then you must use
csrf_protect on any views that use the csrf_token template tag, as
well as those that accept the POST data.
You're seeing the help section of this page because you have DEBUG =
True in your Django settings file. Change that to False, and only
the initial error message will be displayed.
You can customize this page using the CSRF_FAILURE_VIEW setting.
/quote
as an answer ... :( As I have allowed the whole domain
*libreoffice* to NoScript and have disabled AdBlock Edge on our
sites, I am not sure, if it is just a problem on my system or if
anyone else is affected ... :( Could you have a look at the site,
please (and hopefully will be able to fix it ... ;) )?
TIA
Thomas.
--
Yifan Jiang
SUSE Desktop, Libreoffice /SUSE
Contact: yifan - irc.freenode.net/libreoffice
=
http://www.libreoffice.org/
http://www.documentfoundation.org/
___
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [MozTrap] CSRF error after login (error 403)
Hey Thormas, On Fri, Dec 27, 2013 at 05:07:11AM +0100, Thomas Hackert wrote: Good morning Yifan, Good morning :) I did not encounter this problem the last days nor shortly after I sent my mail ... :( So as I understand, the problem is not there any more, right?:) Preferrably to launch a brand new instance of firefox without plugins and make sure cookies is accepted. For example, close all your firefox window, then launch firefox -safe-mode. Even though I was able to log in again? If the problem was there persistently, this was to exclude all plugins that may cause such a problem. It helps to identify the problematic area on the browser side. Then we may define where the problem is. On the other hand, to see if your account works, you could also try logging using another browser like Chrome. Well, Chromium does not want to load data from any LO site, even if I delete its ~/.config/chromium directory ... :( That was weird. I ever tested it on Chrome, it worked just fine (though some of the UI part looks a bit weird). Best wishes, Yifan -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Moztrap: relation between version/runs/cases
Hi Sophie / all, It is really amazing the testing was push forward so much :) Thanks a lot for working on this. Merry Christmas and God bless us all! Best wishes, Yifan On Fri, Dec 20, 2013 at 01:18:47PM +0100, Sophie Gautier wrote: Hi Yifan, Le 19/12/2013 10:56, Yifan Jiang a écrit : Hi Sophie, A direct guess of mine is the new test cases not included in a test suite, which is the only way to get test cases in a test run. Your guess is right, I missed this step, I thought once added the P3 tag, it goes in the series. That is to say, when a test run is being created, selecting corresponding test suite wrapping those cases is the only method for us to select wanted test cases. So before the test run is created, the target test cases should be there in a test suite. This is actually a way I don't really like :( ok, I understand. So when creating a brand new test case, there's chance to select which test suite should it be in: http://manual-test.libreoffice.org/manage/case/add/ ok, I see my mistake here Unfortunately the UI seems not there when editing an existed case. In this situation, we need to edit the test suite in: http://manual-test.libreoffice.org/manage/suites/ and edit P3 test suite here: http://manual-test.libreoffice.org/manage/suite/12/ in the page, the p3 tag search in the left hand side panel is usable to get the wizard specific cases. ok, thanks a lot, I see now how I can correct my mistake :) In addition, updating the test suite does not update test run on the fly, we need to recreate a test run after doing proper test suite set up. ah ok, noted. It seems we are having much more fun with Moztrap, my further comments as below. Cheers :) Best wishes, Yifan On Wed, Dec 18, 2013 at 11:51:21AM +0100, Sophie Gautier wrote: Hi Yi Fan, all I need some help to understand the relation between test cases, run, versions. - when I create a new Version (LibreOffice 4.2.0), I indicate that test cases are based on LibreOffice 0 Yes, that's right. - then I've created 4.2.0 beta2 and 4.2.0RC1 runs always based on LibreOffice 0 test cases When you created test runs, the cases in them are based on test suites in the version. So in this scenario, the runs should have been based on Libreoffice 4.2.0 Version+Selected test suites Yes, I see the difference, thanks The problem I encounter currently is that if I modify test cases or create new one on LibreOffice 0, they are not updated to the new run I will create for this version. Before creating 4.0.2 RC1 run, I've added two new test cases to check wizards feature in Writer, but they don't appear in this run. It seems that the test cases appear only if I create a new Version. Or may be there is something wrong that I'm doing? I've tried several time without finding a solution. Ditto :) May be I can delete the actual LibreOffice 4.2.0 version and recreate it, but what would happen to the existing 4.2.0 beta2 run? It doesn't help if test cases are not in a test suite :) Yes, thank you very much for your help. I think I've understand the whole process now. Cheers Sophie -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Moztrap: relation between version/runs/cases
Hi Sophie, A direct guess of mine is the new test cases not included in a test suite, which is the only way to get test cases in a test run. That is to say, when a test run is being created, selecting corresponding test suite wrapping those cases is the only method for us to select wanted test cases. So before the test run is created, the target test cases should be there in a test suite. This is actually a way I don't really like :( So when creating a brand new test case, there's chance to select which test suite should it be in: http://manual-test.libreoffice.org/manage/case/add/ Unfortunately the UI seems not there when editing an existed case. In this situation, we need to edit the test suite in: http://manual-test.libreoffice.org/manage/suites/ and edit P3 test suite here: http://manual-test.libreoffice.org/manage/suite/12/ in the page, the p3 tag search in the left hand side panel is usable to get the wizard specific cases. In addition, updating the test suite does not update test run on the fly, we need to recreate a test run after doing proper test suite set up. It seems we are having much more fun with Moztrap, my further comments as below. Cheers :) Best wishes, Yifan On Wed, Dec 18, 2013 at 11:51:21AM +0100, Sophie Gautier wrote: Hi Yi Fan, all I need some help to understand the relation between test cases, run, versions. - when I create a new Version (LibreOffice 4.2.0), I indicate that test cases are based on LibreOffice 0 Yes, that's right. - then I've created 4.2.0 beta2 and 4.2.0RC1 runs always based on LibreOffice 0 test cases When you created test runs, the cases in them are based on test suites in the version. So in this scenario, the runs should have been based on Libreoffice 4.2.0 Version+Selected test suites The problem I encounter currently is that if I modify test cases or create new one on LibreOffice 0, they are not updated to the new run I will create for this version. Before creating 4.0.2 RC1 run, I've added two new test cases to check wizards feature in Writer, but they don't appear in this run. It seems that the test cases appear only if I create a new Version. Or may be there is something wrong that I'm doing? I've tried several time without finding a solution. Ditto :) May be I can delete the actual LibreOffice 4.2.0 version and recreate it, but what would happen to the existing 4.2.0 beta2 run? It doesn't help if test cases are not in a test suite :) Best wishes, Yifan -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Existing German translation in Moztrap
Hi Thomas, On Wed, Nov 20, 2013 at 07:47:34AM +0100, Thomas Hackert wrote: 1. the UI level The content is relatively static in moztrap *code*, usually *translated* word by word. What does code in this context mean? I need to use a programming language like C++, JS, or something else? It means strings in Moztrap native code, mostly they are Python, JS, html ... stuff in Django framework. Yes, with Django framework, it is not that complicated to leverage pootle approach to translate the UI of Moztrap, where UI indicates any text else except test cases content in Moztrap. And that would mean, I need an account to look at TDF's implementation of Django? I am not sure if I got it, but you can just see my git repo directly. But, I doubt it is really necessary to implement and MAINTAIN this even if the word-by-word translation has been highly automated: http://translate.google.com/translate?hl=ensl=entl=deu=http://manual-test.libreoffice.org/results/runs/ Ah, O.K. Now I know, why some of the test cases sound strange ... ;) As stated, this is supposed to only use for UI level, instead of the test cases translation, which still need human works. The test cases may be translated by the automation translator currently just because we did not implement the test case level localiztion yet. In my experience, translation engines are good to get a short overview, what is inside a text. But they loose, if you want a reliable translation ... :( 2. the test case level The content is relatively dynamic in moztrap *database*, usually *localized* sentence by setence or paragraphs by paragraphs. The similar way of UI level translation does make sense in the first place, however *database* level translation is not that easy as what we did in *code* level in a Django framwork. It may be needed to deeply reconstruct the moztrap core tables, and hook together pootle like translation content with the refined moztrap database. If we could translate it via Pootle, then it would ease the work for us translators ... ;) How difficult would it be to implement it? In another word, yes, technically it can be more complicated :) Well ... ;) Yes, I understand, it is good to have feature, the only limitation is time and effort. As I have not officially been working as full-time libreoffice qa for some months, many other works just block actions from mine:( That's why we are currently have such a workaround for translation, I hope it won't bring too much pain. Thank you for your answer No problem :) Thomas. TOFU removed Best wishes, Yifan -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Existing German translation in Moztrap
Hi Thomas, On Wed, Nov 20, 2013 at 10:11:05AM +0100, Thomas Hackert wrote: And that would mean, I need an account to look at TDF's implementation of Django? I am not sure if I got it, but you can just see my git repo directly. To see it is not the problem, but to correct spelling errors or to translate the strings ... :( If it is needed to change the code or implement code level translation, just fork the repo and submit pull request :) If it is needed to update test case translation, simply a matter of case management through Moztrap web UI, while you'll need to have a case manager role. I think @Sophie also knows much about it. Best wishes, Yifan -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Existing German translation in Moztrap
Hi Thomas, On Wed, Nov 20, 2013 at 11:56:33AM +0100, Thomas Hackert wrote: Hello Yifan, *, On Mittwoch, 20. November 2013 10:44 Yifan Jiang wrote: On Wed, Nov 20, 2013 at 10:11:05AM +0100, Thomas Hackert wrote: And that would mean, I need an account to look at TDF's implementation of Django? I am not sure if I got it, but you can just see my git repo directly. To see it is not the problem, but to correct spelling errors or to translate the strings ... :( If it is needed to change the code or implement code level translation, just fork the repo and submit pull request :) huh ... Will I need an account at github then? And you know, that I am not a programmer, don't you ;? Hehe, sure you'll need an account to submit a pull request :) Best wishes, Yifan -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Existing German translation in Moztrap
Hi *, The repo is here (silently), may be I should make it available for @tdf searching :) https://github.com/yifanjiang/moztrap The important hacking on this branch is OpenID login implementation. As for the localization, IMHO, is inevitably considered divided into 2 levels: 1. the UI level The content is relatively static in moztrap *code*, usually *translated* word by word. Yes, with Django framework, it is not that complicated to leverage pootle approach to translate the UI of Moztrap, where UI indicates any text else except test cases content in Moztrap. But, I doubt it is really necessary to implement and MAINTAIN this even if the word-by-word translation has been highly automated: http://translate.google.com/translate?hl=ensl=entl=deu=http://manual-test.libreoffice.org/results/runs/ 2. the test case level The content is relatively dynamic in moztrap *database*, usually *localized* sentence by setence or paragraphs by paragraphs. The similar way of UI level translation does make sense in the first place, however *database* level translation is not that easy as what we did in *code* level in a Django framwork. It may be needed to deeply reconstruct the moztrap core tables, and hook together pootle like translation content with the refined moztrap database. In another word, yes, technically it can be more complicated :) Best wishes, Yifan On Thu, Nov 14, 2013 at 09:20:32AM +0100, Thomas Hackert wrote: Hello Robinson, *, On Donnerstag, 14. November 2013 08:59 Robinson Tryon wrote: On Thu, Nov 14, 2013 at 2:50 AM, Thomas Hackert thack...@nexgo.de wrote: At least the first part (localizing the interface) seems like it just needs someone to go poke at the code for a while (no, I'm not raising my hand right now... :-) I cannot help here as well, sorry ... :( If I read the above side right, it is just some kind of Python hook. But how do I see, if localization is enabled in our Django installation? I don't see a TDF fork of moztrap up on Github: https://github.com/search?q=%40tdf++moztrap there is no Django fork there as well (https://github.com/search?q=%40tdf++django) ... :( Maybe the repo lives elsewhere? Maybe. But I am not one of our admins, so ... ;) And I found on https://docs.djangoproject.com/en/1.6/internals/contributing/localizing/, that the project uses transiflex (https://www.transifex.com/) itself to translate their software. Does it mean, we have to switch to transiflex as well? I think that's just what django uses for their own work. I think that lots of tools support the .po file format, so we can stick with whatever system we currently have in place (pootle, IIRC). That would be nice :) Thomas. -- Yifan Jiang SUSE Desktop, Libreoffice /SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] moztrap documentation feedback
Hi Petr, Thomas, On Tue, Apr 09, 2013 at 01:17:37PM +0200, Petr Mladek wrote: And I seem to remember (from the ToDo list, I think ... ;) ) that an import of old test cases is planned. What is the status here? Are translations of them are imported as well? AFAIK, Yi Fan already ported all useful test cases. I think that he ignored some trivial tests that did not make much sense. But I might be wrong. Yes, the status is done :) We have migrated old cases from Litmus and got them updated to Moztrap with merging duplicated steps or cases as well as wording revising. 2. How do I get my translated test cases in a test run? A short test during my translation shows me, that they were not in there ... :( Or does someone have to prove them first and then they get active? I am afraid that this is related to the branches. If you modify the text in the branch 0 it is not automatically updated in other branches. I am not sure if there is a way to synchronize this. Yi Fan? Usually when we have a newly added test case translation in version 0, it will take effect as a new version is created, which is copied from test case version 0. By the fact a test cases in a test run is bound to a specific version, to make the translation taking effect for an existed test run, we need to update the test case content for a speific version more than for version 0 only. That is to say, switch to the right test case version then paste the newly added translation from version 0 :) Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [Moztrap] Can someone explain test case #122 to me?
Hi Miguel, On Sun, Apr 07, 2013 at 09:42:38AM -0700, mariosv wrote: Hi Thomas, I did the test, and certain that the bad English is mine. Hah, actually I tried to make the scenario easier but have never done it in a better way, sadly even in Chinese :) I think it is not because of English, but the relevant complicated scenario. I feel the steps are concise and clear enough, certainly it will be nice when some one makes it more generally easy to understand for testers without little format conditional background knowledge. Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [Moztrap] Can someone explain test case #122 to me?
Hi Thomas, The translation style looks beautiful, it is fancy to see how the progress is being made :) Thanks a lot for your effort! For the conditional formatting case particularly, let's take a look at left hand side of the sheet (ONE condition by cell). The idea is cells background color in Column H is conditionally formatted, according to numeric comparison between cells in Column F and J in the same row. Meanwhile the numeric comparison is defined by Column I. For example, for a random row 11, the test case indicates background color of the cell H11 should be green when F11 (value 2) J11 ( adjustable value 3). Otherwise when F11 (value 2) = J11 (adjustable value 0), the background color of H11 should be none. Actually all steps are following the ideas above, we simply do similar kind of check for each rows in both left-hand side area (left to column N) and right-hand side area (right to column O). However the test case was described in a generic way to save the wording length. May be this brings some confusing? In addition, I tested the case myself with a master build and did not find problems: Version 4.1.0.0.alpha0+ (Build ID: 0552b4334c2fb6b130ec05934b952b60418aadc) Hope these information helps better understanding :) Best wishes, Yifan On Sun, Apr 07, 2013 at 05:57:05PM +0200, Thomas Hackert wrote: Hello Yifan, *, during my translation I stumble upon test case #122 (http://vm12.documentfoundation.org/manage/case/122/) and downloaded the attached file (http://manual-test.libreoffice.org/media/attachments/2012/09/12/LibreO_qaTest_ConditionalFormatting_3-5-5-3_en.ods). But either my English is too bad or maybe I found a bug, but when I follow the instructions in the test case, I can only see one change in the file: In LO Version 4.0.2.2 (Build ID: 4c82dcdd6efcd48b1d8bba66bfe1989deee49c3) under Debian Testing AMD64 I see only cell H6 (column #4) changing its background colour ... :( Now I am not sure, if there is something wrong on my system, the file and/or the description in the test case ... :( For any clearance and help I would be really thankful :) Have a nice evening Thomas. -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [Moztrap] Some questions to the test cases, particularly #130
Hi Thomas, On Tue, Apr 02, 2013 at 06:02:52PM +0200, Thomas Hackert wrote: In English // steps Launch LibreOffice main program (vp. 1) Click on the File menu (vp. 2 vp. 3) In English // expected result 1. The welcome window should be appeared That is a different point, I nearly forgot ... :( IIRC the welcome window only appears, when no LO was installed before, or am I wrong ;? Thanks for your answer (and forwarding this mail to the dev ML as well Thomas. Rest snipped Did you mean splash screen, which is I usually called, with loading progress bar during launching LO? It disappears after LO loaded. The welcome window in test case means the first screen shown immediately after launching `libreoffice` without components (writer, calc, base etc.) opened. The screenshot explains it :) Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ attachment: welcome-screen.png___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [Moztrap] Some questions to the test cases, particularly #130
Hi Devs, Sorry to interrupt, just to confirm, were you aware of some changes to the menu item File-Template (it seemed gone since 4.x)? The background is we were talking about vp.6 in a regression test case: http://manual-test.libreoffice.org/manage/cases/?pagenumber=1pagesize=20sortfield=created_onsortdirection=descfilter-id=130 And if we ever removed the menu, we will update the test case as well. Thanks for your confirmative information:) Hi Thomas, Thanks for the thorough review of the testing :) Yes, I found the submenu seems gone on my 4.x build as well. The case may need update since it was written based on the 3.6.x build. Referring to your question, vp means verify point, vp.1 appended to a step indicates the expected result will start with numbering 1, eg.: In English // steps Launch LibreOffice main program (vp. 1) Click on the File menu (vp. 2 vp. 3) In English // expected result 1. The welcome window should be appeared 2. These menu items should be activated: New, Open, Recent Documents, Wizards, Close, Templates, Exit Best wishes, Yifan On Tue, Apr 02, 2013 at 04:56:43AM +0200, Yifan Jiang wrote: Good morning Yifan, *, during my translation of the test cases in Moztrap into German, I stumbled upon #130 (http://vm12.documentfoundation.org/manage/case/130/). There I found quote Click on the mouse on the File - Templates sub-menu and click on each items inside (vp. 6) /quote ... I have looked in my installed LO Version 4.0.2.2 (Build ID: 4c82dcdd6efcd48b1d8bba66bfe1989deee49c3) as well as in Version 4.0.1.2 (Build ID: 84102822e3d61eb989ddd325abf1ac077904985) under Debian Testing AMD64, but have neither in my Germanophone UI nor when I switch the UI to English such an entry in the menu ... :( I have only found one under File - New (in German: Datei - Neu...). Now I am not sure, if this is a relic from former versions (or from OOo times) or if you use a different version or the like ... :( What version of LO (with what locale on your OS, which OS etc.) are you using to create the test cases? Is there a difference to my system/version? If not, could you correct the test and its expected result? And I would like to see some native speakers of the English language to proofread the test cases ... ;) Maybe my English is too bad and/or there were some changes since my last lessons, but the example above sounds a little bit strange to me* ... :( Do not get me wrong: I neither want to be rude nor belittle your work at Moztrap, but should this not be quote Click on the File - Templates sub-menu and click on each item inside (vp. 6) /quote instead? And as a final question: What does this vp. mean? * This is not the only description, where I was wondering about the wording ... ;) Sorry for the inconvenience and have a nice day Thomas. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
[Libreoffice-qa] [Moztrap] Some questions to the test cases, particularly #130
Good morning Yifan, *, during my translation of the test cases in Moztrap into German, I stumbled upon #130 (http://vm12.documentfoundation.org/manage/case/130/). There I found quote Click on the mouse on the File - Templates sub-menu and click on each items inside (vp. 6) /quote ... I have looked in my installed LO Version 4.0.2.2 (Build ID: 4c82dcdd6efcd48b1d8bba66bfe1989deee49c3) as well as in Version 4.0.1.2 (Build ID: 84102822e3d61eb989ddd325abf1ac077904985) under Debian Testing AMD64, but have neither in my Germanophone UI nor when I switch the UI to English such an entry in the menu ... :( I have only found one under File - New (in German: Datei - Neu...). Now I am not sure, if this is a relic from former versions (or from OOo times) or if you use a different version or the like ... :( What version of LO (with what locale on your OS, which OS etc.) are you using to create the test cases? Is there a difference to my system/version? If not, could you correct the test and its expected result? And I would like to see some native speakers of the English language to proofread the test cases ... ;) Maybe my English is too bad and/or there were some changes since my last lessons, but the example above sounds a little bit strange to me* ... :( Do not get me wrong: I neither want to be rude nor belittle your work at Moztrap, but should this not be quote Click on the File - Templates sub-menu and click on each item inside (vp. 6) /quote instead? And as a final question: What does this vp. mean? * This is not the only description, where I was wondering about the wording ... ;) Sorry for the inconvenience and have a nice day Thomas. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Moztrap: Test case export?
Hi Nino, On Thu, Mar 28, 2013 at 11:43:23PM +0100, Nino Novak wrote: Hi, is there an test case export feature from Moztrap? Not directly from UI at the moment :) But we've got RESTful API work. This will save all version 0 cases to your local file: $ curl -o lomt_cases_v0 http://manual-test.libreoffice.org/api/v1/caseversion/?format=jsonproductversion__version=0 JSON is the only format available. See more about the api filters: https://moztrap.readthedocs.org/en/1.3.7/userguide/api/library.html#case Well, the data fulfilled with one-line is not that readable. underscore-cli, a NodeJS based tool is pretty good to do further rendering of the data. https://github.com/ddopson/underscore-cli Besides I think there are many other json data manipulating toys :) It is worthy of mentioning json can not be directly imported to Moztrap from front UI. Theoretically it can be imported on a tool on server backend, but I have never tried it... In what kind of database are the cases stored? MySQL is what we are using. I'm asking because I somehow feel the necessity to organize the cases according to my own preferences. Therefore I thought it would be fine to export them and experiment locally. You could locally store your cases freely by following the 'Gherkin-esque' format each case: https://wiki.documentfoundation.org/Moztrap/Moztrap_User_Guide#Add_bulk_test_cases Then it is allowed to be imported together. However I highly recommended to edit the test case in the front web UI one by one, in which you always know where you are. Without careful debugging, Bulk test case adding is, to some extent, risky that imported test case can be unreadable if the format is not precisely matched. With Moztrap filters it does not seem possible to get fine grained views as unfortunately I was not able to tell the filter to use AND as operand for the same criterion type (i.e. tag). It always seemed to use OR. So tags do not provide enough selection power for now. Yes, we will get used to it or otherwise hack it :) Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] about moztrap attachment missing :(
Hi Thomas, I am happy things are clear now :) On Thu, Mar 28, 2013 at 06:20:03PM +0100, Thomas Hackert wrote: Good idea, I also want to have a contribution list in a visible place. That would be great :) The features were not in Moztrap at the moment, but I'd love to put it on my working list :) For your specific idea, would you like to open an enhancement bug and assign it to me? At b.f.o or the one for Moztrap itself at mozilla.org? Libreoffice fdo bugzilla will do the work: https://bugs.freedesktop.org/enter_bug.cgi?product=LibreOffice;component=WWW;short_desc=MozTrap:%20;bug_status=UNCONFIRMED;version=? It looks like that, see upstream test case for Moztrap: https://moztrap.mozilla.org/results/cases/?pagenumber=1pagesize=20sortfield=created_onsortdirection=descfilter-name=web+appfilter-name=open+web+app Ah, O.K. Thanks for the link :) Have a nice evening, nice Easter days and the like Thanks, have a nice day and Easter days ;-) Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] about moztrap attachment missing :(
Hi Thomas, On Tue, Mar 26, 2013 at 07:00:24PM +0100, Thomas Hackert wrote: What does Version 0 mean? Is this the same as 4.0 from LO? Version 0 is a special version we defined in Moztrap as a *base* version for every other versions. Other versions should be consistent with the target LO version that we are gonna test. Hm ... Does this base version relate to the version of the test case? Say: If I write a new test case e.g. for Base (no pun intended ... ;) ), then this would be version 0 (or base version) of this test? Yes it is. And other versions would be an enhancement to this test? Or a translation? Or ... ? Other versions of test cases could be a direct copy to the 'version 0' test case or an enhancement. As you may already notice, currently the translation of a test case is embedded inside the test case instructions, of course they may be different in different versions for a same test case. Except for that we do not leverage versions specifically here. When we create a test run for testing, we have to specify a VERSION, which means exactly only the cases in that VERSION will be shown in the target test run. O.K. I think (after logging in Moztrap and clicking through the different menus ... ;) ), I am understanding it now (I hope ... ;) ): 1. You write the test, which is not LO version specific. 2. Moztrap will assign version 0 to it. Yes. 3. After saving the test and clicking on Editing $Name of the test you can assign it as an new test (does this the +1 (add this version) mean?) or to any existing test case (now only 3.6 and 4.0 are available in the menu). Am I right? If not, please correct me ... ;) You are right and the situation can be more than that. Say we have three Versions as 0, 3.5, 4.0: 1. When creating a new test case: - Select 0 in the Version field - Check on And Later Versions box besides Version field = this case will be saved for EVERY VERSIONs that LATER than 0, which are 3.5 and 4.0 here. This is why 0 is a proper base version, that Moztrap is smart to internally identify different versions chronologically: 4.0 is later than 3.5, 0 is ealier than $everything. 2. And as you said, in the the situation that the box above is not checked when adding a case, there's still a chance to add the case into other versions one by one by clicking the +x (add this version) button. All the above reveals the VERSION importance when we managing test cases. Moztrap has a solution for how to correctly manage versions. I am not gonna be verbose here, Not ;? I think, you are verbose (but in a positive way) ... ;) I really hope I could read more Shakespeare to write more elegantly ^^ but the reason we want to at least update everything generic in Version 0 is we want a consistent case version management. As a result: - we have a community agreement that each case's Version 0 is the latest edited version of the test case Why do you put agreement in quotation marks? Is it not an official agreement? Well, I simply felt the word agreement is somewhat strong... :) - every new version is branched from Version 0, so that they can include the latest update of the test cases But how do you branch it? The branch in practice is when we creating a new VERSION, Copy environments and cases from optional field would do the trick. This is the way how we are gonna manage versions in a cooperative way. More details can be found in the headache documents above :) Just out of curiosity: When I start translating a test, would it keep the version number 0 or would it become version 1 then? It's version 0 by default as mentioned. See the top right version field if you wanna confirm when translating: Just in case, in practice, the 'version' operation of a test case can be found on the right top corner when you edit a case like: http://manual-test.libreoffice.org/manage/caseversion/419/ Leave other uninteresting cases for other people who may be interested in them. That is to say, other people can see which cases have been executed by you, and vice versa :) Would this be NL specific or would I see all testers? What does it mean NL speific? All results are equally sharable and people do not special permission to see results. Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more
Re: [Libreoffice-qa] about moztrap attachment missing :(
Hi Thomas, On Wed, Mar 27, 2013 at 07:02:02PM +0100, Thomas Hackert wrote: You are right and the situation can be more than that. Say we have three Versions as 0, 3.5, 4.0: 1. When creating a new test case: - Select 0 in the Version field - Check on And Later Versions box besides Version field = this case will be saved for EVERY VERSIONs that LATER than 0, which are 3.5 and 4.0 here. This is why 0 is a proper base version, that Moztrap is smart to internally identify different versions chronologically: 4.0 is later than 3.5, 0 is ealier than $everything. But how do you write a test, if you just want it to be used for new functions – e.g. for the upcoming version 4.1 and later? To add this case, follow the situation 2 as below, keeping the check box off :) Later on, when creating futher test versions (like 4.2, 4.3), we branch version 0. 2. And as you said, in the the situation that the box above is not checked when adding a case, there's still a chance to add the case into other versions one by one by clicking the +x (add this version) button. Do you have to assign the test manually to every version then? Yes, as I understand, the first situation is the *only* chance to add a test case to all existing versions in batch. This is the way how we are gonna manage versions in a cooperative way. More details can be found in the headache documents above :) Just out of curiosity: When I start translating a test, would it keep the version number 0 or would it become version 1 then? It's version 0 by default as mentioned. See the top right version field if you wanna confirm when translating: I thought, translating and saving my translated part, would count as a new version or so ... ;) Nah, VERSION is a Moztrap internal concept tightly related with test case, instead of instructions. Just in case, in practice, the 'version' operation of a test case can be found on the right top corner when you edit a case like: http://manual-test.libreoffice.org/manage/caseversion/419/ Leave other uninteresting cases for other people who may be interested in them. That is to say, other people can see which cases have been executed by you, and vice versa :) Would this be NL specific or would I see all testers? What does it mean NL speific? All results are equally sharable and people do not special permission to see results. Native language ... ;) Is there a possibility to see team member $X from the Germanophone project, where he is testing and the like? I would like to see some kind of filter to filter testers, so I am able to see, where they are testing, which test failed on them and the like ... ;) Good idea, I also want to have a contribution list in a visible place. The features were not in Moztrap at the moment, but I'd love to put it on my working list :) For your specific idea, would you like to open an enhancement bug and assign it to me? And I have two further questions: 1. When I edit (translate) a test, there is a pulldown menu at the bottom of the page, where I can choose between active (which seems to be the default), disabled and draft. What happens, if I use draft? Is this related to the whole test case, or is there a possibility to use draft for my translation? The status is for the whole test case. 2. On the top of the page there is a button named Install as Open Web App. What is this? To install on a cell phone or the like? It looks like that, see upstream test case for Moztrap: https://moztrap.mozilla.org/results/cases/?pagenumber=1pagesize=20sortfield=created_onsortdirection=descfilter-name=web+appfilter-name=open+web+app -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] about moztrap attachment missing :(
be interested in them. That is to say, other people can see which cases have been executed by you, and vice versa :) Thank you for your information and have a nice evening Thank you for your comments and questions :) They help out to refine the documentation. Please let me know if you find anything obsolete or unclear in wiki page. -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] about moztrap attachment missing :(
Hi Thomas, On Mon, Mar 25, 2013 at 05:10:51AM +0100, Thomas Hackert wrote: Is there a possibility to set my rights in Moztrap so, that I can translate the Germanophone Test Cases? I was the guy, who started the discussion on the Germanophone discuss ML about manual testing (Moztrap vs. Litmus) to contribute (again) more to the QA of LO ... Though due to my job I am only able to contribute on weekends a little bit ... :( But I would like to translate some of the test cases to relieve N00bs without (or too less) English knowledge their contribution to QA ... ;) Of course, I saw your previous discussion as well. Thanks and your account has been upgraded to test manager, who's able to edit and create new cases. Please be aware to let the mailing list know if you want to revise more than test cases (test suite, version etc.). Ideally we do not frequently adjust other settings except for runs and test cases. Please take a bit time to look through this wiki page, which could be helpful for a quick start: https://wiki.documentfoundation.org/QA/Testing/Test_Case#Add_a_new_test_case For thorough understanding of Moztrap management, please consult: https://moztrap.readthedocs.org/en/1.3/userguide/index.html To summarize what is *very* *important* to add a test cases in LO Moztrap: 1. The test case is categorized into proper test suite (Priority 1/2/3/4) 2. The test cases are needed to tag with priority and component. i.e for each test case, we need at least *2* tags like writer, p1, calc, p2. Visist the tags management view to see how many tags are available. http://manual-test.libreoffice.org/manage/tags/ Though Your permission is allow to add extra tags, the component and priority tags are very important for our test cases filter from practice. 3. A new test case has to add to a Version 0 and checked as later versions, so that all the versions of the test case would have the new test case included. Any questions please do not hesitate to ask :) And I miss a function to skip tests in Moztrap ... :( Is it possible to implement it? Sorry I did not notice features like that. How does that work as? Would you specify a little bit more. Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [ANN] LibreOffice 4.0.2 RC1 available
Hi Pedro, On Thu, Mar 21, 2013 at 05:28:07AM -0700, Pedro wrote: Hi Yifan Some comments/questions Yifan Jiang wrote 3. Select your envionment: Archs, Locale, Platforms What if my Locale doesn't exist? If I run LO with the English US UI but all my other settings are set to my Locale does that count as EN_US? It's a very good question, We should define the item clearly. The environment dimensions (arch, locale, platform...) are extremely flexible in Moztrap, we customized these items all by ourselves. The original idea I set Locale as an option is to record the testing operating system locale. Then everything else is supposed to left to libreoffice to decide automatically. Of course things come to be a bit jumbly, as you mentioned, testing environment options can be manually tuned, the additional language packages can be freely installed and uninstalled , etc. Even more, localization includes a lot of detailed meanings and coverage: * UI translation * Document translation (help doc) * Document Language bundles * Spell check dictionaries * Thesaurus * Grammar check * Hyphenation * East-Asian specific and complicated text support So it may make sense to keep the concept as simple as operating system locale. When a testing environment is selected as fr for example, I do *NOT* presume the above detailed coverages would be all related with French packages or settings. Alternatively these language specific testing would be reflected in *testcases* description and steps: * Ideally these language specific *testcases* should be tested in relevant environments. * Conversely those non-language specific *testcases* should be tested in only one environment. which I think are Moztrap's future roles to take care about how to map different cases to different locales. Finally as far as I can concern, everything manually set, inluding language settings, which is relevant to the results should go to comments field when marking the test results. I would like to keep the topic open, and get more people with l10n experience involved. :) @Petr, Sophie, all, any input here? Yifan Jiang wrote Finally, click the bright green button on the bottom right and start to hunt bugs! Actually you may use the latest 4.0.2 RC build (i.e from RC1 to RC2...RC3 etc.), for testing. Should the user follow the Test Order number? Not yet. The test cases have no restricted orders so far :) I think the numbering is a new feature where we may leverage afterwards when needed. Why is the Install test (which is first action you take) numbered 16? hehe, I'll have to hide it if really confusing. Is it considered low priority (Priority 2) that the user has a successful Install??? I am not sure I got the question. But the priority does matter. The high priority (P1) cases are supposed to run before low priority (P2) ones in any case, beacause test cases having high priority usually are designed to test fundamental and most important functionality, wrt: https://wiki.documentfoundation.org/QA/Testing/Test_Cases_Organization#Priority FYI OpenID login fails with 500. That's an error. The server could not process your request. That's all we know. Thanks for trying. Which openid provider were you using? I did test the OpenID scenario when upgrading and ever worked for me with google, wordpress and yahoo ids. Unfortunately the server seems down at the moment so I couldn't do further testing for now. But I'll certainly take care of this :) Also, the Wiki screenshot is outdated. It doesn't show the OpenID option. I can make a new screenshot and upload it to the wiki, but without edits (you can do that later?) Great, I can do it. But I think if it doesn't take too much time, you are the better person to update the wiki since you found the problem. Thank you! Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [ANN] LibreOffice 4.0.2 RC1 available
On Fri, Mar 15, 2013 at 02:43:02PM +0100, Thorsten Behrens wrote: A good way to assess the release candidate quality is to run some specific manual tests on it, our TCM wiki page has more details: http://wiki.documentfoundation.org/QA/Testing/Regression_Tests#Full_Regression_Test - or checkout our manual test database for starting right away - http://manual-test.libreoffice.org/runtests/ Dear all, To fire on the regression test on 4.0.2 RC immediately, the usage of Moztrap can't be more simple: 1. Login manual test URL as above 2. Select right run: Libreoffice - 4.0 - 4.0 RC Regression Test 3. Select your envionment: Archs, Locale, Platforms Finally, click the bright green button on the bottom right and start to hunt bugs! Actually you may use the latest 4.0.2 RC build (i.e from RC1 to RC2...RC3 etc.), for testing. Thanks for joining and we are looking forward to your contribution! PS. Further details of usage of Moztrap can be found: https://wiki.documentfoundation.org/Moztrap/Moztrap_User_Guide Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [ANN] LibreOffice 3.6.6 RC1 test builds available
Dear all, To fire on a regular regression test set for 3.6.6 RC immediately, the usage of Moztrap can't be more simple: 1. Login manual test URL: http://manual-test.libreoffice.org/runtests/ 2. Select right run: Libreoffice - 3.6 - 3.6.6 RC Regression Test 3. Select your envionment: Archs, Locale, Platforms Finally, click the bright green button on the bottom right and start to hunt bugs! Actually you may use the latest 3.6.6 RC build (i.e from RC1 to RC2...RC3 etc.) for testing. Further details of usage of Moztrap can be found: https://wiki.documentfoundation.org/Moztrap/Moztrap_User_Guide Thanks for joining and we are looking forward to your contribution! Best wishes, Yifan On Thu, Mar 21, 2013 at 08:34:35AM +0100, Fridrich Strba wrote: Hi *, for the upcoming new version 3.6.6, the RC1 builds now start to be available on pre-releases. This build is slated to be first release candidate build on the way towards 3.6.6, please refer to our release plan timings here: http://wiki.documentfoundation.org/ReleasePlan#3.6_release Builds are now being uploaded to a public (but non-mirrored - so don't spread news too widely!) place, as soon as they're available. Grab them here: http://dev-builds.libreoffice.org/pre-releases/ If you've a bit of time, please give them a try report *critical* bugs not yet in bugzilla here, so we can incorporate them into the release notes. Please note that it takes approximately 24 hours to populate the mirrors, so that's about the time we have to collect feedback. NOTE: This build is in a release configuration and _will_ replace your existing LibreOffice install on Windows. The list of fixed bugs relative to 3.6.5 is here: http://dev-builds.libreoffice.org/pre-releases/src/bugs-libreoffice-3-6-6-release-3.6.6.1.log So playing with the areas touched there also greatly appreciated - and validation that those bugs are really fixed. Thanks a lot for your help, Fridrich ___ LibreOffice mailing list libreoff...@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
[Libreoffice-qa] Moztrap test cases update.
Dear people, With the goal of making Moztrap test cases available for productive testing, I updated all Moztrap test cases in recent days: 1. Merge new cases (mostly from Lucian mail, which is attached here) 2. Refine all cases steps and results. Make sure most of them are more clearly understandable. 3. Refine test case layout by using markdown all the way 4. Attach all necessary test documents in Moztrap test case, instead of referring for external links I think it is all set to use in 4.0+ testing phase! Please any one who has a plan to organize some regression test session, I am right here to support at any time :) I will review the release schedule and make a test run for next build any way. You are welcome to comment, send more revision of test cases, or create new test cases. Thanks all! Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Fwd: Re: [tdf-discuss] LibreOffice 4.0 Test Marathon
Hi Lucian, I was trying to review and merge testcases in Moztrap these days. You may find out what I have done via sort by Modified date: http://manual-test.libreoffice.org/manage/cases/ Though the progress is still ongoing, major revising of your cases is I sometimes merged many of the rows of your document into a single test case. For example, all of the Welcome screen menu items check is merged to a single case: http://manual-test.libreoffice.org/manage/case/130/ The reason is we would be better off breaking down test cases in a *proper* level. Instead of having many test cases covering similar things in the same area, a relatively compact case including those test items as detached steps/verify points could be more easy to navigate, maintain and run. I'd also love to do the similar thing for other components as well. Please let me know if you have other thoughts about it. And of course I'd highly expect you can join the maintain of Moztrap since you have written great cases :) Thanks! Best wishes, Yifan On Sun, Dec 16, 2012 at 03:00:32AM +0200, oprea.l...@gmail.com wrote: Hi, I could not type anything into MozTrap because I focused on coordinating activities in Romania - http://tableta.ceata.org/p/Maraton_de_testare_de_LibreOffice_4.0_la_Ceata But for those who want to test may use and populate the files available online at * Test LibreOffice Launcher - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdEhwOWJ5WkprLW90T09RZWw1ZkFBSkE#gid=0 * Test Writer LibreOffice - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdDliQV9YQjhiWFFrTW53SDhqUWVZbVE#gid=0 * Test Calc LibreOffice - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdE0xRllDNGd1QWpOR25tZnk2UkI2aFE#gid=0 * Test Impress LibreOffice - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdGdtb2h1V0Z5U2lXS29mMEQyaXIxdGc#gid=0 * Test LibreOffice Base - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdEpOQXViNW9leVBjeEZjc09TUTlCZlE#gid=0 * Test LibreOffice Math - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdC1RcWROZnZkbS02VjNFbzN6cExhUGc#gid=0 * Test LIbreOffice other - https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdEtKYjkzcW5Ua2lvVmJxLVp1TU9JOXc#gid=0 and obtained scenarios will be introduced in future tests MozTrap. If you agree, should be popularized these links. I will monitor the work. Lucian, - Lucian Oprea Telefon: 0745 592602 Comunitatea openSUSE Romania - http://suseromania.ro Comunitatea LibreOffice Romania - http://libreoffice.ro Pe 13.12.2012 04:50, Yi Fan Jiang a scris: Hi Lucian, On Thu, Dec 13, 2012 at 03:17:10AM +0200, oprea.l...@gmail.com wrote: Hi Yifan, In MozTrap seems easy to work (there are a few things you need to understand). It's wonderful you feel it is easy to work with :) Until then, we have created here https://docs.google.com/spreadsheet/ccc?key=0At9lOM8_6gsLdEhwOWJ5WkprLW90T09RZWw1ZkFBSkE # gid = 0 several test scenarios (we went from something I found on the wiki). I think we created enough test scenariosfor Luncher LibreOffice . Please see and bring observations if necessary. Thanks, they look good, and I guess many items were covered by existed Moztrap test cases. I saw a bunch of different launch scenario, which is nice. But in Moztrap they could be classified to be 2-3 test cases for easy and neat navigating, then detaching each of the rows in the gdocs to different steps in a Moztrap test case. Unfortunately, it's a lot of work and time does not allow me to complete this work in a few days alone. The first step merging of these test cases to Moztrap need not to be perfect. Please feel free to copy and paste. I really hope you can smoothly try to employ Moztrap as much as possible this time. The only 2 tiny problems need to be concern are avoiding duplication and merging some rows of cases with similar subjects to one single test case. Meanwhile, we will definitely find a way to work this out before 16 Dec :) How about point the QA people to Moztrap, which simply having a 4.0 testing run based on existed cases. Together referring to your gdoc link for further testing. In that case, QA may naturally filter out those duplicated ones at this time. Nice things of adapting Moztrap for manual testing benifits to all of people is we can share the testing results among each other, the bugs found can be attached to a test case, more over no duplication effort will be conducted in Moztrap. And more ... Propose that future TDF to support a group of tests to ensure the quality LibreOffice. I'd be happy to take care of this activity. It would be great if you are gonna be part of this! Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http
Re: [Libreoffice-qa] Litmus
Dear fellows, Thanks for a nice discussion with Petr, Rimas, Sophie etc, with Florian's great help, we now have a much more pretty name for Moztrap :-) manual-test.libreoffice.org I have got the stuff updated in Moztrap side today and done a bit of testing of the new name myself. It looks working smoothly! It could be brilliant if you can test the new name accordingly, and influential scenarios I could imagine are: 1. Login with Persona/OpenID/password (in which the Persona is considered has been mostly affected) 2. Native user's Password change and reset 3. new user registration 4. pictures/page styles in any pages should not be missing 5. Random function test of Moztrap itself (I don't personally think there will be a functional influence of this change) In addition, the wiki page needs to update as well, I guess it will be as simple as changing all vm12.documentfoundation.org to the new name :) I'll do it later on any way. If no problem found in this week, I'll announce the switching to the new name next Monday. Thanks a lot! Merry X'mas~~~! :-) Best wishes, Yifan On Sat, Dec 15, 2012 at 02:07:10PM +0100, Florian Effenberger wrote: Hello, so, manual-test.libreoffice.org ist now set up and ready, and the Litmus instance should be reachable under this name as well as via vm12.documentfoundation.org Can you check if everything works? Thanks, Florian -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [libreoffice-qa] Moztrap
Sorry for the confusing mail subject, I am replyting to update it to Moztrap just for catching attentions :) ~Yifan On Thu, Dec 20, 2012 at 07:02:17PM +0800, Yifan Jiang wrote: Dear fellows, Thanks for a nice discussion with Petr, Rimas, Sophie etc, with Florian's great help, we now have a much more pretty name for Moztrap :-) manual-test.libreoffice.org I have got the stuff updated in Moztrap side today and done a bit of testing of the new name myself. It looks working smoothly! It could be brilliant if you can test the new name accordingly, and influential scenarios I could imagine are: 1. Login with Persona/OpenID/password (in which the Persona is considered has been mostly affected) 2. Native user's Password change and reset 3. new user registration 4. pictures/page styles in any pages should not be missing 5. Random function test of Moztrap itself (I don't personally think there will be a functional influence of this change) In addition, the wiki page needs to update as well, I guess it will be as simple as changing all vm12.documentfoundation.org to the new name :) I'll do it later on any way. If no problem found in this week, I'll announce the switching to the new name next Monday. Thanks a lot! Merry X'mas~~~! :-) Best wishes, Yifan On Sat, Dec 15, 2012 at 02:07:10PM +0100, Florian Effenberger wrote: Hello, so, manual-test.libreoffice.org ist now set up and ready, and the Litmus instance should be reachable under this name as well as via vm12.documentfoundation.org Can you check if everything works? Thanks, Florian -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Fwd: Re: [tdf-discuss] LibreOffice 4.0 Test Marathon
Thanks Cor to fwd this! Hi Lucian, It would be great if you would take the testing organization! I have assigned your account to test manager. Please take care of the organiztion referring to wiki: Moztrap portal: http://wiki.documentfoundation.org/Moztrap How test cases are to be organized: http://wiki.documentfoundation.org/QA/Testing/Test_Cases_Organization Moztrap admin stuff: http://wiki.documentfoundation.org/Moztrap/Moztrap_Admin_Guide Full regression test ideas: http://wiki.documentfoundation.org/QA/Testing/Regression_Tests#Full_Regression_Test Basically we need to create test runs for 4.0 on demand. If a test case content update is needed, please update them in the Version 0, which would be the base of other future test case versions. Surely if something you find out not clear, please let me and the list know :) Thank you to take this! Best wishes, Yifan On Wed, Dec 12, 2012 at 06:29:44AM +0100, Cor Nouws wrote: Hi, Can someone on the QA list pls answer Lucian? thanks, Cor Original Message Subject: Re: [tdf-discuss] LibreOffice 4.0 Test Marathon Date: Wed, 12 Dec 2012 01:41:43 +0200 From: oprea.l...@gmail.com oprea.l...@gmail.com To: disc...@documentfoundation.org Hi, There are test scenarios? MozTrap to configure something? Who MozTrap admin rights? In Romania, organized on December 16 test sessions LibreOffice 4.0. If I had MozTrap rights, I try to organize our testing there. Best regards, Lucian - Lucian Oprea Telefon: 0745 592602 Pe 11.12.2012 08:14, Cor Nouws a scris: Hi all, For those interested in helping a bit out with the 4.0: we have a LibreOffice 4.0 Test Marathon from December 14 - 19: http://wiki.documentfoundation.org/QA/Test_Marathon_LibreOffice_4.0 I hope to see many of you there :-) Kind regards, -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 4.0 Test week/marathon Re: [...-qa] 3.7 bug hunt party ?
Hi, On Tue, Dec 04, 2012 at 11:23:34AM +0100, Petr Mladek wrote: [*]: We are going to use http://manual-test.libreoffice.org instead of http://vm12.documentfoundation.org . I hope that it will happen later this week. You will probably find http://manual-test.libreoffice.org is usable now, but we still need some tuning of Moztrap. The login seems not working with the new name :) I'll update everything in the coming weeks. Please stick to the vm12 until there's an official announcement. Thanks! Best wishes, Yifan -- Yifan Jiang Libreoffice / SUSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] moztrap registration error
Hi, Sigh, I spent more than 10 hr on this since found the problem, it behaves really weird and I could not reproduce it in my local environment. I was trying to ask help from upstream developers but didn't answers yet :( Hope it would not some fundamental issues related with apache wsgi mode. To figure out a fix, I will try other ways... Thanks for understanding! Best wishes, Yifan On Wed, Sep 12, 2012 at 04:13:34PM -0400, Mas wrote: Thanks everyone , I was able to get in after refreshing and then changing my password to 8 characters. This worked using Firefox. Mas On Wed, Sep 12, 2012 at 8:42 AM, Sophie Gautier gautier.sop...@gmail.comwrote: Hi Mas, On 12/09/2012 14:25, Mas wrote: I attempted to register for the moztrap application and received the following error Forbidden (403)CSRF verification failed. Request aborted.More information is available with DEBUG=True. It seems you didn't read http://wiki.**documentfoundation.org/**Moztrap/Moztrap_User_Guidehttp://wiki.documentfoundation.org/Moztrap/Moztrap_User_Guide I get this error with Chrome and couldn't log but Firefox is ok for me. Kind regards Sophie ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ -- Yifan Jiang Libreoffice / SuSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Regression test (Moztrap) test case localization temporary solution.
Hi Sophie, Petr, all, Thanks for the discussion and sorry for the late response that I was busy of other works these days :( Visually the second one is also my most favorite, so let's start here. In this week, I will arrange time to update the test cases and make their format consistent with the prototype 2. http://vm12.documentfoundation.org/manage/case/99/ @Sophie, As for the 3.6.3 testing, do you have an estimation when would it be happening? Currently there are only English cases in the database migrated from Litmus. We will still need some time to migrate other language versions before arranging tests. Looking back to Litmus, there are 4 languages version of test cases: en/de/fr/pt-Br, if necessary we could also create Ja section awaiting for input. As for the l10n related hacking, they need large scope of changes in both frontend and backend. I would plan at least 1 dev month to do all the stuff including: Moztrap UI l10n implementing, test case l10n hacking and the relevant tests before they are good enough to be online. Best wishes, Yifan On Mon, Sep 03, 2012 at 07:57:22PM +0200, Sophie Gautier wrote: Hi Petr, all, On 03/09/2012 17:34, Petr Mladek wrote: [...] So the feedback from our team is the 2nd presentation that is preferred over the others, it seems to be more readable for all of them. May be we can go for the first presentation until we get more languages but do not forget about another solution where the steps and the expected results are grouped for the same language. Sophie, thanks a lot for getting the feedback. When I think about it, also the 2nd approach can be parsed by a script. It is more complicated, probably more error prone but possible. Thus it can be moved into to really localized framework almost automatically as well. So, the second approach is probably the best temporary solution. Thanks for your investigation :) For information, I will try to organize a testing session on Moztrap for the 3.6.3 version with the FR project. May be we can try to organize a full session with very few languages, like may be DE, JA, FR at the same time? Also do you have an idea of the time needed (day per man dev) and the cost to make Moztrap fully localized (UI + content) ? Kind regards Sophie -- Yifan Jiang Libreoffice / SuSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Moztrap, some questions
Hi Sophie, On Mon, Aug 13, 2012 at 11:14:34AM +0200, Sophie Gautier wrote: Hi all, Yifan, So after a long time (sorry for this long absence), I would like to come back to Moztrap and try to help to test it. Welcome back :) Please let me know if you got problem of registration. My first question, the most important for us, is there now a possibility of localization for : - the tests - the environment From what I see on the todo list Yifan has written long ago the point : 5. Look into i18n and l10n of Moztrap, we need translation system for both test cases and Moztrap UI. Does it mean that it's possible, or does it mean it's not yet possible or that we need to find/define a translation process? I ever had a talk with the Moztrap developers and it seems neither of the localization plans has been on the rador yet :( It needs some deep hacking of the code if we want to do that particularly to Libreoffice. So it is not easy to handle the localization in Moztrap yet. One way to workaround is to mix localized wording of test cases into the existing English ones, as what we did for Litmus. But it is rather time consuming to translate and maintain, because Moztrap split test cases into steps. With this method, we need to *manually* maintain translation for each test steps and their corresponding expected results. Besides, for some of the test cases, localized version might not share exact steps with the English version. So I did not put anything localized to the Moztrap test case base yet. I feel it is possibly better to maintain translation version of test cases in a different data set, either by hacking the existing Moztrap database or putting the translation for test cases to somewhere else like wiki, then manually put the link back to Moztrap original English test case. Any way, we currently really need more people to get involved, try Moztrap, make contribution and share ideas :) One question is can we termporarily survive with pure English wording test cases, though they are allowed to choose running under any localizations? My second question: is there a possibility to store sample documents on Moztrap. Yes, one of the moztrap's nice feature is attachement support per test case :) -- Yifan Jiang Libreoffice / SuSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] qa test results
Hello, We used to leverage Litmus as our manual test tool, QA contributors would review the tests stored there and record their testing results for the latest build. The testing results are fully visible: https://tcm.documentfoundation.org/ Now we are considering to improve it to a more easy-to-use tool Moztrap. Its running instance for experiments and evaluation can be visited: http://vm12.documentfoundation.org/ In addition, how our overall testing are organized can be found: https://wiki.documentfoundation.org/QA probably the regression tests part are what you concerned: https://wiki.documentfoundation.org/QA/Testing/Regression_Tests Though Moztrap related wiki was not updated yet, please see my previous announcement of Moztrap evealuation site for more details. ~Yifan On Wed, May 23, 2012 at 03:27:49PM +0100, e-letter wrote: Readers, It is not very clear whether the latest stable version of LO has passed qa tests. Please, where is the web page that shows the tests performed, the results and the software version tested? Thanks. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ -- Yifan Jiang (江一帆) Libreoffice / SuSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] [ANN] Moztrap evaluation online - code and bugs
Hi Rainer, Thanks and the description of the subcomponent is done :) Best wishes, Yifan On Tue, May 22, 2012 at 11:50:50AM +0200, Rainer Bielefeld wrote: Hi, I defined a Sub Component MozTrap on https://wiki.documentfoundation.org/BugReport_Details#WWW Can someone please leave a short description on https://wiki.documentfoundation.org/QA-FAQ#Moztrap? Thank you and best regards Rainer -- Yifan Jiang Libreoffice / SuSE Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] minutes of LibreOffice QA call 2012-03-09 15:00 UTC
Hi Bjoern, On Fri, Mar 09, 2012 at 06:42:45PM +0100, Bjoern Michaelsen wrote: Hi all, here are the minutes of the QA call 2012-03-09 15:00 UTC attendants: Cor, Ivan, Rainer, Markus, Michael, Korrawit, Petr, Kendy, Bjoern + Yifan :) Thanks for the minutes and pushing on everything. Actually I was there and maybe not pretty audible because of the bad net circumstances and skype. I hope it would be better next time. - structured manual testing: - getting testcases for 3.5 from Litmus to Checkbox - we currently have some ~50 testcases in Litmus -- it should be easy to just copypaste them as the time window is limited until Ubuntu 12.04 beta2 (Bjoern) FYI, here are the latest test cases we got: https://tcm.documentfoundation.org/show_test.cgi?searchType=by_categoryproduct_id=1branch_id=20 The testcases marked as [OBSOLETE] are merged into ordinary cases, so they will be useless to populate into checkbox. Thank you to handle them :) AA- copypaste existing manual test as a one off for 3.5/Precise (Bjoern) AA- Check if test documents are URLs properly distributed to Checkbox (Bjoern) - find a good way to script sync testcases from Litmus to Checkbox and maybe even results back to Litmus TBD later: (maybe discuss when Nicholas Skaggs and Yifan Jiang are around) Sure, Rimas was doing smart things in Litmus system and contributed a lot, it would be great if he could join the meeting next time as well. Best wishes, Yifan ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Ubuntu/Canonical doing more manual testing for LibreOffice?
Hi Bjoern/fellows, Thanks for the lovely news! I am CCing Rimas for his concern as well. Hi Nicholas, I am Yifan from libreoffice QA :) I just took a view of the Checkbox wikipage quoted in your original mail and here are some related information available. We have got a bunch of regression test cases specifically for managed in Litmus test case management tool: https://tcm.documentfoundation.org/ [details referring to wiki] http://wiki.documentfoundation.org/Litmus http://wiki.documentfoundation.org/QA/Testing/Regression_Tests#Full_Regression_Test Meanwhile there are sample test cases description in Litmus can be found as: https://tcm.documentfoundation.org/show_test.cgi?id=1302 https://tcm.documentfoundation.org/show_test.cgi?id=1067 From my understanding it seems possible to format selective test cases and populate them to CheckBox. But the problem is the test cases are constantly being updated/created/removed through libreoffice life cycle, there may be a frustrating work to do not only for populating the cases to Checkbox but maintaining 2 versions of test base (Litmus and Checkbox). So just wonder did you experience similar situation or have any nice ideas for this (for example, syncing 2 test bases)? Thanks again for the great news :) Best wishes, Yifan On Fri, Feb 24, 2012 at 11:12:39AM +0100, Bjoern Michaelsen wrote: Hi Sophie, Cor, QA-List please see: https://lists.ubuntu.com/archives/ubuntu-desktop/2012-February/003745.html this is an opportunity to get more structured manual testing done on LibreOffice. Could you have a look if you could help out coordinating this with our own efforts to get LibreOffice tested even better? Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] credits for people doing QA
Hi there, Nice topic, 2 cents from me as follows. What I could think about roles to be considered: 1. bugzilla reporters This is pretty intuitive to count, as mentioned by Rainer, +1 when one reporting happens. 2. bugzilla information provider (confirming, following up) Also as Rainer mentioned, not so easy to count... 3. wiki editors Not sure how we distinguish those QA specific wiki authors. 4. Litmus test case runners +1 when one test case is run 5. Litmus test case authors +1 when one test case is created 6. Litmus test case update This happens when a user updates an existed test case by rivising or translating original contents. Did I miss anything else? Maybe what we want to do is to find an understandable measurement by gathering these information as QA credits, either a solo measurement to account for everything or split them to something like bug contributor/wiki contributor/litmus contributor...Meanwhile, the number of bugzilla reporters, Litmus test case runners and Litmus test case authors are relatively intuitive to count just like dev's commit numbers. Best wishes, Yifan On Sat, Feb 04, 2012 at 11:39:02PM +0100, Cor Nouws wrote: Hi, Today with my little QA presentation at FOSDEM, one person asked about how we give credits to people doing this work. Well, not structural/in a visible place, as far as I know. What about adding something here: http://www.libreoffice.org/about-us/credits/ ? We need to have some rule of course, based on which people get mentioned. Suggestions remarks? ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] preparing QA talk for FOSDEM
Hi Nino, Thanks for the thoughts about quality. For those who use Libreoffice quite frequently, practical general usage of software is pretty valuable to find issues thus improve quality, even not that surprisingly, it some time reveals bugs we never found in testing lab :) I believe the regression testing we are doing quite fits what you care about. http://wiki.documentfoundation.org/QA/Testing/Regression_Tests Especially for the mentioned Litmus. It is also a nice place to start to take part in monitoring bug-free usage: - Improve cases to share your opinion of daily usage, by improving existing tests, adding new cases or translating the cases to your own native language version. - Run test cases for each release in beta and rc place, mark it as pass/fail, accordingly report bugs. - And I believe some of the high priority test cases could be proper reference when you want to try daily/master build before any announcement. Do not hesitate to let us know when you have troubles to handle anything. Thank you! Best wishes, Yifan On Fri, Jan 27, 2012 at 03:09:03PM +0100, Nino Novak wrote: Am Freitag, 27. Januar 2012, 00:05:48 schrieb Cor Nouws: Ideas, things to add? Personally I have the impression / concern that Quality is getting one of the main challenges of LibreOffice as it seems to decrease more and more - and software acceptance to a good deal depends on quality (especially in corporate environments). So anything that /really/ raises quality is good ;-) My own ideas go into the direction of monitoring bug-free usage, e.g. by logging main user actions (module used, opening/closing files, menu path, toolbar icon klicks and so on) into a simple text file. But I really don't know if this is a good subject to talk about at FOSDEM. Nino ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] preparing QA talk for FOSDEM
Hi Cor, This is a great news, thanks for sharing :) As far as I concern, Regression is our routine testing for each release, which is pretty important to make sure the legacy important functions of Libreoffice does not regress in a new release. http://wiki.documentfoundation.org/QA/Testing/Regression_Tests For full regression test, would you like to give a view to Litmus as well? http://wiki.documentfoundation.org/Litmus And currently we encourage people to: 1. Run tests http://wiki.documentfoundation.org/Litmus/Litmus_User_Guide#Run_Tests 2. Update regression test cases - Add new test cases http://wiki.documentfoundation.org/Litmus/Litmus_User_Guide#Populate_New_Test_Cases - Translate exisiting test cases into different version Best wishes, Yifan On Fri, Jan 27, 2012 at 12:05:48AM +0100, Cor Nouws wrote: Hi, Saturday at FOSDEM I'll give a talk on QA. LibreOffice QA: as handy and joyful as possible Michael Meeks was so friendly to provide some text, while I was with my attention with different work: Come and hear how the LibreOffice QA team works, get hands on experience with bugzilla, find out where our daily snapshots are, and how to use them, and chat with fellow QA guys. See some feedback on the progress of bug tracking and some pretty graphs. Below I'll sketch the items I would like to show. I'll also check our QA wiki for important items. Is there anything that I miss, please let me know. Also: are there other QA-ers that are at FOSDEM and would like to join the talk :-) I think to target for those that are not afraid to use a BugZilla, but will greatly enjoy tips for working handy. so: - daily snapshots installation - show the page http://wiki.documentfoundation.org/BugReport_Details - most important do's and don'ts in BugZilla - learn how to use tag's / save use queries - learn how to find/search dups - bundling issues - ... - contact Ideas, things to add? Thanks, ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] QA ideas for 3.5 WAS Re: [Libreoffice] Improving the QA and Release for 3.5
Hi Cor, I hope I didn't interrupt anything :) But here's some of my comments. 1. Thanks for Rimas and Petr's actively participating, what we did in the past two month to improve the structure of Litmus and now the new stuff is online and a 3.5.0 beta regression test run is created, in which we are encouraging people to run the tests. https://tcm.documentfoundation.org/ There are a few important changes need to notify people, especially for l10n testers that: - we now have the localization test cases merged together with English cases, both in content and structure. Then we add locale-dependent property for each test case. https://tcm.documentfoundation.org/show_test.cgi?id=3105 Specifically, a test result submitted for a locale dependent cases will not influence the progress statistics of the test cases running in another locale condition. For a locale independent case,the executing result will be shared among all tests whatever locales were seleted. For locale selection I meant the bottom field here in test configuration details set: http://wiki.documentfoundation.org/File:Litmus_chose-de.png So it is important to make people informed correct locale selection is necessary before submitting test results. Also it is important to let l10n people know the translation version of test cases are not disappeared, all languages versions of test cases were just put together to save our effort depend on how much they related with languages. - we abandoned the build id selection to avoid duplicating test effort. So it is important to make people informed to actively use correct build submitting a result in the corresponding test run. For build id selection I also meant the field in configuration details set (the screenshot needs to update for this...): http://wiki.documentfoundation.org/File:Litmus_chose-de.png - we grouped test cases with priorities, and it is preferred the higher priority test cases are executed before lower ones. This can be seen when it comes to test group selection after the above configuration set. 2. We also got the wiki page updated and cleaned up. The important start pages to understand all related stuff can be: http://wiki.documentfoundation.org/Litmus http://wiki.documentfoundation.org/QA/Testing/Regression_Tests 3. As for the regression test plan, we got a full regression test related section in the QA processes page typically to describe what a Litmus admin should do when there comes a build: http://wiki.documentfoundation.org/QA/Processes#Full_regression_test_management Briefly speaking, we create a single test run for every beta and rc phases, during whose time slot the test cases inside test runs are supposed to be executed. For example, from the release plan, we will have 3 beta builds from 5/Dec to 9/Jan. In this phase, we do NOT create three test runs, instead the only test run 3.5.0 beta regression test will be recommended there for nearly a month, when continuously submitting test result in the latest 3.5 beta build is expected. 4. Is it possible to emphasize the regression strategy in the bug hunting session you are planning? :) Not sure if some of the points answered your original question :) Please let me know if anyone needs more regression test and litmus related information! Thank you! Best wishes, Yifan On Tue, Dec 13, 2011 at 11:35:44PM +0100, Cor Nouws wrote: Hi Petr, Cor Nouws wrote (09-12-11 13:44) On the wiki, I added a section with the draft for the later to create separate wiki page: http://wiki.documentfoundation.org/QA/Improving_QA-Release-3.5#DRAFT_for_page_with_info_for_bug-hunting_session Is it possible for you - without wanting to stress you - to tell smthg more about the expected planning? tjanks a lot ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Test Structure in Litmus
Hi Petr, Rimas and all, Besides the comments in the bottom, I also did Litmus update yesterday: - Master Feature branch - The empty branch has exactly the same structure of its Function counterpart, since we might also have priority and language specific consideration in feature testing. A version-specific branch, in which the practical test cases should be populated, can be always cloned from this branch. @Petr, but do we want a more concise structure for this one? - Master Function branch - 8 test groups, only test cases in priority 1, 2 were populated, we didn't have P3/P4 cases yet. The cases were just categorized to priority from my understanding from Petr's comments: P1 - highest: used for very basic tests, e.g. app can be installed; it starts; is able to load/save some test documents; so it a kind of smoketest P2 - high: test very common functionality that is used by most users. e.g. able to write text, insert picture; draw elements; create table; use function in calc; create graph, run presentation P3 - medium: test common functionality that is used by typical a bit experienced office user, e.g. create borders around tables; do animation between slides; modify text style; modify master slide page; P4 - low: test functionality for hi-tech users, e.g. writing macros, using Calc solver, complex operations with data bases - The name of function test cases are updated without ID prefix - The name of l10n test cases are updated without locale prefix, but I still keep the component#number prefix (instead in a appendix brackets) mainly because the test cases will be sorted by components prefix so that all the test cases for the same components canbe reviewed together. - In P3-L10N Test group, there are some orphan german test cases because previously the german cases were not exactly synchronized with English versions, these orphan cases are remained after my manual synchronization using google translater. I didn't finish review all of them but I'll remove duplicated ones while refer to valuable ones to create new English test cases. Please just ignore them, I'll clean them up later on :) On Mon, Nov 21, 2011 at 12:18:36PM +0100, Petr Mladek wrote: IMHO, attach it to test case is the most safe way from the design's view, which gives us the most flexibility of expanding the system. I slightly prefer this because of the flexibility. We could switch the meaning of any test case at any time without moving it in the structure. It would help us to create easier structure. Yes, I talked to Rimas days before in irc and we also agreed to add the checkbox to the test case at this phase. Rimas also mentioned it might be possible to do further hacking to have an all check box indicated all cases in a group/subgroup are language dependent/independent, but it is another thing after all. But in UI level, subgroup maybe a better place because the test cases will be created by different people, some of whom may not notice the usage of the checkbox for each test case. It might be solved by reasonable default. I would set it as functional test by default because most tests should be language independent. It should be possible to fix the status at any time, see below. Yes, it's perfectly fine :) Best wishes, Yifan ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Test Structure in Litmus
Hi Rimas, On Thu, Nov 17, 2011 at 11:06:43AM +0200, Rimas Kudelis wrote: + many QA people do not know English; they might be discouraged to do the biggest group of functional tests + non-intuitive solution; people need to follow an ugly rule defined somewhere I think it may be possible to extend Litmus to allow restricting testruns by locale, and leave English as the only possible locale to run functional tests on. That would still be a workaround though... Yes. In the context of our testing method, the English mentioned here can be actually Any language. But I suspect it will be easier than Option 2 to design and implement in Litmus code :) So by this method, we will have: - L10n Test Run for all locales in which each test case canbe considered as fully tested only if it has been run in all locales. - Functional Test Run for any of the locales in which each test case canbe considered as fully tested if only it has been run in any of the locales. But as Petr mentioned the option 1 may discourage, at least to some extent, people who does not know English. Besides option 1 brings us back to the style of testrun dividing. So for the choice of workaround, I prefer option 2. 2. Remove the locale setting in the run tests dialog and ignore it as we suggest to ignore the build id. Note that the l10n tests are duplicated in the subgroups: Advantages: + easy to implement + close to what we have now + the l10n tests are localized without hacking Litmus server code + allows to create extra l10n test case for a particular language (is it an advantage? creates a mess?) Disadvantages: + it might be hard to maintain the l10n tests because you need to monitor changes in the en group + people will see l10n test cases also for another languages + it will be hard to see how many l10n test cases were finished in the various localizations; you would need to enter the run tests dialog with different setting Just found an alternative way is Reporting - test runs can show it more easily. Instead of percentage of finishing, detailed execution record of each test case in group/subgroups are summarized there. This is indeed very close to what we have now and could be considered as a possible temporary workaround. It doesn't sound nice in the long-term though. 3. Do some more changes in Litmus (suggested by Rimas): a) add extra checkbox into the test case edit dialog (os somewhere); it will mark the test case as language specific or language independent b) count the statistic of finished test cases according to the check box; locale will be ignored for language-independent tests; c) allow to transparently localize test cases = you will see different text in different locales (can be done later) d) show statistic of finished l10n tests per locale on a single page (can be done later) Advantages: + clear solution + it is on the way where we want to go, see http://wiki.documentfoundation.org/Litmus_TODO + will help to keep l10n tests in sync Disadvantages: + needs hacking in litmus (developer and time) My opinion: --- I very like the 3rd proposal (created by Rimas). I think that it is worth to spend some time with hacking Litmus. We will profit from this in the future a lot. Rimas, what do you think about it? Would you have time and appetite to look into it? I of course agree that it's the cleanest solution. But I'm not sure how much time and skill I'll have to implement this. In any case, I think I'll try to at least add that checkbox rather sooner than later, that would be a good start already. :) Rimas, thanks a lot for doing them! :) I have one small question lingering on my mind though: is it better to add that checkbox to testcases or some higher hierarchical component (subgroup, group?) I'm quite sure a testcase makes most sense, but just want to check with you guys. IMHO, attach it to test case is the most safe way from the design's view, which gives us the most flexibility of expanding the system. But in UI level, subgroup maybe a better place because the test cases will be created by different people, some of whom may not notice the usage of the checkbox for each test case. Once the checkbox is checked in an inappropriate manner, it will be hard to manage and the test statistics is potentially calculated in a wrong(or tricky) way. Keeping the checkbox in a subgroup could be more controlable because of the stability of subgroups contents and structure. Test case authors would only need to put their new test cases in the correct subgroup like we are doing now and nothing else. A potential problem is if we will have mixed
Re: [Libreoffice-qa] Test Structure in Litmus
Hi Petr, Thanks for the detailed summary, I hope I can review this before sending mine:) If I understand you correctly, scheme D ignores a requirement, doesn't it? 7. Test run creation: + requirements: + create test run for a given component (only Writer modified) In the latest mail I just replied to Test case naming I shared a similar idea. I can see the main differences of ours focused on 2 points: 1. Do we need components split in L10n tests? I have ever considered the leraning curve when l10n and function test doesn't have a consistent scheme. But from reconsidering, it is probably natural for a user to see such language split when clicking a Localizatoin test group. Plus it reduces the test group size significantly, which is my favorite as well. So I give up this concern :) Another question is still about how many cases will we have. One thing I didn't mention is with this scheme, when we create a test case we need to review all the cases to make sure they were not created before. Currently we only have 23 cases, it is okay if we review all of the cases summaries when creating a new one. But if we get even more cases ( 50 for example), it becomes painful and hard to manage. 2. Do we need an extra feature branch? It is pretty fine for me to have another branch, I did actually prefer to leverage branches in a reasonable scale :) Best wishes, Yifan On Tue, Nov 15, 2011 at 07:44:59PM +0100, Petr Mladek wrote: Hi, I am going to summarize the ideas from the mail Test case naming. I am sorry for the long mail. The problem is not easy. There are many ideas. I tried to sort them from different point of view. I hope that I did not miss anything important and it will help to find the final conclusion. -- Let's start by a short summary. Litmus allows to split test cases by: + product + branch + testgroup + subgroup We want to split test cases by: + priority (P1,P2,P3,P4) + component (General,Base,Calc,Draw,Impress,Math,Writer) + meaning (functional, l10n, feature) + l10n tests in languages (en,de,fr,it,ja,.) + platform (Linux,Windows,MAC) We need to take care of the following scenarios: + maintenance: + creating test cases + updating test cases + localization + branch for new release + test run for new release/build + doing tests + analyzing results --- Let's look at some conditions: 1. priority: + why + reduced test run for bugfix releases + overal prioritization of the work + main requirements: + allow to create test run for given priority + easily run tests according to priority + easily see test results by priority 2. component: + why + reduced test run if only writer is modified + helps to split work by expertize + main requirements: + allow to create test run for given component + allow to run tests by component 3. meaning: + why + functional tests do not depend on localization + the list of language dependent tests might be different for each language + feature tests are not well described; are done only once by one or few dedicated people + requirements: + function test appears only once in test run; can be approved by any user with any locale and any platform + language dependent tests must be approved more times in all locales + test results must clearly show what tests were done; functional tests are important by priority; l10n tests are important by priority and language; e.g. German is more important than Czech because there are more active users 4. platform: + why: + some tests might depend on platform; for example the open/save dialogs look different way on Linux,Windows, and MAX; also KDE vs. GNOME + main requirements: + these tests must be done on all affected platforms + test results must clearly show where it was tested 5. Maintenance: + problems: + searching for a particular test case + put test cases into the right position + comparing en-US template with localized test cases + localizing the whole bunch of test cases into new language +
Re: [Libreoffice-qa] Test case naming
Hi Rimas and Petr, Thanks for the great discussion :) I appreciate Rimas' new scheme with some questions remaining. Let me take the opportunity to give a mid term summary and share my thoughts based on the new scheme. 1. Firstly through the complicated mail thread, I feel it is necessary to clarify the definition of test types helping us to have a common sense: - Function Regression test (Language Neutral Tests) It is language neutral and we only need to run it once in any of the languages. The localization for this kind of tests could be word-by-word. - L10N Regression test (Language Dependent Tests) It is language dependent and we need to run them in all language. The localization for this kind of tests could be slightly different language versions based on the libreoffice features. - Feature test It contains new features testing for each release of libreoffice. Test cases are added on demand in each new release, and the test cases are not reusable in the next release. 2. In the new scheme pdf, it looks feature tests are not included while a legacy Basic Function Test is there. Just to make things more clear, I am udpating Rimas' chart a bit: Basic Function Test - Feature Tests General Function Test - Function Regression Tests I believe we didn't really have misunderstanding on this but the legacy problem, right? 3. Do we want components split in L10n test? I don't have a definite answer for this yet but prone to keep the components. On one hand, at least for impress and calc, we have some specific areas to cover as tempalte, slide show in multiple language, calc cell format etc. On the other hand, the consistent structure may make people's learning curve more mild. 4. The amount of how many localization is another concern. In the new scheme, for each priority we will have 7 components * 4 locale * 4 priority (112) groups. If I understand it right, when there are 10 locales, the number will be 2800 ?! Image we have 112 test groups in the first section when running test: http://wiki.documentfoundation.org/images/7/77/Litmus_submit-de.png While another choice to be considered could be switching the position of test types and components, something like: / p1-en-Feature Tests - 7 components (en) / p1-xx-Feature Tests - 7 components (xx) / p1-en-L10n Regresstion Tests - 7 components (en) / p1-xx-L10n Regresstion Tests - 7 components (xx) master - p1-en-Function Regresstion Tests - 7 components (ATM: multi-language case) \ p1-xx-Function Regresstion Tests / Besides it will reduce the groups size to 3*locale*priority, by the fact we can't select subgroups when creating test run (I didn't notice it before, thanks Rimas), we can create a test run for particular test types, which might be more likely to happen than creating test runs for particular components. It would be more clear of the strategy by comments Rimas' origin plan: * create testruns based on priority, locale, component, or any combination of these - create testruns based on priority, locale, test type, or any combination of these * create a single catch-all testrun for a single version of LibO * share the same General Functional Tests subgroup between testgroups designated for different locales (that is, you would create this subgroup once, and add it to all locale groups) - share the same components subgroup in Function Regression Tests testgroups. * drop 75% of the testgroups (there would be about 28 of them initially) if/when proper testcase localization is implemented, still not rendering the remaining testgroups useless. - This is the thing I am not pretty sure. Will this switch cause unhandlable problems for our potential translation system ? 5. Another critical question raised about different testing execution times of Function and L10N regression. 2011.11.15 17:30, Petr Mladek rašė: I didn't say they have to be translated word-to-word. They should be *localized*, and I would expect a localized testcase to suggest localized number and date formats and other stuff. I see. Well another question is how it appears in the test run. The language independent tests are proceed once. The language dependent tests need to be proceed for every localization. The test case transaltion can be considered into 2 layers in current test strategy: - For Function Regression Tests (Language Neutral Tests) Review the definition on the top of the mail, the translation version case id is supposed to be the same with its English version. So all of the people run the same test cases but read them in different language. - For L10n Regression Tests (Language Dependent Tests) Review the definition on the top of the mail, the translation version case id is not the same one as its English version. So all of the test cases will be
Re: [Libreoffice-qa] Test Structure in Litmus
Hi Petr, Hehe, I missed this mail before I started summarizing today. I'll have a review for sure :) Best wishes, Yifan On Tue, Nov 15, 2011 at 07:44:59PM +0100, Petr Mladek wrote: Hi, I am going to summarize the ideas from the mail Test case naming. I am sorry for the long mail. The problem is not easy. There are many ideas. I tried to sort them from different point of view. I hope that I did not miss anything important and it will help to find the final conclusion. -- Let's start by a short summary. Litmus allows to split test cases by: + product + branch + testgroup + subgroup We want to split test cases by: + priority (P1,P2,P3,P4) + component (General,Base,Calc,Draw,Impress,Math,Writer) + meaning (functional, l10n, feature) + l10n tests in languages (en,de,fr,it,ja,.) + platform (Linux,Windows,MAC) We need to take care of the following scenarios: + maintenance: + creating test cases + updating test cases + localization + branch for new release + test run for new release/build + doing tests + analyzing results --- Let's look at some conditions: 1. priority: + why + reduced test run for bugfix releases + overal prioritization of the work + main requirements: + allow to create test run for given priority + easily run tests according to priority + easily see test results by priority 2. component: + why + reduced test run if only writer is modified + helps to split work by expertize + main requirements: + allow to create test run for given component + allow to run tests by component 3. meaning: + why + functional tests do not depend on localization + the list of language dependent tests might be different for each language + feature tests are not well described; are done only once by one or few dedicated people + requirements: + function test appears only once in test run; can be approved by any user with any locale and any platform + language dependent tests must be approved more times in all locales + test results must clearly show what tests were done; functional tests are important by priority; l10n tests are important by priority and language; e.g. German is more important than Czech because there are more active users 4. platform: + why: + some tests might depend on platform; for example the open/save dialogs look different way on Linux,Windows, and MAX; also KDE vs. GNOME + main requirements: + these tests must be done on all affected platforms + test results must clearly show where it was tested 5. Maintenance: + problems: + searching for a particular test case + put test cases into the right position + comparing en-US template with localized test cases + localizing the whole bunch of test cases into new language + requirements: + balance the number of branches, testgroups, subgroups, and test cases in subgroup + must be easy to copy test cases for new language + often switched criteria must be in high level; for example, do people switch more often between priority or category or language when they work with the test cases? 6. Branch for new release: + requirements: + must be easy to do the branch + must be easy to see all tests for the given release 7. Test run creation: + requirements: + 1 or very small number of active test runs because it is compilcated to switch between them and analyze results from many test runs + create test run for a given priority (basic vs. full test) + create test run for a given component (only Writer modified) 8. Doing tests: + requirements: + must be easy to search what needs testing + must be possible to sort testing by priority and component + priority is more important than component because it does not help to
Re: [Libreoffice-qa] Test case naming
Hi Rimas, Petr and all, Thanks for the nice discussions, my ideas as follows :) On Sat, Nov 12, 2011 at 10:46:48PM +0200, Rimas Kudelis wrote: This is thus irrelevant, cause there's nothing to modify. :) 2. I am not sure what is the meaning of the numbers 001, 002, 003. It looks like they define the order in which we should process the test cases. If this is true, it does not look ideal: + if we do another important test case, we will need to rename all less important test cases to keep the right order + test cases will be checked by many people; we can't force them to do it in an exact order; the result would be that all people will test the same test case in parallel Hmm, we need to encourage people to do the test cases in random order. We still should somehow prioritize the test cases. I personally don't like current naming with ugly prefices at all, but it's Yifan's call, and I suppose he has a good reason for that naming scheme. However, I'm afraid we can't randomize testcase order by default. Currently, this would probably have to be done manually each time a relevant subgroup is updated. That would be a PITA. On the other hand, it's not really that bad. You can still run the tests in random order, but you will always see them in the specified order. From my understanding, I neither mean to encourage users to run cases in a specific order. The ids were originally being there since I got involved. I think it still makes sense to kept them there mainly because of the syncing problem of test cases in different languages. For example: #EN - w001 xxx is supposed to have the same content with (but in different version of language): #FR - w001 xxx #DE - w001 xxx #pt-BR - w001 xxx These give us reasonable information showing which cases are supposed to be synced to each other (they may not have exact same steps of testing because of the diversity of language settings, but they should test the same areas). So for current testing organization, I think these ids are still playing their role in L10N test branches. Otherwise, syncing of cases could be painful. Meanwhile in Function Regression testing branch, by the fact we are now using a single case to host all language versions of test case, it may not make sense to keep the id any more. I suggest to split test cases into several levels by priorities: P1 - highest: used for very basic tests, e.g. app can be installed; it starts; is able to load/save some test documents; so it a kind of smoketest P2 - high: test very common functionality that is used by most users. e.g. able to write text, insert picture; draw elements; create table; use function in calc; create graph, run presentation P3 - medium: test common functionality that is used by typical a bit experienced office user, e.g. create borders around tables; do animation between slides; modify text style; modify master slide page; P4 - low: test functionality for hi-tech users, e.g. writing macros, using Calc solver, complex operations with data bases I suggest to use the names: p1g - summary of a P1 global test p1w - summary of a P1 Writer test p2g - summary of a P2 global test p2w - summary of a P2 Writer test Then we will have all p1 test cases listed before p2 test cases. What do you think? Prioritizing is probably a good idea, but like I said, random order would require some Litmus modification. While certainly possible (and probably quite easy), I'm not sure that is what we indeed want. Actually it is a great idea to have priority here, at least they are helpful for us to define subset of test runs. For example, we can create smoke test runs by select P1 only test cases when creating a test run from a full regression branch containing all cases. That is to say, even before we sort out how order of the test cases could be implemented, we can always create specific test runs on demand via the information of the priority tags. i.e We can define smoke test runs, basic test runs, full regression test run by selecting cases, in which all the test cases are divided physically in test runs and sorting becomes trivial. But it still makes sense if we can do some hacking in ordering, which could save much of the test cases selection effort and makes the system more flexible. :) At the same time, as stated before, for L10N test case branch, we probably still want test case id until we have some better solution for translation syncing problem. Best wishes, Yifan ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change
