Re: Test Cases wiki questions
Nicholas, I've finished getting the rest of the test cases added to the spreadsheet. The only thing I have left to do is add the empty application tests. I was going to enter the empty tests more as a placeholder on the spreadsheet. But I think I'll finish that up next week. Have a great weekend everybody! Aaron On 02/06/2012 05:58 PM, Nicholas Skaggs wrote: Aaron, yes I noticed someone cataloguing everything in there :-) It makes it alot easier to rewrite them by just moving them to the rewrite tab and then finally to completed. Thanks! Nicholas On 02/06/2012 07:54 PM, Aaron Albright wrote: Hey Nicholas, I've been working on putting all of the test cases into the spreadsheet under the tab 'all test cases from wiki'. I'm not quite done yet--but hopefully this will help in updating them. Tomorrow I will work on adding the test cases I haven't gotten to yet. Is it helpful for me to do that? Thanks! Aaron albrigha On Mon, 2012-02-06 at 14:18 -0500, Nicholas Skaggs wrote: Andrew, we're currently in process of updating all of our wiki test cases. See this spreadsheet: https://docs.google.com/spreadsheet/ccc?key=0AgtV30nnv18edFQzNVB4S2duOWNOT05zaHo3S0pNekE The idea is to go thru each of the test cases, edit them on the doc and ask the list for review. Once someone else has verified your work, we've been updating the wiki with the results. You can see the ones we've completed under the completed test cases tab. I would love for you to update the testcase you mention: Live Session Persistence Case ID: dls-003. It needs to be re-written, and the case id should be TC-DLS-003 to conform to our naming convention. You can make your changes in the spreadsheet I mentioned above and we'll review. Please do have a look and update any other test cases you see/find on the wiki using the same process. Thanks! Nicholas On 01/29/2012 09:58 AM, Charlie Kravetz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, 29 Jan 2012 14:52:04 +1030 Andrew McDonnell wrote: Hello, I'm new here, still feeling my way around :-) is it possible to file a bug against the test cases and wiki? or are we allowed to simply edit this? I have some questions / issues: On https://wiki.ubuntu.com/Testing/ISO/FAQ 'The Live CD installer crashes -- Can be a wide range of issues, but bugs can be filed agaist ubiquity and will be triaged from there. ' The link to ubiquity in the above - https://bugs.beta.launchpad.net/ubuntu/+source/ubiquity/+bugs - is broken for me with a DNS failure On http://testcases.qa.ubuntu.com/Install/DesktopLiveSession the procedure for actually enabling persistence is a little ambiguous I would like to extend it to something along the lines of "hit a key when the little icon shows at the bottom of the screen after CD booting starts, then press F6, then you have to hit ESC to hide the little ACPI options menu, then you actually are focussed on the kernel command line and can hit SPACE followed by persistence" Or am I over doing it i.e. is this sort of thing assumed knowledge for testers? Third: https://wiki.ubuntu.com/Testing/Activities#Bugs --> https://help.ubuntu.com/community/ReportingBugs specifically mentions a separate Kubuntu page - is anyone aware of a separate Xubuntu page? Andrew For the most part, Xubuntu follows the Ubuntu processes. All bugs are reported directly to launchpad the same as for Ubuntu. Kubuntu, on the other hand, does not report most bugs directly to launchpad, but to upstream. Documentation for testing and filing bugs for Xubuntu is : https://wiki.ubuntu.com/Xubuntu/Testing https://wiki.ubuntu.com/Xubuntu/Bugs - -- Charlie Kravetz Linux Registered User Number 425914 [http://counter.li.org/] Never let anyone steal your DREAM. [http://keepingdreams.com] -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBAgAGBQJPJV6hAAoJEFNEIRz9dxbAn40IAIlvNjGqlw41q9LqdVTNu5Mo 87xgAO6wJKLBdSmuHJUTX41GClKyo0OOQdy9iehZi6XiQKN0m5QTQ108syJUA/VB 61fCWJUIXYVNfT7penUmZXha5K0obhJNuHJP3FIy5nSo9+mnamhjjLBeHbQMQ+Pm aCsDR+bTguK3JKHYEVxP8JIAUfWRPu/u11gyQw/WxeuDyeof2JWiwcw/69Zy5N/Z DCJ2FSzqlYmMIqeIDb3QYFC0EJ/+tpw4DDnOSQC1NWl/CXTPBpy8F2hqrV+IoYzD aHQ6lSPz0GiuuNdEroyILM752OEEVUdI+SDvy6leSBrj7xbanRsNlXIhCAC3S+Y= =hPx3 -END PGP SIGNATURE- -- Ubuntu-qa mailing list Ubuntu-qa@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
Re: Test Cases wiki questions
Hey Nicholas, I've been working on putting all of the test cases into the spreadsheet under the tab 'all test cases from wiki'. I'm not quite done yet--but hopefully this will help in updating them. Tomorrow I will work on adding the test cases I haven't gotten to yet. Is it helpful for me to do that? Thanks! Aaron albrigha On Mon, 2012-02-06 at 14:18 -0500, Nicholas Skaggs wrote: > Andrew, we're currently in process of updating all of our wiki test > cases. See this spreadsheet: > > https://docs.google.com/spreadsheet/ccc?key=0AgtV30nnv18edFQzNVB4S2duOWNOT05zaHo3S0pNekE > > The idea is to go thru each of the test cases, edit them on the doc > and ask the list for review. Once someone else has verified your work, > we've been updating the wiki with the results. You can see the ones > we've completed under the completed test cases tab. > > I would love for you to update the testcase you mention: Live Session > Persistence Case ID: dls-003. It needs to be re-written, and the case > id should be TC-DLS-003 to conform to our naming convention. You can > make your changes in the spreadsheet I mentioned above and we'll > review. Please do have a look and update any other test cases you > see/find on the wiki using the same process. Thanks! > > Nicholas > > On 01/29/2012 09:58 AM, Charlie Kravetz wrote: > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA1 > > > > On Sun, 29 Jan 2012 14:52:04 +1030 > > Andrew McDonnell wrote: > > > > > Hello, > > > > > > I'm new here, still feeling my way around :-) > > > > > > is it possible to file a bug against the test cases and wiki? or are we > > > allowed to simply edit this? > > > > > > I have some questions / issues: > > > > > > > > > > > > On https://wiki.ubuntu.com/Testing/ISO/FAQ > > > > > > 'The Live CD installer crashes -- Can be a wide range of issues, but bugs > > > can > > > be filed agaist ubiquity and will be triaged from there. ' > > > > > > The link to ubiquity in the above - > > > https://bugs.beta.launchpad.net/ubuntu/+source/ubiquity/+bugs - is broken > > > for > > > me with a DNS failure > > > > > > > > > > > > On http://testcases.qa.ubuntu.com/Install/DesktopLiveSession > > > > > > the procedure for actually enabling persistence is a little ambiguous I > > > would > > > like to extend it to something along the lines of > > > > > > "hit a key when the little icon shows at the bottom of the screen after CD > > > booting starts, > > > then press F6, then you have to hit ESC to hide the little ACPI options > > > menu, > > > then you actually are focussed on the kernel command line and can hit > > > SPACE > > > followed by persistence" > > > > > > Or am I over doing it i.e. is this sort of thing assumed knowledge for > > > testers? > > > > > > > > > Third: > > > > > > https://wiki.ubuntu.com/Testing/Activities#Bugs --> > > > https://help.ubuntu.com/community/ReportingBugs specifically mentions a > > > separate Kubuntu page - is anyone aware of a separate Xubuntu page? > > > > > > > > > Andrew > > > > > > > > For the most part, Xubuntu follows the Ubuntu processes. All bugs are > > reported directly to launchpad the same as for Ubuntu. Kubuntu, on the > > other hand, does not report most bugs directly to launchpad, but to > > upstream. > > > > Documentation for testing and filing bugs for Xubuntu is : > > > > https://wiki.ubuntu.com/Xubuntu/Testing > > https://wiki.ubuntu.com/Xubuntu/Bugs > > > > - -- > > Charlie Kravetz > > Linux Registered User Number 425914 [http://counter.li.org/] > > Never let anyone steal your DREAM. [http://keepingdreams.com] > > -BEGIN PGP SIGNATURE- > > Version: GnuPG v1.4.11 (GNU/Linux) > > > > iQEcBAEBAgAGBQJPJV6hAAoJEFNEIRz9dxbAn40IAIlvNjGqlw41q9LqdVTNu5Mo > > 87xgAO6wJKLBdSmuHJUTX41GClKyo0OOQdy9iehZi6XiQKN0m5QTQ108syJUA/VB > > 61fCWJUIXYVNfT7penUmZXha5K0obhJNuHJP3FIy5nSo9+mnamhjjLBeHbQMQ+Pm > > aCsDR+bTguK3JKHYEVxP8JIAUfWRPu/u11gyQw/WxeuDyeof2JWiwcw/69Zy5N/Z > > DCJ2FSzqlYmMIqeIDb3QYFC0EJ/+tpw4DDnOSQC1NWl/CXTPBpy8F2hqrV+IoYzD > > aHQ6lSPz0GiuuNdEroyILM752OEEVUdI+SDvy6leSBrj7xbanRsNlXIhCAC3S+Y= > > =hPx3 > > -END PGP SIGNATURE- > > -- Ubuntu-qa mailing list Ubuntu-qa@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
Re: ISO manifest inconsistent with installed packages
Well I'm happy to report that the installed applications from the LiveCD do indeed match the application manifest! Sorry about that. A On Tue, Dec 6, 2011 at 7:57 AM, Aaron wrote: > I did not select to install any updates or third party applications, but > I'm going to check this again on a LiveCD and see how the differences look. > > Thanks for your time! > > Aaron > > > On Tue, Dec 6, 2011 at 12:55 AM, Jean-Baptiste Lallement < > jean-bapti...@ubuntu.com> wrote: > >> On 12/06/2011 01:45 AM, Aaron wrote: >> >>> I am looking at how consistent the ISO manifest is with the actually >>> installed packages in Precise. The basic steps I did are: >>> >>> 1. Download ISO from >>> http://cdimage.ubuntu.com/**daily-live/current/<http://cdimage.ubuntu.com/daily-live/current/> >>> (download with date stamp for today) I did a install (not live cd) in >>> VirtualBox. >>> >>> 2. Download the manifest for the ISO (for i386 below is the path) >>> (download with date stamp for today) >>> >>> wget >>> http://cdimage.ubuntu.com/**daily-live/current/precise-** >>> desktop-i386.manifest<http://cdimage.ubuntu.com/daily-live/current/precise-desktop-i386.manifest> >>> >>> Just to clear a confusion here, the manifest on cdimage.u.c lists the >> contents of the live filesystem, not the installed filesystem. >> >> Content of the live filesystem and a freshly installed system are >> different because, some packages on the Live CD are only useful to the Live >> CD and the installer, only the language (and langpacks) the user selects is >> installed, additional software can be installed (in your case virtualbox >> packages, but also flashplugin, codecs, updates, additional langpacks not >> on the CD, i386 packages due to multiarch support if you're installing on >> amd64 and need i386 libs, ...) >> >> >> -- >> Jean-Baptiste >> irc: jibel >> >> >> -- >> Ubuntu-qa mailing list >> Ubuntu-qa@lists.ubuntu.com >> Modify settings or unsubscribe at: https://lists.ubuntu.com/** >> mailman/listinfo/ubuntu-qa<https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa> >> > > -- Ubuntu-qa mailing list Ubuntu-qa@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
Re: ISO manifest inconsistent with installed packages
I did not select to install any updates or third party applications, but I'm going to check this again on a LiveCD and see how the differences look. Thanks for your time! Aaron On Tue, Dec 6, 2011 at 12:55 AM, Jean-Baptiste Lallement < jean-bapti...@ubuntu.com> wrote: > On 12/06/2011 01:45 AM, Aaron wrote: > >> I am looking at how consistent the ISO manifest is with the actually >> installed packages in Precise. The basic steps I did are: >> >> 1. Download ISO from >> http://cdimage.ubuntu.com/**daily-live/current/<http://cdimage.ubuntu.com/daily-live/current/> >> (download with date stamp for today) I did a install (not live cd) in >> VirtualBox. >> >> 2. Download the manifest for the ISO (for i386 below is the path) >> (download with date stamp for today) >> >> wget >> http://cdimage.ubuntu.com/**daily-live/current/precise-** >> desktop-i386.manifest<http://cdimage.ubuntu.com/daily-live/current/precise-desktop-i386.manifest> >> >> Just to clear a confusion here, the manifest on cdimage.u.c lists the > contents of the live filesystem, not the installed filesystem. > > Content of the live filesystem and a freshly installed system are > different because, some packages on the Live CD are only useful to the Live > CD and the installer, only the language (and langpacks) the user selects is > installed, additional software can be installed (in your case virtualbox > packages, but also flashplugin, codecs, updates, additional langpacks not > on the CD, i386 packages due to multiarch support if you're installing on > amd64 and need i386 libs, ...) > > > -- > Jean-Baptiste > irc: jibel > > > -- > Ubuntu-qa mailing list > Ubuntu-qa@lists.ubuntu.com > Modify settings or unsubscribe at: https://lists.ubuntu.com/** > mailman/listinfo/ubuntu-qa<https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa> > -- Ubuntu-qa mailing list Ubuntu-qa@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
Re: QA tasks available
I'd be interested in starting on ISO Testing and task 1. I think it would give me some good initial exposure to the overall picture--installed applications, some testing priority and making some ISO smoke test--test cases. If that sounds okay? Is there a notification for when packages with changes are submitted to the code base? Or some other process for knowing what packages have been added and changed? Has there been any initial discussions on testing priority? I'm sure I can come up with an initial list and we can start from there if not. I'd be interested in having a clear picture of what has changed between builds/releases and help determine what should be included in the smoke testing, etc. Getting a list of the packages isn't hard so I was thinking about how to determine testing priority and making the test cases, etc. Unless someone is already on it or if you'd like me to look at something else instead... Thanks! Aaron On Mon, Dec 5, 2011 at 8:24 AM, Gema Gomez wrote: > Dear QA Team, > > as promised, here it is a list of tasks that need to be done and we are > in the process of doing that you could own if you have the time: > > - ISO testing tasks > ( > https://blueprints.launchpad.net/ubuntu/+spec/other-p-builds-smoke-testing > ): > > 1) Compile a list of applications that are installed by default by the > ISO installers (one for Desktop, one for Server) and propose two or > three basic test cases that could be run post install giving us basic > confidence that the ISO is good for further testing (i.e. compile a list > of post-install smoke tests that we could run with Jenkins). >- This task is not about generating code, but about thinking of what > packages of the ones installed are important and worth testing in a > daily test suite. We could split it in different tasks for different > people if we generate first a list of apps that we can use for the > generation of test cases. > 2) We need to fix the existing test cases in the tracker and convert > them to a better, more understandable format. Basically we need to > convert them to unambiguous and meaningful test cases. Some of them are > redundant, some of them are too long to be just one test case, some > others do not make sense anymore. This is a tidy up task that needs to > be done. > > - Metrics > (https://blueprints.launchpad.net/ubuntu/+spec/other-p-qa-metrics): > 3) I have some tasks here that could use some help. We need to look at > the codebase of Ubuntu main and see how to instrument the code so that > we can start generating code coverage metrics. This is about compiling > the Ubuntu code with gcov and generating binaries that can be used > (still to be seen how to install them) for this end. >- This task requires code in-depth knowledge and familiarity on how > things are built and can be changed to build in a different way. We > should decide where to start instrumenting and why. > > 4) Look into how to do test escape analysis with launchpad. TEA is an > analysis that will tell us, after Precise, if we missed some problems > that were found by someone after we did our testing and that should help > us understand whether we should be adding new test cases in those > "missed" areas or not. > > 5) Gather test cases from defects. This is about making a list of > defects that have been fixed for Oneiric and that have a set of steps to > reproduce the problem that needs to be gathered and written into a > proper test case. > > - Test Case Management System > ( > https://blueprints.launchpad.net/ubuntu/+spec/other-p-qa-test-case-management-tool > ) > 6) Still not available, but when it is, review and give feedback > about litmus and its usability. Also help decide how to configure it to > make it more suitable for the Ubuntu community testing. > > > - QA Backlog tasks > (https://blueprints.launchpad.net/ubuntu/+spec/other-p-qa-backlog) > 7) Review and change the wiki to reflect the new approach to QA. > > > Please, bear in mind that since we don't have the test case management > tool up and running yet, we need to keep our test cases in text files or > open office documents (prefereably spreadsheets) for now. As soon as we > have chosen a tool to handle them, we will be using that. > > I have added a template at the bottom of the test cases page, feel free > to use it for your newly generated test cases: > https://wiki.ubuntu.com/QATeam/TestCase > > You can also modify it to contain a link to the old test case whenever > you are improving an existing test case. > > > Let us know which tasks you are interested in and I will be mapping > tasks in the blueprints to people, so that we keep track of what > everyone's doing an
well, here's my hello as well
I've been on the list for awhile but haven't been very active in the community for a year or so. I was on the HP Linux Imaging and Printing project for 5 years doing QA and Technical Support. I would like to now find a project or area of focus so that I can continue to be active in the community. If there is an area or team that would have me, I'd love to help with the next release. Here is my linkedin profile for more of my background: http://www.linkedin.com/in/albrigha Ideally I'm looking for a specific project to be assigned that I can help with. So please feel free to be in touch on how I can participate! Thanks for your time! Aaron -- Ubuntu-qa mailing list Ubuntu-qa@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-qa
