Sridhar Dhanapalan <[email protected]> writes: > I am keen to explore ways to improve the quality and delivery time of > code. Is there any work being done to automate testing of code?
Yes, I've been working on this whenever I have time. We already have a data store test suite [3-5] and a semi-automated set of tests for the WLAN part of our NetworkManager UI (Neighbourhood, Frame). The remainder of this mail will focus on UI tests for Sugar (and Activities). The public D-Bus API for Sugar is quite limited; the datastore test suite already covers the important part. Most of the rest should be replaced by other protocols that have since been agreed on by other desktop environments. We could test the public (Python) API provided by sugar-toolkit, but I haven't looked yet into that yet. There may be some promising targets in the non-UI part of sugar-toolkit (e.g. the Activity / Journal / Journal Entry Bundle related code). However, automated UI tests would exercise at least some of the more commonly used code paths in sugar-toolkit. The basic framework for Sugar UI tests is available [6], but not of much use with current upstream Sugar. There are two problems: 1. Ensuring the "accessibility" helpers are running. The test suite already enables them, but they need to actually get started. This is done automatically by the upstream version of gnome-session, but sugar-toolkit ships an ancient copy that won't start the helpers. Starting them manually was problematic. For this reason (in addition to generally being a good idea), I've ported Sugar over to upstream gnome-session. Upstream now provides [7] API to run alternative (i.e. non-Gnome) sessions. A WIP patch for using the new API is available [8] in my sugar clone on git.sl.o. The remaining blockers are the Shutdown and Reboot actions. Upstream gnome-session currently doesn't [9] provide API to trigger shutdown or reboot from outside of gnome-session, either without showing a dialog (for shutdown) or at all (for reboot). I've cleaned up a patch for this recently in the hope of pushing it forwards (the RFE was filed over three years ago), but upstream doesn't seem interested in it. The systemd support landed in sugar-toolkit doesn't help with this, BTW. It's used with our own copy of gnome-session and will be called after we already did the session part, closing all activities. With upstream gnome-session, we wouldn't get that chance (nor need to, if upstream just added the two small functions we need). 2. The Zoom Views (i.e. Home View, Group View, Neighbourhood) and the Journal are based on hippo-canvas, which doesn't support the GTK/Gnome accessibility framework. This means pretty much all of the core functionality isn't testable by this approach, either because it uses hippo-canvas or because it can only be triggered via some other part of the UI that uses hippo-canvas. This has been the major blocker for a long time. However, Simon recently ported sugar from hippo-canvas to plain GTK (thanks a lot, Simon!) as a milestone of the GTK3 migration and during the last Development Team meeting [10] we agreed on a plan that enables us to make use this intermediate result in upstream Sugar. I still haven't seen the cleaned-up patches, but Caspar and me talked about it extensively last weekend and we may simply accept the patches with only minor cleanups. We'd fully expect major bugs, but the hope is that a) the accumulated amount of manual testing (we're early in the development cycle for 0.98) and b) the ability to write automated tests would more than offset this risk. If we start right away, we might even have some simple test cases during the GTK3 migration, speeding up testing each individual change and thus the entire development phase. And once we have automated tests, we can more confidently clean up the current Zoom Views related code. The Views all pretty much provide the same functionality in terms of layout, but they're doing it in different ways. A single (set of) base class(es) used by all Zoom Views would help a lot to make the code more manageable and prevent Zoom View specific bugs. Like the GTK2→GTK3 migration, this is a major change. But with sufficient test coverage, the risk is low. > We recently had some university students working with us to create an > activity [1], and they were using the Robot Framework [2]. For activities, we're in a much better shape because virtually none of them use hippo-canvas. With some glue, you may be able to start right away - as your students apparently have done. I'd be quite interested in their usage of the Robot Framework, as I've taken a look at it before and it didn't seem to do much that would be useful enough in the context of Sugar UI testing (or even any of the automated testing I do for my customers). Neither AU#634 nor the NoteBoard source code gives any hint about the automated testing done for this activity. For implementing the test cases, taking a look at Strongwind [9] may be a good idea. It's similar to dogtail, but apparently easier to use. Just for completeness, I'd like to point out that there are alternatives to the GTK/Gnome "accessibility" approach, e.g. XTEST based tools like xautomation [10] or hooking into the Python side like SugarBot [11] does. However, they come with a significant maintenance overhead: tests implemented using low-level functionality need to be updated on many minor UI changes and Python side hooking is likely to break during the GTK2→GTK3 migration. Sascha > [1] https://dev.laptop.org.au/issues/634 > [2] https://code.google.com/p/robotframework/ [3] https://patchwork.sugarlabs.org/patch/708/ [4] https://patchwork.sugarlabs.org/patch/642/ [5] https://patchwork.sugarlabs.org/patch/640/ [6] https://people.sugarlabs.org/silbe/git/sugar-ui-tests.git [7] https://bugzilla.gnome.org/show_bug.cgi?id=633276 [8] http://git.sugarlabs.org/~sascha_silbe/sugar/silbe/commits/gnome-session [9] https://bugzilla.gnome.org/show_bug.cgi?id=575880 [10] http://meeting.sugarlabs.org/sugar-meeting/meetings/2012-06-26 [11] https://medsphere.org/community/project/strongwind [12] http://hoopajoo.net/projects/xautomation.html [13] https://wiki.sugarlabs.org/go/BugSquad/SugarBot -- http://sascha.silbe.org/ http://www.infra-silbe.de/
pgprQrmk5nSe3.pgp
Description: PGP signature
_______________________________________________ Testing mailing list [email protected] http://lists.laptop.org/listinfo/testing
