On 05/ 4/11 03:13 AM, John Rice wrote:
Hi Brock, after discussion with Michal we have some comments below,
which should help clarify things:
JR
On 27/04/2011 23:34, Brock Pytlik wrote:
On 04/27/11 04:43 AM, Michal Pryc wrote:
Hi,
The webrev is available at:
http://cr.opensolaris.org/~migi/27_04_2011_automated_tests_18072/
Hi Michal, thanks for taking this on and giving us an advanced look
at a gui test suite that's coming.
This fixes the automated tests, which works with ldtp 2.x.
Please note that in the gate there is ldtp 1.x version, but we need
to open separate bug to ship 2.x. Those tests will not work with
ldtp 1.x, that is why we do need to remove the ldtp 1.x after
building the gate (see below).
Which means that these changes won't land until we ship ldtp2.x,
correct?
I would prefer to get the changes landed now as the current set of gui
tests in the gate are broken. Conny who is doing the testing can
install the required version of ldtp herself, but I'd like her to be
working off tests in the gate.
Michal will open up a separate bug to get ldtp upgraded to 2.x in the
gate.
Michal will modify the gui tests to detect if ldtp 2.x is installed
and terminate if this is not the case with an appropriate warning.
My concern here is that, once the new gui tests are landed, it will
suddenly look less important to get ldtp 2.X integrated as a package. If
you want to land the tests first, I won't stand in the way, but I think
it's reasonable to assume they won't get a lot of use until we have the
ldtp 2.X package integrated.
I am providing the steps to install ldtp 2.x and run the tests as
normal user. The tests will not work as different user then the
currently running GNOME desktop with A11Y enabled as the pyatspi
module will not be able to connect between those users, so please do
not su into root. All of the IPS images used for tests are user
ones, so all the tests will pass without need of running as root.
Given this, I'd like to see changes made so that the gui tests are
not part of the default set of tests to run. We know that it's
necessary to run some of the existing non-gui tests as root in order
for them to work. I'd like to see an option which enables the gui
tests and disables the other sets of tests from running.
Ok Michal will disable the gui tests by default and add a -g flag to
run.py to run them.
Thanks :)
1. Installing ldtp-2
$ pfexec su
$ cd ~/
$ wget http://download.freedesktop.org/ldtp/2.x/2.1.x/ldtp-2.1.1.tar.gz
$ gunzip ldtp-2.1.1.tar.gz
$ tar -xvf ldtp-2.1.1.tar
$ cd ldtp-2.1.1
### installed_files.txt can be used to uninstall the bits after the
### tests, please see below.
$ python2.6 setup.py install --record installed_files.txt
-------
2. After building the IPS gate please remove all of the ldtp from
there, so the proper one is picked up:
$ cd ips-gate/src
$ export
PATH=/usr/sbin:/opt/solstudioex/bin:/usr/ccs/bin:/usr/bin:/usr/gnu/bin:/usr/X11/bin:/sbin:/usr/sfw/bin
$ make clean
$ make install
$ rm -rf ../proto/root_i386/usr/lib/python2.6/vendor-packages/ldtp*
-------
3. Enable accessibility
gnome-panel menus:
System -> Preferences -> Assistive Technologies -> Enable
assistive technologies
Close and Log Out
I would hope that, in the final version, the test suite can
automatically enable assistive techs when run, then restore them to
the preexisting state after they're finished.
Nope, as the user needs to enable the assistive techs and then log out
and log back in again, we are not planning to support this auto
comfiguration.
That's unfortunate, but if changing assistive techs requires logging out
and back in, then I guess there's nothing you can do about it. Where
would be the appropriate place to file a rfe/bug about this?
If that's impossible, then I assume the test suite will check to see
if they're enabled, and if not, stop immediately with a sensible
error message.
Yes this is currently the way the patch works.
Good.
-------
4. Run tests (as normal user - not root):
$ cd ips-gate/src/tests
$ ./run.py -o t_pm_*
--------------
Uninstalling ldtp-2
--------------
# If you wish to remove all of the ldtp-2 bits previously installed,
# please run the following command using the previously created
# installed_files.txt.
$ pfexec su
$ cd ~/ldtp-2.1.1/
$ rm $(cat installed_files.txt)
Other comments:
The copyrights are incorrect. They shouldn't be 2009, 2010, 2011,
instead it should be the first and last year. It looks like, 2009,
2011 will cover most of your files.
Ok will be changed.
These tests look like a reasonable start at testing functionality. I
assume more tests are in the works/planned to deal with error
conditions, edge cases, etc?
Yep these tests can be extended given time and resources. Its another
reason why I would like this in the gate, so Conny can work on a
current functional base, instead of maintaining her own patch set.
Do I have to be logged in locally to the machine I'm running these
test cases on? What happens if I'm ssh'd in to a machine and run
these tests? I assume that if I don't have x forwarding enabled, it
fails, but what if I do have it enabled?
The only way to do this is to run the tests over a VNC remote session,
it will not work with ssh and x forwarding due to bonobo limitations
in Gnome.
That's again unfortunate.
Michal will make the changes discussed above and respin the webrev.
Thanks,
Brock
Thanks.
JR
Thanks,
Brock
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
