Dan Price wrote:
On Tue 23 Jun 2009 at 05:15PM, Michal Pryc wrote:
Hey,
I have prepared yet another webrev for the automated tests for IPS GUI.
Those two webrevs can be applied separately and are for one purpose, to
make automated tests available for the IPS GUI.
My high level comment is that this set of wads needs more work.
I could be confused, but at a high level:
- ldtp.sh has no copyright or cddl.
Will fix that. Also other things doesn't have cddl license, so I need to
add those.
- Don't write scripts in bash in our gate. It's POSIX sh, ksh,
or python.
There are more scripts then just ldtp.sh written in bash as this is
default shell in opensolaris, so changing this now will be a little
tricky, but doable if this is the requirement.
- Don't we have code in setup.py to download stuff? Why
can't we use that?
The setup.py is used to build things, which are using setup.py. Also it
has been targeted to build portable across all platforms things. I have
started doing this in the setup.py, but then after replay from Tom
Mueller I have decided to do it outside of it.
- Would be it be better to just get JDS guys to add ldtp?
It is an option, but now it would take a while to get this into release
repo.
- To follow the convention I established, LDTP should
go into either src/extern or src/gui/extern.
Will change this.
- Why are you doing the build in a shell script? Isn't
that why we have make? How will you ensure we don't
keep remaking this?
Doing this in the Makefile is quite dirty way, as we need to run all
commands like configure, aclocal etc... Also checking for hash sums and
using functions would make the makefile too complex.
DTrace scripts are lacking copyrights and licenses. A note that
something is "based on" something else is not sufficient.
Will add this.
run_dtrace.sh:
- similar complaints as above.
- RESULT_DIR should be setable from the environment. Directories
are directories, not folders
I will add this part.
*.py: I'm pretty unhappy that we built a very nice test infrastructure
which you are not using. pkg5testcase should be able to supply a lot
of what you need or could be subclasses or extended to do what you
want. It avoids things like hardcoding localhost:10000 everywhere,
provides logging, and would make all of your tests discoverable by
'run.py'.
The pkg5testcase is the unittest, while this part is automated test for
the GUI only. I don't think we could integrate this easily to the
pkg5testcase, as each of the test scripts is using specific to ldtp
functions. The problem with testing GUI is that some user have to
"click" "type into field" or do another action in the specific part of
the application, which ldtp is doing automatically.
- __author__ should be eliminated (and it isn't even spelled
right in test_ipsgui_start.py)
Ok.
- Test cases need copyright and license.
Ok.
- Occasionally there is code commented out
(test_ipsgui_remove.py:42)
Will remove.
- wait(10), wait(5), wait(15) in test cases seems awful. Please
DO NOT commit this sin.
This is wait from ldtp, not python:
http://ldtp.freedesktop.org/user-doc/wait.html
For some components like dialogs it is possible to wait for specific
action (like window show/hide):
http://ldtp.freedesktop.org/user-doc/waittillguiexist.html
Unfortunatelly there are some parts which can't be triggered this way.
For example if the dialog is printing text and the progress is bouncing,
then we can't wait for progress to be finished, we could only check for
the specific text in the textfield. This could be done, but those test
should run in different languages, so I am not sure what is better wait
or english only test. Because we are using local image and publisher,
then we can be pretty sure that this test will be finished in the given
time.
--
best
Michal Pryc
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
