On Wed 24 Jun 2009 at 10:50AM, Michal Pryc wrote:
> > - Don't write scripts in bash in our gate. It's POSIX sh, ksh,
> > or python.
> >
> There are more scripts then just ldtp.sh written in bash as this is
> default shell in opensolaris, so changing this now will be a little
> tricky, but doable if this is the requirement.
I'm declaring it to be the gate policy in order to prevent language
sprawl.
> >*.py: I'm pretty unhappy that we built a very nice test infrastructure
> >which you are not using. pkg5testcase should be able to supply a lot
> >of what you need or could be subclasses or extended to do what you
> >want. It avoids things like hardcoding localhost:10000 everywhere,
> >provides logging, and would make all of your tests discoverable by
> >'run.py'.
> >
> >
> The pkg5testcase is the unittest, while this part is automated test for
> the GUI only. I don't think we could integrate this easily to the
> pkg5testcase, as each of the test scripts is using specific to ldtp
> functions. The problem with testing GUI is that some user have to
> "click" "type into field" or do another action in the specific part of
> the application, which ldtp is doing automatically.
> > - __author__ should be eliminated (and it isn't even spelled
> > right in test_ipsgui_start.py)
Your tests are also unittest tests. What pkg5testcase and friends
give you is an infrastructure for setting up repos and images with
contents and then tearing those down. You subclass from it. It is a
subclass of unittest. You can bring in ldtp or whatever else as well.
How would you write this test, in your current framework:
- Start pkg manager
- Do an image-update (no effect, no changes available)
- Add contents to the repository
- Do an image-update (will work, new changes available).
Or:
- Start pkg manager (depot is running)
- Shut down depot
- Try to do an image-update from the pkg manager, and
test that it handles the error gracefully.
With your static setup, as best I can see, you can't. You can only
really write the most simplistic of tests. pkg5testcase (or your
subclass of it) would give you all of the power you would need. AT
LEAST read it and understand it and copy from it.
> This is wait from ldtp, not python:
> http://ldtp.freedesktop.org/user-doc/wait.html
Right, I know.
> For some components like dialogs it is possible to wait for specific
> action (like window show/hide):
...
> the specific text in the textfield. This could be done, but those test
> should run in different languages, so I am not sure what is better wait
> or english only test.
It's 1000x better to test for the english language text, or to invent
a special TESTING locale which makes it easy to test for certain events.
It's more precise, and will run a lot faster. It is simply not
acceptable to have a simple test case taking a minute to run. A minute
doesn't sound like much, until you have 400 test cases. Then, you and
everyone else will be sad. Please-- this is the voice of experience--
don't neglect this.
-dp
--
Daniel Price, Solaris Kernel Engineering http://blogs.sun.com/dp
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
