Brad,
Please consider these comments.
1. +1 on moving the memleaks to a separate target.
2. There is a similar code block at the top of setup.py and run.py that
detects the OS name/type. Maybe those blocks could be put into a shared
file (maybe testutil.py or maybe a new buildutil.py).
3. run.py, line 178: there is a portable.is_admin() method that tries to
do the right thing for other platforms.
From a multi-platform perspective, I like how you've structured this.
I'm hoping that eventually we'll get the CLI tests to run on Windows
too, but we have some work in the depotcontroller to make that happen.
Thanks.
Tom
Brad Hall wrote:
Hi,
Link to CR: http://cr.opensolaris.org/~bhall/bug-2664/
Link to BUG: http://defect.opensolaris.org/bz/show_bug.cgi?id=2664
These are basically just enhancements (some of them from code reviews of 2627)
to the test suite.
- More verbose failure reporting:
* FAILURE (successes=4, failures=2, errors=1, mismatches=2)
- Merge {api,cli}-complete.py into one file (run.py)
* This allows for a single baseline file (baseline.txt) instead of two
separate ones.
- Change run.py to be the main way of running the test harness:
* All the make targets still work, but they call run.py instead of
setup.py (you can still run make test, test-verbose, test-generate,
etc.)
* This gets rid of the ugly passing arguments through setup.py just
to have them re-parsed in *-complete.py
* Gets rid of code duplication in *-complete.py which will continue to
get worse as we add more stuff.
- Add -o <regex> (--only) option for tests:
* The -o option takes a regex to determine what tests to run, this
lets you do things like:
- python run.py -o cli (runs only cli tests)
- python run.py -o api (runs only api tests)
- python run.py -o depot (runs only depot related tests)
- python run.py -o mytestname (runs single test)
- .. you get the idea.
- Add the -b option for specifying which baseline file to use (if you have an
alternate one)
- Baseline results are now sorted (this should help with diffs :))
- Moved memleaks to a separate Makefile target; I can move it back if people
feel strongly about it but I'm tired of having to scroll up to see if the
actual test suite failed. A lot of the time I find myself just wanting to
run the test cases.. do others agree? Maybe we can make the 'make test'
target run the test suite + memleaks, and provide a different target for
only running the test cases.
- There is one part of bug 2664 that I didn't address; Brock's suggestion of
displaying the assertion/error after a failed test -- I'll file that in
another bug since I'm not sure of a good way of doing that yet.
Comments, suggestions?
Thanks,
Brad
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
begin:vcard
fn:Tom Mueller
n:Mueller;Tom
org:Sun Microsystems, Inc.;Update Center Software
adr:;;21915 Hillandale Dr;Elkhorn;NE;68022;USA
email;internet:[EMAIL PROTECTED]
title:Senior Staff Engineer
tel;work:877-250-4011
tel;fax:877-250-4011
tel;home:402-916-9943
x-mozilla-html:TRUE
version:2.1
end:vcard
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
