Paulex Yang wrote:
Jimmy, Jing Lv wrote:
Hi,
As it is hard to write unit test for package instrument, I now
have an idea: write down some documents for details of non-unit test
cases for instrument. The detail of such test cases contain:
1. The test run on which platform, including special environment;
2. The test description;
3. How to run test (command line, etc.);
4. test result;
5. resource (or links) for tests, including c/java code for test if any.
I hope ensure the code quality in this way. Like unit test, such
tests can be checked and re-run for regression, though un-automatically.
You mean these tests' result need to be checked manually? I think it is
not acceptable. IIUC, the issue here is some instrument agent needs to
be specified in JVM args, so these tests are somthing that need special
startup configuration, we can find a way to handle these tests, say,
specify the configuration in instrument's build.xml, we can even fork a
new JVM process per these tests, (Not sure if TestNG has any special
support for this?).
Sorry I did not explain myself very well. Some of the tests for
instrument, IMO, can not be test automatic well. However, other tests of
instrument can be certainly written into unit test, or check automatic.
The main difference between the tests of instrument and other modules is
that it should raise a new VM to run. I believe it can be controlled by
ant or something.
Most happy paths can be checked automatic, e.g, check if the certain
class has been transformed, the size of the object is correct, etc.
However, there are still lots of things that can not be checked well: VM
crashes, exceptions, and special output by VM, etc. These documented and
undocumented behavior should be checked carefully, but is not easy for
automatization as I have explained. Only in this situation shall I take
action to document them down as tests.
After all, any suggestion on these special tests are welcome. I hope we
can find out a way to deal these issue without documents, but with an
automatic mechanism.
And I'll put them in Harmony-wiki[1], and any test-description is
welcome!. (Geir, I've found an advancement of wiki :) )
Any suggestions? :)
[1]http://wiki.apache.org/harmony/INSTRUMENT
> Geir Magnusson Jr wrote:
Never appeal to me using a Wiki as an authority :) It's like finding
something written in chalk on the sidewalk.
However, if you had pointed out
http://incubator.apache.org/harmony/auth_cont_quest.html
I'd have agreed w/o having the opportunity to make fun of the Wiki.
So I agree :)
geir
--
Best Regards!
Jimmy, Jing Lv
China Software Development Lab, IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
