Alexei Zakharov wrote:
Hi Richard,
Not sure if we really want to involve another migration: TestNG javadoc
-> TestNG annotation. Any comments?
Well, IMHO this depends on time constraints - when do we plan to have
the support for anotations? If the answer is about a couple of weeks -
no problem, we can wait. But if this is several months...
About the "migration" - I don't think this will be a real painfull
migration, all infrastructure will remain the same. We will only need
to convert javadocs to annotations (one-one correspondence) and this
task can be easily automated.
Sounds reasonable. :-) Maybe drlvm guys or Oliver could tell us when we
will have a VM with annotation support?
Thanks,
2006/8/1, Richard Liang <[EMAIL PROTECTED]>:
Alexei Zakharov wrote:
> Hi,
>
> I have created this new thread as a single place for discussions
> started in "Re: [testing] Peace" and "[classlib] Testing conventions –
> a proposal" threads.
>
> What did we have in the previous threads?
> * Test classification proposed by Vladimir
> * Test classification and group names proposed by George
> * Solution for Ant and TestNG scripting issues (still being discussed)
>
> Since a lot of people are asking about TestNG and wanting TestNG I
> decide to put some effort and take a closer look at this piece of
> software. Thus during the last few days I was playing with TestNG - I
> tried to run different kind of tests with it, to perform various
> workloads, generate reports in different ways and etc. The purpose of
> all this activity was to try TestNG in a real work, understand is
> TestNG really worth our credits and how expensive can be moving to
> TestNG from our currently implemented testing infrastructure. Now I
> have some thoughts and facts I'd like to share with the community.
> I've put it in the form of list for convenience.
>
> * TestNG works ok in normal conditions, no visible bugs
> * It is possible to define and use various (possibly intersecting)
> test groups with TestNG
> * TestNG-style metadata is more convenient than JUnit test suites (now
> I agree with this statement). IMHO this is the main TestNG benefit.
> * It is possible to run TestNG from command line
> * There is also the special ant task for running TestNG
> * Not everything can be configured with the ant task or command-line
> params, sometimes extra test suite definition file "testng.xml" is
> needed
> * It is possible to run jUnit tests with TestNG ("testng.xml" is
> needed in this case)
> * It is possible to run junit tests we currently have in Harmony with
> TestNG without any problems and modifications of the source code.
> However, we probably should write some number of TestNG test suite
> definition files "testng.xml" to be able to run all our junit tests (I
> have tried tests for bean module and some tests for luni)
> * We can mix jUnit tests and TestNG tests in the single test suite
> configuration – i.e. single testng.xml file. We can add TestNG
> metadata to some test classes and leave other test classes untouched
> * TestNG generates HTML reports in its own style, not a very
> convenient one IMHO
> * It is also possible to generate JUnitReports from the output
> generated by TestNG
> * Such reports will have a little bit different structure since TestNG
> doesn't provider any information about enclosing type for test
> methods. Names for tests (replacement for JUnit "test classes") and
> test suites should be externally configured in "testng.xml"
> * TestNG for Java 5 doesn't work on Harmony because some necessary
> classes from java.util.concurrent package are missing and it seems
> that jsr14 target (we are currently using) doesn't support annotations
> * TestNG for Java 1.4 (javadoc version) currently works on Harmony
> * I have half-way done script that converts TestNG 1.4 metadata
> (javadoc) tests to TestNG 1.5 (5.0 annotations) tests.
>
Excellent summary! Thanks a lot
> The question I'd like to raise now is – aren't we ready for TestNG
> right now?
I suppose we will use Java 5.0 annotations of TestNG, so it seems now we
are not ready for TestNG. But we can continue our feasibility study,
just like what you have done, to know if TestNG really meets our
requirements or if there are any potential problems. Maybe we could list
a prerequisite list. e.g,
1) Harmony can fully self-host TestNG with Java5 annotations
2) Test groups are well-defined and agreed in community
3) Guidelines to write TestNG testcases
4) Take one module to run a pilot case
....
Please correct me if I'm wrong
> For example, we could replace our harness from jUnit to
> TestNG and lazily start converting of some failing and platform
> dependent tests to javadoc version of TestNG. The rest of the tests
> will remain jUnit in fact. And when our VM will be ready to handle
> annotations we can convert all our TestNG 1.4 tests to TestNG 1.5. I
> understand that this idea may seem to be too early. But anyway we will
> need to change things some day since many people are unhappy with the
> current testing infrastructure (me for example).
Not sure if we really want to involve another migration: TestNG javadoc
-> TestNG annotation. Any comments?
>
> Thought? Suggestions? Opposite opinions?
>
> With Best Regards,
>
>
--
Richard Liang
China Software Development Lab, IBM
--
Richard Liang
China Software Development Lab, IBM
---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
