Re: [classlib] Testing conventions - a proposal

2006-07-26 Thread George Harley
Alexei Zakharov wrote: Hi George, Sorry for the late reply. Hi Alexei, Not a problem. Especially when my reply to you is even later (sorry). It looks like you are using an os.any group for those test methods (the majority) which may be run anywhere. That's a different approach to what I

Re: [classlib] Testing conventions - a proposal

2006-07-26 Thread Paulex Yang
FYI, I haven't studied it yet, but seems new TestNG 5 support ant task with JVM parameter[1] [1] http://www.theserverside.com/news/thread.tss?thread_id=41479 Richard Liang wrote: Just thinking about using TestNG to execute Harmony test cases. :-) Look at our build.xml (e.g.,

Re: [classlib] Testing conventions - a proposal

2006-07-25 Thread Stepan Mishura
On 7/20/06, George Harley wrote: SNIP! Anyway, the point I guess that I am trying to make here is that it is possible in TestNG to select the methods to test dynamically using a little bit of scripting that (a) gives us a lot more power than the include/exclude technique and (b) will work the

Re: [classlib] Testing conventions - a proposal

2006-07-24 Thread Alexei Zakharov
Hi George, Sorry for the late reply. It looks like you are using an os.any group for those test methods (the majority) which may be run anywhere. That's a different approach to what I have been doing. I have been thinking more along the lines of avoiding the creation of groups that cover the

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread Alexei Zakharov
Hi George, Wow, they are fast guys! Thanks for the link. Do you know when do they plan to release 5.0 officially? Regards, 2006/7/19, George Harley [EMAIL PROTECTED]: Hi Alexei, I just downloaded the latest working build of TestNG 5.0 [1] and support for the jvm attribute is in there. This

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread George Harley
Richard Liang wrote: George Harley wrote: Richard Liang wrote: George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread George Harley
Alexei Zakharov wrote: Hi George, Wow, they are fast guys! Thanks for the link. Do you know when do they plan to release 5.0 officially? Regards, Hi Alexei, Actually, I just saw this announcement in my news reader about 15 minutes ago ... http://beust.com/weblog/archives/000400.html

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread Alexei Zakharov
George, I remember my past experience with BeanShell - I was trying to create the custom BeanShell task for ant 1.6.1. I can't say I haven't succeeded. But I remember this as a rather unpleasant experience. At that time BeanShell appeared to me as a not very well tested framework. Please don't

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread George Harley
Alexei Zakharov wrote: George, I remember my past experience with BeanShell - I was trying to create the custom BeanShell task for ant 1.6.1. I can't say I haven't succeeded. But I remember this as a rather unpleasant experience. At that time BeanShell appeared to me as a not very well tested

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the possibilities that the

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
According to TestNG Ant Task [1], it seems that the TestNG Ant task does not support to fork a new JVM, that is, we must launch ant using Harmony itself. Any comments? Thanks a lot. [1]http://testng.org/doc/ant.html Best regards, Richard George Harley wrote: Andrew Zhang wrote: On 7/18/06,

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Alexei Zakharov
Hmm, do we have problems with launching ant? I thought we have problems with launching TestNG. Just checked - running tests for beans on j9+fresh classlib works fine. I.e. ant -Dbuild.module=beans -Dbuild.compiler=org.eclipse.jdt.core.JDTCompilerAdapter test 2006/7/19, Richard Liang [EMAIL

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
Just thinking about using TestNG to execute Harmony test cases. :-) Look at our build.xml (e.g., modules/luni/build.xml), you will see something like: .. junit fork=yes forkmode=once printsummary=withOutAndErr

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Richard Liang wrote: George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Hi Richard, Actually the Ant task always runs the tests in a forked VM. At present, however, the task does not support specifying the forked VM (i.e. there is no equivalent to the JUnit Ant task's jvm attribute). This matter has already been raised with the TestNG folks who seem happy to

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Alexei Zakharov
Probably my previous message was not clear enough. Why can't we just invoke everything including ant on top of Harmony for now? At least I was able to build and run test-14 examples from TestNG 4.7 distribution solely on top of j9 + our classlib today. C:\Java\testng-4.7\test-14set

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Hi Alexei, It's encouraging to hear that (Ant + TestNG + sample tests) all worked fine together on Harmony. In answer to your question I suppose that the ability to fork the tests in a separate VM means that we do not run the risk of possible bugs in Harmony affecting the test harness and

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Alexei Zakharov
Hi George, Agree, we may experience problems in case of VM hang or crash. I suggest this only as a temporary solution. BTW, the fact that TestNG ant task still doesn't have such attributes looks like a sign for me - TestNG can be still immature in some aspects. Still comparing TestNG and JUnit.

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Hi Alexei, I just downloaded the latest working build of TestNG 5.0 [1] and support for the jvm attribute is in there. This is not the official release build. Best regards, George [1] http://testng.org/testng-5.0.zip Alexei Zakharov wrote: Hi George, Agree, we may experience problems in

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
George Harley wrote: Hi Alexei, It's encouraging to hear that (Ant + TestNG + sample tests) all worked fine together on Harmony. In answer to your question I suppose that the ability to fork the tests in a separate VM means that we do not run the risk of possible bugs in Harmony affecting

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
George Harley wrote: Richard Liang wrote: George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Oliver Deakin
George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: * type.impl -- tests that are specific to Harmony So tests are

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: * type.impl -- tests that are specific to

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Oliver Deakin
George Harley wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: * type.impl -- tests

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Andrew Zhang
On 7/18/06, George Harley [EMAIL PROTECTED] wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Alexei Zakharov
Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are platform-specific. I volunteer to do this trial for beans module. I'm not sure that

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Andrew Zhang
On 7/18/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are platform-specific. I

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Oliver Deakin wrote: George Harley wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: *

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Andrew Zhang wrote: On 7/18/06, George Harley [EMAIL PROTECTED] wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Oliver Deakin
George Harley wrote: Oliver Deakin wrote: George Harley wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Andrew Zhang wrote: On 7/18/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Alexei Zakharov wrote: Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are platform-specific. I volunteer to do this trial for beans

Re: [classlib] Testing conventions - a proposal

2006-07-17 Thread George Harley
Andrew Zhang wrote: On 7/14/06, George Harley [EMAIL PROTECTED] wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the

Re: [classlib] Testing conventions - a proposal

2006-07-14 Thread George Harley
Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the possibilities that the TestNG @Test annotation

Re: [classlib] Testing conventions - a proposal

2006-07-14 Thread Andrew Zhang
On 7/14/06, George Harley [EMAIL PROTECTED] wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the

Re: Re: [classlib] Testing conventions - a proposal

2006-07-11 Thread Alexei Zakharov
Hi Alex, It's a pitty what you didn't find common sense in my post. Probably I was not clear enough. My key points are: 1. JUnit is much like a standard of unit testing today 2. We are using JUnit already, have thousands of tests 3. May be I was not correct about bugs in TestNG - I assume that

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Oliver Deakin
Geir Magnusson Jr wrote: Oliver Deakin wrote: George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class

Re: Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Alexei Zakharov
Actually, there's a very valid benefit for using TestNG markers (= annotations/JavaDoc) for grouping tests; the directory structure is a tree, whereas the markers can form any slice of tests, and the sets Concerning TestNG vs JUnit. I just like to pay your attention on the fact what it is

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread George Harley
Alexei Zakharov wrote: Actually, there's a very valid benefit for using TestNG markers (= annotations/JavaDoc) for grouping tests; the directory structure is a tree, whereas the markers can form any slice of tests, and the sets Concerning TestNG vs JUnit. I just like to pay your attention on

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Geir Magnusson Jr
Oliver Deakin wrote: Geir Magnusson Jr wrote: Oliver Deakin wrote: George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Geir Magnusson Jr
George Harley wrote: Alexei Zakharov wrote: Actually, there's a very valid benefit for using TestNG markers (= annotations/JavaDoc) for grouping tests; the directory structure is a tree, whereas the markers can form any slice of tests, and the sets Concerning TestNG vs JUnit. I just like

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Alexei Zakharov
Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact statement was referring to TestNG as probably unstable rather than simply unstable. ;) This statement was

Re: Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Alex Blewitt
On 10/07/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact statement was referring to TestNG as probably

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread George Harley
Alexei Zakharov wrote: Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact statement was referring to TestNG as probably unstable rather than simply unstable.

[classlib] TestNG v. JUnit (was: RE: [classlib] Testing conventions - a proposal)

2006-07-10 Thread Nathan Beyer
@incubator.apache.org Subject: Re: [classlib] Testing conventions - a proposal Alexei Zakharov wrote: Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact

Re: [classlib] Testing conventions - a proposal

2006-07-09 Thread Richard Liang
Richard Liang wrote: Paulex Yang wrote: Richard Liang wrote: Hello All, After read through the document recommended by Alex, I think TestNG can really meet our requirement. It provides much flexibility for test configuration. ;-) If we decide to transfer to TestNG, we shall: 1.

Re: [classlib] Testing conventions - a proposal

2006-07-08 Thread Geir Magnusson Jr
Nathan Beyer wrote: -Original Message- From: Geir Magnusson Jr [mailto:[EMAIL PROTECTED] This is a fun thread. I plan to read it from end to end later today and comment. Initial thoughts are that I've been wanting to use TestNG for months (hence my resistance to any JUnit

Re: Re: [classlib] Testing conventions - a proposal

2006-07-08 Thread Alex Blewitt
On 08/07/06, Geir Magnusson Jr [EMAIL PROTECTED] wrote: So while I like the annotations, and expect we can use them effectively, I have an instinctive skepticism of annotations right now because in general (in general in Java), I'm not convinced we've used them enough to grok good design

Re: [classlib] Testing conventions - a proposal

2006-07-08 Thread Geir Magnusson Jr
Alex Blewitt wrote: On 08/07/06, Geir Magnusson Jr [EMAIL PROTECTED] wrote: So while I like the annotations, and expect we can use them effectively, I have an instinctive skepticism of annotations right now because in general (in general in Java), I'm not convinced we've used them enough

Re: [classlib] Testing conventions - a proposal

2006-07-07 Thread Mark Hindess
On 6 July 2006 at 21:02, Nathan Beyer [EMAIL PROTECTED] wrote: I think Tim has a valid point, or at least the point I'm inferring seems valid: the testing technology is not the real issue. This problem can be solved by either JUnit or TestNG. More specifically, this problem can be solved

Re: [classlib] Testing conventions - a proposal

2006-07-07 Thread Geir Magnusson Jr
Oliver Deakin wrote: George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library test layouts. The

RE: [classlib] Testing conventions - a proposal

2006-07-07 Thread Nathan Beyer
-Original Message- From: Geir Magnusson Jr [mailto:[EMAIL PROTECTED] Maven layout? We were doing that layout in Jakarta projects long before maven And I would guess the Maven designers would agree. Much of their documentation talks about how the conventions inferred in the

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Mikhail Loenko
2006/7/5, George Harley [EMAIL PROTECTED]: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library test layouts. The current

Re: Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Alex Blewitt
On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: George Harley wrote: A couple of weeks ago I mentioned that the TestNG framework [2] seemed like a reasonably good way of allowing us to both group together different kinds of tests and permit the exclusion of individual tests/groups of

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Richard Liang
Alex Blewitt wrote: On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: George Harley wrote: A couple of weeks ago I mentioned that the TestNG framework [2] seemed like a reasonably good way of allowing us to both group together different kinds of tests and permit the exclusion of

Re: Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Alex Blewitt
On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: It seems that you're very familiar with TestNG. ;-) So would you please identify what we shall do to transfer from junit to TestNG? Thanks a lot. Me? I'm just highly opinionated :-) There's guidelines for migrating from JUnit to TestNG at

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Richard Liang
Alex Blewitt wrote: On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: It seems that you're very familiar with TestNG. ;-) So would you please identify what we shall do to transfer from junit to TestNG? Thanks a lot. Me? I'm just highly opinionated :-) There's guidelines for migrating

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread George Harley
Alex Blewitt wrote: On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: It seems that you're very familiar with TestNG. ;-) So would you please identify what we shall do to transfer from junit to TestNG? Thanks a lot. Me? I'm just highly opinionated :-) Hi Alex, I think we are all

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Tim Ellison
Mikhail Loenko wrote: 2006/7/5, George Harley [EMAIL PROTECTED]: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library

RE: [classlib] Testing conventions - a proposal

2006-07-06 Thread Nathan Beyer
I think Tim has a valid point, or at least the point I'm inferring seems valid: the testing technology is not the real issue. This problem can be solved by either JUnit or TestNG. More specifically, this problem can be solved utilizing the grouping of arbitrary tests. I'm been playing with

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Andrew Zhang
On 7/5/06, George Harley [EMAIL PROTECTED] wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library test layouts. The

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Richard Liang
Hello All, After read through the document recommended by Alex, I think TestNG can really meet our requirement. It provides much flexibility for test configuration. ;-) If we decide to transfer to TestNG, we shall: 1. Identify Harmony testing strategy. (It's not easy) 2. Define TestNG

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Paulex Yang
Richard Liang wrote: Hello All, After read through the document recommended by Alex, I think TestNG can really meet our requirement. It provides much flexibility for test configuration. ;-) If we decide to transfer to TestNG, we shall: 1. Identify Harmony testing strategy. (It's not easy)

Re: [classlib] Testing conventions - a proposal

2006-07-05 Thread Richard Liang
George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library test layouts. The current proposal [1] sets