Stepan Mishura wrote: > A test framework is quite essential part and we should define it on early > stage. > > I think first we should agree on which test we are going to provide and form > requirements to them. There are no doubts that we will provide unit tests > for classlib. So unit tests are provided by code authors in JUnit format and > intended to verify all code functionality. They are placed in the same > package as tested code and run on the bootclasspath. > > Anything to add? Are there any objections?
You have seen that there *are* objections to putting the test code into the implementation packages and running it on the bootclasspath. I can restate the reasons if it helps. > Conformance tests we will receive form TCK and what about other test suites: > functional, stress tests, performance? Will we create a separate framework > for each? Running the TCK is a certification step, not a suite of tests that we develop against to get a progressively compliant implementation. I believe we need: - implementation tests : using the 'trick' that Geir refers to below to expose some internal APIs to the application code. We would only expect these to pass on Harmony code. - API unit tests : driving the published APIs, these should pass on all compliant implementations (modulo known bugs in those impls.). - then, as you say, additional 'system', 'stress', 'performance', tests which are likely to fall outside the unit test framework. Regards, Tim > Thanks, > Stepan Mishura, > Intel Middleware Products Division > > > On 1/16/06, Geir Magnusson Jr <[EMAIL PROTECTED]> wrote: >> One thing that's popped up on the "Test suite layout" thread is the >> thought that we need to b0rk the canonical package and naming >> conventions for unit tests in order to be able to run things on the boot >> classpath of the VM. I think this issue is important enough and >> fundamental enough to warrant it's own thread. >> >> First, do we really need to do this? I thought that we (Tim and I) had >> informally discussed this at ApacheCon and came to some good conclusion >> where we were able to figure out a trick. >> >> Second, shouldn't we think about providing a test environment in which >> we can completely control the environment - we can test the class >> library in a container that can be run in any VM so we have full control >> over security and other issues? >> >> Of course, I'd like both. If we do have the "trick" that we talked >> about, then we can use canonical JUnit (or TestNG) naming and package >> conventions, which I think is important. >> >> geir >> >> > -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
