Mikhail Loenko wrote:
I think I can understand now, it's just directory layout, isn't it? I'm fine with it.2006/4/28, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:I suggest physically separate high-level suites:src/test/ | +- impl/ | +- compliant/ [names might be changed] | +- stress/ | \- etc/ Then on package-level separate classpath from bootclasspath stuff 1) We will run compliant to validate test suite 2) We will not run stress before commit 3) etcAre the src/test/impl or so the src directory?Sorry I did not understand the question
I guess Ant would know it needs to run on classpath by keyword "tests"? That's fine.if so, I'm fine with it.And you add stress category we haven't mentioned before, that's fine, too.I've meant that we have other categories of the tests that will be somewhere around.But I think I didn't catch up, where will you put the case 4 (the ones of [classpath] [dependent] [java.*] )?folder impl/ package org.apache.harmony.module.tests.java.something
And how many folders it will be(Ijust not sure what the etc means...;))?I don't know. Couple month ago we were talking about testing in exotic configurations. People suggested separating some types of tests from general test run. These types of tests might reside in other 'etc' folders.
Agree.
Thanks, MikhailWhat do you think? Thanks, Mikhail 2006/4/28, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:Hi Paulex 2006/4/28, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:Then how to run only independent tests? Such a run would make sense to validate the tests or if we switch to different RI versionIt should be easy for ant script to identify independent tests by package name convention, e.g, we have a keyword "API" for the independent tests (1), and 2 can be easily excluded. For the intended different behavior (4), test can be added to ant's exclude list (I don't expect there are too many such cases)Did you count such tests? As I remember we have a lof of them in security.Lots of ? Ooops...Seems I didn't understand the situation clearly (I knew I must miss sth. ;-) )If these tests are in the same package and in the same directory then we will have something like org.apache.harmony.security.tests.api.java.security.CodeSignerTest1 - impl specific test methods org.apache.harmony.security.tests.api.java.security.CodeSignerTest2 - impl independent test methodsIf there are many similar cases, I agree that it is unacceptable(although I think the test case can be named more meaningfully than xxxTest1/xxxTest2), because I prefer 1 test for 1 class in general . It will be perfect if we can introduce some mechanism to control test execution in method level, but I have no good idea on this so far. So do you have any better ideas on this issue?How do you like it? Thanks, MikhailThanks, Mikhail 2006/4/28, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:And how would you split 1,2,4,5 between these two folders?I thought it is obvious. test 1,2,4 goes to src/test test 5 goes to src/impltest (internl/injected/boot anything is fine) and all implementation goes to src/main and they can be compiled to corresponding bin dir, i.e. bin/test bin/impltest bin/main bin/main and bin/impltest goes to bootclasspath bin/test goes to classpath Pls. correct me if I made mistake or missed sth.Thanks, Mikhail 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:If it's separated, I'm OK with it Can you suggest dir names?Great! Either boot or injected(as Mark said) sounds good keyword to me. so it can be one of below: 1. src/test.injected/ 2. src/test/injected 3. src/test.boot/ 4. src/test/boot Mmm...seems injected is more self-descriptive, I prefer option 1. So the layout would be: src/main src/test src/test.injected ideas?Thanks, Mikhail 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:The problem is the tests will be mixed like that: org.apache.harmony.security.asn1 -- tests in bootclasspath org.apache.harmony.security.cert -- tests in bootclasspath org.apache.harmony.security.tests -- tests in classpath org.apache.harmony.security.x509 -- tests in bootclasspathGood catch! :) It's not so hard for script to separate, if the package name doesn't contains "test", it belongs to bootclasspath And if you concern about human reading, we can separate src directory for them, i.e. src/main/java src/test/java src/boottest/java or src/test/boot/java comments?Thanks, Mikhail 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:Paulex, we have at least 8 categories of tests: running from classpath or bootclasspath implementation specific or independent testing org.apache.harmony.* or java.* Could you please list how all the tests will be namedMikhail, OK, let me try. I'm not sure if all 8 categories are necessary, 1. [classpath], [independent], [java.*] o.a.h.<module name>.tests.api.<package name>, e.g., o.a.h.luni.tests.api.java.util is test package for java.util I consider the module name is optional and can be removed, so it can be o.a.h.tests.api.java.util for java.util 2. [classpath], [dependent], [o.a.h.*] o.a.h.<module name>.tests.<package name without o.a.h...prefix>, e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util Again, the module name is optional, but if module name is not prefix, it should be part of package name, so it is: o.a.h.tests.luni.util for o.a.h.luni.util 3. [classpath], [independent], [o.a.h.*] I don't expect these classes/tests exist. 4. [classpath], [dependent], [java.*] if my understanding is correct, this category means the tests for intended different behavior between Harmony and RI, I consider these tests should be put in same package with API tests, and separate them by running script instead of different package, because RI may fix bug in the revision later(e.g, Sun JDK 1.5.1 maybe?), and IIRC in another thread we have agreed that we always follow latest version of RI, so it will be much easier to change the different behavior tests to API tests if we use script to separate them, and vice versa. 5. [bootclasspath], which covers 4 categories you mentioned, the test cases are in same package with implementions e.g. java.util is test package for java.util and o.a.h.luni.util is test packag for o.a.h.luni.util As a summary, we'll have four kind of test packages: a. o.a.h.<module>.tests.api.java.*** for case 1,4 above b. o.a.h.<module>.tests.*** for case 2 above c. java.***, for case 5 above d. o.a.h.***, for case 5 above comments?Thanks, Mikhail 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>:Mikhail Loenko wrote:Geir, The problem is that no one yet suggested a consistent solution that would fit for all the tests and would not get into packages like org.apache.harmony.security.tests.org.apache.harmony.security.util Everybody seems to agree that SOME implementation specific tests are in the same package as implementation. But if we have ALL the tests in the same package as implementation, then we can't run e.g. tests against java.beans.Beans from classpath, so we can't run regression test for HARMONY-358 The question is where we put 'classpath' test. If we put them into o.a.h.module.tests.[package under test] we result in that ugly package names like o.a.h.security.tests.o.a.h.security.utilMikhail, Why my prior proposal that o.a.h.security.tests.impl.util is unacceptable? And I have no objection if "impl" considered redundant, too.Thanks, Mikhail 2006/4/27, Geir Magnusson Jr <[EMAIL PROTECTED]>:Geir Magnusson Jr wrote:George Harley wrote:>>Of course, the text module has only "implementation-independent tests that designed to be run from classpath". For modules that have got implementation-specific tests then I suppose we could use something like "org.apache.harmony.[module].tests.impl.[package under test]" or "org.apache.harmony.[module].tests.internal.[package under test]" etc. I've got no preference.-1 Put them in the same package as the class being tested, please, if there is no problem being run that way.Sorry to be so snippy. Been a long day. Why would we want this convoluted package name? Why wouldn't we want to let them be in the same package as the class being tested since it's implementation testing? geir --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
-- Paulex Yang China Software Development Lab IBM --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]