Hi Paulex 2006/4/28, Paulex Yang <[EMAIL PROTECTED]>: > Mikhail Loenko wrote: > > Then how to run only independent tests? > > > > Such a run would make sense to validate the tests or > > if we switch to different RI version > > > It should be easy for ant script to identify independent tests by > package name convention, e.g, we have a keyword "API" for the > independent tests (1), and 2 can be easily excluded. > > For the intended different behavior (4), test can be added to ant's > exclude list (I don't expect there are too many such cases)
Did you count such tests? As I remember we have a lof of them in security. If these tests are in the same package and in the same directory then we will have something like org.apache.harmony.security.tests.api.java.security.CodeSignerTest1 - impl specific test methods org.apache.harmony.security.tests.api.java.security.CodeSignerTest2 - impl independent test methods How do you like it? Thanks, Mikhail > > > Thanks, > > Mikhail > > > > 2006/4/28, Paulex Yang <[EMAIL PROTECTED]>: > > > >> Mikhail Loenko wrote: > >> > >>> And how would you split 1,2,4,5 between these two folders? > >>> > >>> > >> I thought it is obvious. > >> test 1,2,4 goes to src/test > >> test 5 goes to src/impltest (internl/injected/boot anything is fine) > >> and all implementation goes to src/main > >> > >> and they can be compiled to corresponding bin dir, i.e. > >> bin/test > >> bin/impltest > >> bin/main > >> > >> bin/main and bin/impltest goes to bootclasspath > >> bin/test goes to classpath > >> > >> Pls. correct me if I made mistake or missed sth. > >> > >>> Thanks, > >>> Mikhail > >>> > >>> 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>: > >>> > >>> > >>>> Mikhail Loenko wrote: > >>>> > >>>> > >>>>> If it's separated, I'm OK with it > >>>>> > >>>>> Can you suggest dir names? > >>>>> > >>>>> > >>>>> > >>>> Great! > >>>> > >>>> Either boot or injected(as Mark said) sounds good keyword to me. so it > >>>> can be one of below: > >>>> 1. src/test.injected/ > >>>> 2. src/test/injected > >>>> 3. src/test.boot/ > >>>> 4. src/test/boot > >>>> > >>>> Mmm...seems injected is more self-descriptive, I prefer option 1. So > >>>> the layout would be: > >>>> src/main > >>>> src/test > >>>> src/test.injected > >>>> > >>>> ideas? > >>>> > >>>> > >>>>> Thanks, > >>>>> Mikhail > >>>>> > >>>>> 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>: > >>>>> > >>>>> > >>>>> > >>>>>> Mikhail Loenko wrote: > >>>>>> > >>>>>> > >>>>>> > >>>>>>> The problem is the tests will be mixed like that: > >>>>>>> > >>>>>>> org.apache.harmony.security.asn1 -- tests in bootclasspath > >>>>>>> org.apache.harmony.security.cert -- tests in bootclasspath > >>>>>>> org.apache.harmony.security.tests -- tests in classpath > >>>>>>> org.apache.harmony.security.x509 -- tests in bootclasspath > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> Good catch! :) > >>>>>> > >>>>>> It's not so hard for script to separate, if the package name doesn't > >>>>>> contains "test", it belongs to bootclasspath > >>>>>> > >>>>>> And if you concern about human reading, we can separate src directory > >>>>>> for them, i.e. > >>>>>> src/main/java > >>>>>> src/test/java > >>>>>> src/boottest/java or src/test/boot/java > >>>>>> > >>>>>> comments? > >>>>>> > >>>>>> > >>>>>> > >>>>>>> Thanks, > >>>>>>> Mikhail > >>>>>>> > >>>>>>> 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>: > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> Mikhail Loenko wrote: > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> Paulex, > >>>>>>>>> > >>>>>>>>> we have at least 8 categories of tests: > >>>>>>>>> running from classpath or bootclasspath > >>>>>>>>> implementation specific or independent > >>>>>>>>> testing org.apache.harmony.* or java.* > >>>>>>>>> Could you please list how all the tests will be named > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> Mikhail, > >>>>>>>> > >>>>>>>> OK, let me try. I'm not sure if all 8 categories are necessary, > >>>>>>>> > >>>>>>>> 1. [classpath], [independent], [java.*] > >>>>>>>> o.a.h.<module name>.tests.api.<package name>, e.g., > >>>>>>>> o.a.h.luni.tests.api.java.util is test package for java.util > >>>>>>>> I consider the module name is optional and can be removed, so it > >>>>>>>> can be > >>>>>>>> o.a.h.tests.api.java.util for java.util > >>>>>>>> > >>>>>>>> 2. [classpath], [dependent], [o.a.h.*] > >>>>>>>> o.a.h.<module name>.tests.<package name without o.a.h...prefix>, > >>>>>>>> e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util > >>>>>>>> Again, the module name is optional, but if module name is not > >>>>>>>> prefix, it should be part of package name, so it is: > >>>>>>>> o.a.h.tests.luni.util for o.a.h.luni.util > >>>>>>>> > >>>>>>>> 3. [classpath], [independent], [o.a.h.*] > >>>>>>>> I don't expect these classes/tests exist. > >>>>>>>> > >>>>>>>> 4. [classpath], [dependent], [java.*] > >>>>>>>> if my understanding is correct, this category means the tests for > >>>>>>>> intended different behavior between Harmony and RI, I consider these > >>>>>>>> tests should be put in same package with API tests, and separate > >>>>>>>> them by > >>>>>>>> running script instead of different package, because RI may fix bug > >>>>>>>> in > >>>>>>>> the revision later(e.g, Sun JDK 1.5.1 maybe?), and IIRC in another > >>>>>>>> thread we have agreed that we always follow latest version of RI, so > >>>>>>>> it > >>>>>>>> will be much easier to change the different behavior tests to API > >>>>>>>> tests > >>>>>>>> if we use script to separate them, and vice versa. > >>>>>>>> > >>>>>>>> 5. [bootclasspath], > >>>>>>>> which covers 4 categories you mentioned, the test cases are in > >>>>>>>> same > >>>>>>>> package with implementions > >>>>>>>> e.g. > >>>>>>>> java.util is test package for java.util > >>>>>>>> and > >>>>>>>> o.a.h.luni.util is test packag for o.a.h.luni.util > >>>>>>>> > >>>>>>>> As a summary, we'll have four kind of test packages: > >>>>>>>> > >>>>>>>> a. o.a.h.<module>.tests.api.java.*** for case 1,4 above > >>>>>>>> b. o.a.h.<module>.tests.*** for case 2 above > >>>>>>>> c. java.***, for case 5 above > >>>>>>>> d. o.a.h.***, for case 5 above > >>>>>>>> > >>>>>>>> comments? > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> Mikhail > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> 2006/4/27, Paulex Yang <[EMAIL PROTECTED]>: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> Mikhail Loenko wrote: > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> Geir, > >>>>>>>>>>> > >>>>>>>>>>> The problem is that no one yet suggested a consistent solution > >>>>>>>>>>> that would fit for all the tests and would not get into packages > >>>>>>>>>>> like > >>>>>>>>>>> > >>>>>>>>>>> org.apache.harmony.security.tests.org.apache.harmony.security.util > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> Everybody seems to agree that SOME implementation specific tests > >>>>>>>>>>> are > >>>>>>>>>>> in the same package as implementation. > >>>>>>>>>>> > >>>>>>>>>>> But if we have ALL the tests in the same package as > >>>>>>>>>>> implementation, > >>>>>>>>>>> then we can't run e.g. tests against java.beans.Beans from > >>>>>>>>>>> classpath, so > >>>>>>>>>>> we can't run regression test for HARMONY-358 > >>>>>>>>>>> > >>>>>>>>>>> The question is where we put 'classpath' test. If we put them > >>>>>>>>>>> into > >>>>>>>>>>> o.a.h.module.tests.[package under test] we result in that ugly > >>>>>>>>>>> package > >>>>>>>>>>> names like > >>>>>>>>>>> o.a.h.security.tests.o.a.h.security.util > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> Mikhail, > >>>>>>>>>> > >>>>>>>>>> Why my prior proposal that o.a.h.security.tests.impl.util is > >>>>>>>>>> unacceptable? > >>>>>>>>>> > >>>>>>>>>> And I have no objection if "impl" considered redundant, too. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> Mikhail > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> 2006/4/27, Geir Magnusson Jr <[EMAIL PROTECTED]>: > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>>> Geir Magnusson Jr wrote: > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>> George Harley wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> >> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>>>> Of course, the text module has only > >>>>>>>>>>>>>> "implementation-independent tests > >>>>>>>>>>>>>> that designed to be run from classpath". For modules that have > >>>>>>>>>>>>>> got > >>>>>>>>>>>>>> implementation-specific tests then I suppose we could use > >>>>>>>>>>>>>> something > >>>>>>>>>>>>>> like "org.apache.harmony.[module].tests.impl.[package under > >>>>>>>>>>>>>> test]" or > >>>>>>>>>>>>>> "org.apache.harmony.[module].tests.internal.[package under > >>>>>>>>>>>>>> test]" etc. > >>>>>>>>>>>>>> I've got no preference. > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> -1 > >>>>>>>>>>>>> > >>>>>>>>>>>>> Put them in the same package as the class being tested, please, > >>>>>>>>>>>>> if there > >>>>>>>>>>>>> is no problem being run that way. > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>> Sorry to be so snippy. Been a long day. > >>>>>>>>>>>> > >>>>>>>>>>>> Why would we want this convoluted package name? Why wouldn't we > >>>>>>>>>>>> want to > >>>>>>>>>>>> let them be in the same package as the class being tested since > >>>>>>>>>>>> it's > >>>>>>>>>>>> implementation testing? > >>>>>>>>>>>> > >>>>>>>>>>>> geir > >>>>>>>>>>>> > >>>>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Paulex Yang > >>>>>>>>>> China Software Development Lab > >>>>>>>>>> IBM > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> --------------------------------------------------------------------- > >>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>> -- > >>>>>>>> Paulex Yang > >>>>>>>> China Software Development Lab > >>>>>>>> IBM > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> --------------------------------------------------------------------- > >>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>> --------------------------------------------------------------------- > >>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>> -- > >>>>>> Paulex Yang > >>>>>> China Software Development Lab > >>>>>> IBM > >>>>>> > >>>>>> > >>>>>> > >>>>>> --------------------------------------------------------------------- > >>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>> --------------------------------------------------------------------- > >>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > >>>> -- > >>>> Paulex Yang > >>>> China Software Development Lab > >>>> IBM > >>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>>> For additional commands, e-mail: [EMAIL PROTECTED] > >>>> > >>>> > >>>> > >>>> > >>> --------------------------------------------------------------------- > >>> Terms of use : http://incubator.apache.org/harmony/mailing.html > >>> To unsubscribe, e-mail: [EMAIL PROTECTED] > >>> For additional commands, e-mail: [EMAIL PROTECTED] > >>> > >>> > >>> > >>> > >> -- > >> Paulex Yang > >> China Software Development Lab > >> IBM > >> > >> > >> > >> --------------------------------------------------------------------- > >> Terms of use : http://incubator.apache.org/harmony/mailing.html > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > >> For additional commands, e-mail: [EMAIL PROTECTED] > >> > >> > >> > > > > --------------------------------------------------------------------- > > Terms of use : http://incubator.apache.org/harmony/mailing.html > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > > > > -- > Paulex Yang > China Software Development Lab > IBM > > > > --------------------------------------------------------------------- > Terms of use : http://incubator.apache.org/harmony/mailing.html > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
