Re: [classlib] testing guidlines -- support classes
I think that we are really struggling here to represent the different combination of tests using the file system directory. So far I think we have : - api tests on application classpath - impl tests on bootclasspath - impl tests on application classpath - support classes - platform specific classes - failing tests (even temp failing and always failing) - resources used in tests and I'm sure we will find more categories as we go. IMHO trying to use a combination of directory and file names to separate these is flawed. Each new categorization adds another layer in source tree layout, and added complexity. I still believe that we should be using suites to organize the tests into groups, irrespective of how they are stored on disk. In this case, categorization is by suite membership, not based on file path. Glad I got that off my chest. Regards, Tim Mikhail Loenko wrote: 2006/7/3, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: All: support classes, impl-classpath, impl-boot, api-classpath, and api-boot tests are compiled into separate directories. ( TestNG looking better by the hour eh? ;-) ) Support classes are compiled first, tests are compiled with support classes in the classpath. Presumably you don't mind if they are all compiled together? Just figuring out how to set up my IDE descriptions. When the tests are running, support classes are accessible the same way as the tests, i.e. via bootclasspath when bootclasspath tests are running and in classpath when classpath tests are running. Ok. If nobody object I can add to the doc something like: Support classes for the tests might be located both within the tests and in the support directory. Support classes that are shared between different types of the tests are recommended to be in the support dir. It may be worth describing what you mean by a 'support class'. Ok When the bootclasspath tests are running all classes in the support dir are available by bootclasspath. Otherwise they are available by classpath. via classpath. A simple example would help too. not sure what you mean... Thanks, Mikhail Regards, Tim 2006/7/3, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: That means that all the API tests will be in the bootclasspath when impl/bootclasspath tests run? Will this run be clear enough? No I don't think that will be clear. So can you describe how the bootclasspath and classpath are set up for running each set of tests? Maybe we should add that info to the document? Also I'm left wondering how the support classes should be compiled? Presumably they can be compiled against the APIs only? Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM
Re: [classlib] testing guidlines -- support classes
2006/7/3, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: All: support classes, impl-classpath, impl-boot, api-classpath, and api-boot tests are compiled into separate directories. ( TestNG looking better by the hour eh? ;-) ) Support classes are compiled first, tests are compiled with support classes in the classpath. Presumably you don't mind if they are all compiled together? Just figuring out how to set up my IDE descriptions. When the tests are running, support classes are accessible the same way as the tests, i.e. via bootclasspath when bootclasspath tests are running and in classpath when classpath tests are running. Ok. If nobody object I can add to the doc something like: Support classes for the tests might be located both within the tests and in the support directory. Support classes that are shared between different types of the tests are recommended to be in the support dir. It may be worth describing what you mean by a 'support class'. Ok When the bootclasspath tests are running all classes in the support dir are available by bootclasspath. Otherwise they are available by classpath. via classpath. A simple example would help too. not sure what you mean... Thanks, Mikhail Regards, Tim 2006/7/3, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: That means that all the API tests will be in the bootclasspath when impl/bootclasspath tests run? Will this run be clear enough? No I don't think that will be clear. So can you describe how the bootclasspath and classpath are set up for running each set of tests? Maybe we should add that info to the document? Also I'm left wondering how the support classes should be compiled? Presumably they can be compiled against the APIs only? Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] testing guidlines -- support classes
Mikhail Loenko wrote: That means that all the API tests will be in the bootclasspath when impl/bootclasspath tests run? Will this run be clear enough? No I don't think that will be clear. So can you describe how the bootclasspath and classpath are set up for running each set of tests? Maybe we should add that info to the document? Also I'm left wondering how the support classes should be compiled? Presumably they can be compiled against the APIs only? Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] testing guidlines -- support classes
All: support classes, impl-classpath, impl-boot, api-classpath, and api-boot tests are compiled into separate directories. Support classes are compiled first, tests are compiled with support classes in the classpath. When the tests are running, support classes are accessible the same way as the tests, i.e. via bootclasspath when bootclasspath tests are running and in classpath when classpath tests are running. If nobody object I can add to the doc something like: Support classes for the tests might be located both within the tests and in the support directory. Support classes that are shared between different types of the tests are recommended to be in the support dir. When the bootclasspath tests are running all classes in the support dir are available by bootclasspath. Otherwise they are available by classpath. via classpath. Thanks, Mikhail 2006/7/3, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: That means that all the API tests will be in the bootclasspath when impl/bootclasspath tests run? Will this run be clear enough? No I don't think that will be clear. So can you describe how the bootclasspath and classpath are set up for running each set of tests? Maybe we should add that info to the document? Also I'm left wondering how the support classes should be compiled? Presumably they can be compiled against the APIs only? Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] testing guidlines -- support classes
Mikhail Loenko wrote: All: support classes, impl-classpath, impl-boot, api-classpath, and api-boot tests are compiled into separate directories. ( TestNG looking better by the hour eh? ;-) ) Support classes are compiled first, tests are compiled with support classes in the classpath. Presumably you don't mind if they are all compiled together? Just figuring out how to set up my IDE descriptions. When the tests are running, support classes are accessible the same way as the tests, i.e. via bootclasspath when bootclasspath tests are running and in classpath when classpath tests are running. Ok. If nobody object I can add to the doc something like: Support classes for the tests might be located both within the tests and in the support directory. Support classes that are shared between different types of the tests are recommended to be in the support dir. It may be worth describing what you mean by a 'support class'. When the bootclasspath tests are running all classes in the support dir are available by bootclasspath. Otherwise they are available by classpath. via classpath. A simple example would help too. Regards, Tim 2006/7/3, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: That means that all the API tests will be in the bootclasspath when impl/bootclasspath tests run? Will this run be clear enough? No I don't think that will be clear. So can you describe how the bootclasspath and classpath are set up for running each set of tests? Maybe we should add that info to the document? Also I'm left wondering how the support classes should be compiled? Presumably they can be compiled against the APIs only? Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
Re: [classlib] testing guidlines -- support classes
Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] testing guidlines -- support classes
I'm refereing to those support classes that are used by both API and impl tests 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] testing guidlines -- support classes
Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib] testing guidlines -- support classes
That means that all the API tests will be in the bootclasspath when impl/bootclasspath tests run? Will this run be clear enough? 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: I'm refereing to those support classes that are used by both API and impl tests Sure, but if the support classes themselves only use API then they can be in the api dir right? i.e. we expect our impl tests to use some APIs too. Regards, Tim 2006/6/30, Tim Ellison [EMAIL PROTECTED]: Mikhail Loenko wrote: There are support classes that are shared by various types of the tests, e.g. api and impl or classpath and bootclasspath We can either separate them from the tests or duplicate. I'm confused. Either the support classes are used by API tests and only make API calls into the code, or they are harmony implementation-specific; and either they need to be loaded on the bootclasspath or not -- right? So can't they just go into the existing bin directory classifications? Regards, Tim I suggest that we separate these classes into folder src/test/support/[platform]/java and org.apache.harmony.module.tests.support.* package. Probably it makes sense to put all the module's support classes there (not only shared ones). Thoughts? Thanks, Mikhail - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
