at the risk of getting dragged into this ;-) I think George's objection is to the extension mechanism being used (i.e. a custom subclass rather than a standard JUnit decoration [1]).
Adding more and more subclasses is not a practical way to go if you think of all the different ways you may choose to extend the framework's behaviour. Regards, Tim [1] http://junit.sourceforge.net/javadoc/junit/extensions/TestDecorator.html Mikhail Loenko wrote: > Sorry, but I cannot catch what is the problem. > If some output is annoying please send the test name and what is wrong with > it. > PerformanceTest class does not add any output to what the tests print. > > For now the only implication from the fact that all the tests extend > PerfromanceTest > is that the security classes are well performance-tuned and on the majority of > scenarios outperform "standard" Java implementations. > > Thanks, > Mikhail Loenko > Intel Middleware Products Division > > On 1/16/06, Geir Magnusson Jr <[EMAIL PROTECTED]> wrote: > >> >>Loenko, Mikhail Y wrote: >> >>>Hi George >>> >>>We will remove all the tests and these QA guys will never disturb us :) >>> >>>Every time we remove a test we leave something untested. >>> >>>For example, SerializationTest is a base for all the tests that check >>>serialization compatibility, and if we remove it because we do not >>>think about serialization right now we will lose that compatibility. >>> >>>The same for performance, it is of importance still. Existing test suite >>>allows us seeking performance regressions. Otherwise we will have >>>to keep in sync two parallel test suites - one PerformanceTest based >>>just for testing the performance and another one - for unit testing. >> >>I love this subject - unit tests an performance tests are different, as >>can be regression tests, although one might argue that many of the >>so-called regression tests are just unit tests that you forgot to do in >>the first place. >> >> >>>So, what is the noise? GUI is thinking that those base classes are >>>tests? >>>Maybe it makes sense to rename PerformanceTest to e.g. PerformanceTost >>>and GUI will be happy? >> >>No. :/ >> >>Can we change the output to be meaningful? I remember a while back on >>another project that I knew the timings of things on my machine just >>from practice, and I could tell when we changed something that impacted >>performance, because the tests ran longer. This helped catch problems >>early. >> >>geir >> >> >> >>>Thanks, >>>Mikhail Loenko >>>Intel Middleware Products Division >>> >>> >>>>-----Original Message----- >>>>From: George Harley1 [mailto:[EMAIL PROTECTED] >>>>Sent: Friday, January 13, 2006 7:24 PM >>>>To: harmony-dev@incubator.apache.org >>>>Subject: RE: Unit test code in HARMONY-16 >>>> >>>>Hi Mikhail, >>>> >>>>Thanks for your very complete answer. >>>> >>>> >>>>>At some point we had different functionality in the >>>>>PerformanceTest but it seems to have died now. That is basically it. >>>> >>>>Do you see this class (and its SecurityTest and SerializationTest >>>>subclasses) as candidates for removal then ? When I run the security >>> >>>unit >>> >>>>tests inside my IDE they add some extra lines to the console output >>> >>>but, >>> >>>>since I am not thinking about performance right now, that is just >>>>"background noise". >>>> >>>>Perhaps additional performance-related functionality would be better >>> >>>moved >>> >>>>out of the test class' hierarchy and into some decorator class ? That >>> >>>way >>> >>>>would give developers a bit more flexibility running the tests with or >>>>without the intervention of the performance measurement code. Sound >>>>reasonable ? >>>> >>>>Best regards, >>>>George >>>>________________________________________ >>>>George C. Harley >>>> >>>> >>>> >>>> >>>>"Loenko, Mikhail Y" <[EMAIL PROTECTED]> >>>>13/01/2006 12:12 >>>>Please respond to >>>>harmony-dev@incubator.apache.org >>>> >>>> >>>>To >>>><harmony-dev@incubator.apache.org> >>>>cc >>>> >>>>Subject >>>>RE: Unit test code in HARMONY-16 >>>> >>>> >>>> >>>> >>>> >>>> >>>>As far as we did not have special performance tests, we used unit tests >>>>to measure performance, i.e. to compare performance of our classes to >>>>performance of "standard" classes. So we ran in cycle a single unit >>> >>>test >>> >>>>on both when there are our security classes in bootclasspath and when >>>>there are not. And compared time. (Of course, not all the tests passed >>>>on "RI") >>>> >>>>Some unit tests print various logs that make execution time volatile. >>> >>>To >>> >>>>make it more stable we used log() instead of System.out.print() and in >>>>the "performance mode" did not print anything. log() is defined in the >>>>PerformanceTest. At some point we had different functionality in the >>>>PerformanceTest but it seems to have died now. That is basically it. >>>> >>>>The results helped us to find a number of performance leaks and improve >>>>overall quality of the code. >>>> >>>>Thanks, >>>>Mikhail Loenko >>>>Intel Middleware Products Division >>>> >>>> >>>> >>>>>-----Original Message----- >>>>>From: George Harley1 [mailto:[EMAIL PROTECTED] >>>>>Sent: Thursday, January 12, 2006 10:05 PM >>>>>To: harmony-dev@incubator.apache.org >>>>>Subject: Unit test code in HARMONY-16 >>>>> >>>>>Hi, >>>>> >>>>>I have been looking into running the JUnit tests included in >>> >>>HARMONY-16 >>> >>>>in >>>> >>>>>my private sandbox. From what I have seen so far most (all ?) of the >>>> >>>>test >>>> >>>>>cases inherit from a base class PerformanceTest in the >>>>>com.openintel.drl.security.test package. What is the purpose of this >>>> >>>>base >>>> >>>>>class ? >>>>> >>>>>Best regards, >>>>>George >>>>>________________________________________ >>>>>George C. Harley >>> >>> > -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
