Loenko, Mikhail Y wrote:
Hi George
We will remove all the tests and these QA guys will never disturb us :)
Every time we remove a test we leave something untested.
For example, SerializationTest is a base for all the tests that check
serialization compatibility, and if we remove it because we do not
think about serialization right now we will lose that compatibility.
The same for performance, it is of importance still. Existing test suite
allows us seeking performance regressions. Otherwise we will have
to keep in sync two parallel test suites - one PerformanceTest based
just for testing the performance and another one - for unit testing.
I love this subject - unit tests an performance tests are different, as
can be regression tests, although one might argue that many of the
so-called regression tests are just unit tests that you forgot to do in
the first place.
So, what is the noise? GUI is thinking that those base classes are
tests?
Maybe it makes sense to rename PerformanceTest to e.g. PerformanceTost
and GUI will be happy?
No. :/
Can we change the output to be meaningful? I remember a while back on
another project that I knew the timings of things on my machine just
from practice, and I could tell when we changed something that impacted
performance, because the tests ran longer. This helped catch problems
early.
geir
Thanks,
Mikhail Loenko
Intel Middleware Products Division
-----Original Message-----
From: George Harley1 [mailto:[EMAIL PROTECTED]
Sent: Friday, January 13, 2006 7:24 PM
To: harmony-dev@incubator.apache.org
Subject: RE: Unit test code in HARMONY-16
Hi Mikhail,
Thanks for your very complete answer.
At some point we had different functionality in the
PerformanceTest but it seems to have died now. That is basically it.
Do you see this class (and its SecurityTest and SerializationTest
subclasses) as candidates for removal then ? When I run the security
unit
tests inside my IDE they add some extra lines to the console output
but,
since I am not thinking about performance right now, that is just
"background noise".
Perhaps additional performance-related functionality would be better
moved
out of the test class' hierarchy and into some decorator class ? That
way
would give developers a bit more flexibility running the tests with or
without the intervention of the performance measurement code. Sound
reasonable ?
Best regards,
George
________________________________________
George C. Harley
"Loenko, Mikhail Y" <[EMAIL PROTECTED]>
13/01/2006 12:12
Please respond to
harmony-dev@incubator.apache.org
To
<harmony-dev@incubator.apache.org>
cc
Subject
RE: Unit test code in HARMONY-16
As far as we did not have special performance tests, we used unit tests
to measure performance, i.e. to compare performance of our classes to
performance of "standard" classes. So we ran in cycle a single unit
test
on both when there are our security classes in bootclasspath and when
there are not. And compared time. (Of course, not all the tests passed
on "RI")
Some unit tests print various logs that make execution time volatile.
To
make it more stable we used log() instead of System.out.print() and in
the "performance mode" did not print anything. log() is defined in the
PerformanceTest. At some point we had different functionality in the
PerformanceTest but it seems to have died now. That is basically it.
The results helped us to find a number of performance leaks and improve
overall quality of the code.
Thanks,
Mikhail Loenko
Intel Middleware Products Division
-----Original Message-----
From: George Harley1 [mailto:[EMAIL PROTECTED]
Sent: Thursday, January 12, 2006 10:05 PM
To: harmony-dev@incubator.apache.org
Subject: Unit test code in HARMONY-16
Hi,
I have been looking into running the JUnit tests included in
HARMONY-16
in
my private sandbox. From what I have seen so far most (all ?) of the
test
cases inherit from a base class PerformanceTest in the
com.openintel.drl.security.test package. What is the purpose of this
base
class ?
Best regards,
George
________________________________________
George C. Harley
