Jonathan Costers wrote:
2010/11/5 Peter Firmstone <[email protected]>
That's, quite an achievement for all involved, is that all the tests?
We have enabled most known QA test categories now. There are some left out
there, but mostly empty ones.
Some of the QA tests were deliberately set to be ignored, like the Kerberos
tests (also bc we need a KDC for that).
For jtreg it's about time I got the KDC server set up, not sure what to do
for a squid proxy server though, do you think it would be safe to run squid
on the same server with the KDC, seeing as they're there mostly for tests?
I guess we need to pose the question to INFRA: if we need external
infrastructure in place for our tests, like a KDC or a proxy server, what
would be the best way to set that up, if at all possible? I believe an
initial discussion took place at one time, but not sure what conclusion was
drawn from that (if any).
Infra provided a solaris zone:
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
-bash-3.00$ uname -a
SunOS river 5.10 Generic_137112-04 i86pc i386 i86pc
-bash-3.00$ prtconf -D
System Configuration: Sun Microsystems i86pc
Memory size: 8192 Megabytes
System Peripherals (Software Nodes):
prtconf: devinfo facility not available
The KDC and squid just need to be set up. We could establish a
registrar there also I suppose.
Setting up a separate zone just for a KDC seems like overkill to me, but I'm
not a system admin. Also considering the fact that any client (the Hudson
builders) would need Kerberos client software installed anyway.
On my machine, I installed and configured a KDC (there is a JIRA ticket
explaining what I did for that, on Ubuntu), and I am running the tests from
the same machine (pointing QA config to my KDC). That seems to work fine.
An additional hurdle for the jtreg suite is the jtreg software that would
need to be installed on the Hudson builders. Jtreg was originally meant to
regression test the JDK, and because JERI & co was originally intended to be
included in the standard Java spec (see JSR-76 and JSR-78), these jtreg
tests came to exist ... Things changed dramatically after both JSRs were
rejected, though.
Maybe we can migrate the valuable tests in that jtreg suite to either JUnit
or QA tests? I am aware there are caveats (test isolation level for
instance), but they seem to be manageable. I'm talking about a gradual
process here, converting test after test over a long period of time. We are
talking about around 100 jtreg tests, with varying complexity and isolation
levels.
Well I don't think Junit would be suitable, since multiple jvm's are
employed. I've also had to modify some jtreg tests that made some
assumptions about ClassLoader visibility (jre/lib/ext related) and
failed later when we made some changes. The qa suite might be suitable,
but I don't think the effort's worthwhile, we can't guarantee that we're
simulating the failure conditions properly, even if only a few of us run
these tests, it's better than everyone running them if they're not
genuinely simulating failure conditions.
Reducing the number of ways to test things is probably also good for general
understanding :-)
JUnit's good when we're only testing a single object implementation, we
can document and expect people to utilse the qa suite for more complex
tests.
An interesting developments for Jtreg is newer versions have had some
speed improvements recently.
Cheers,
Peter.
Regards,
Peter.
Jonathan Costers wrote:
Hooray!
All 1409 tests passed on ubuntu, taking 17hrs to run:
[java]
[java] # of tests started = 1409
[java] # of tests completed = 1409
[java] # of tests skipped = 47
[java] # of tests passed = 1409
[java] # of tests failed = 0
[java]
[java] -----------------------------------------
[java]
[java] Date finished:
[java] Thu Nov 04 03:00:44 UTC 2010
[java] Time elapsed:
[java] 62200 seconds
Thanks to all for getting RIVER-301 out of the way!
At this point, I believe we are sufficiently armed to validate some of the
interesting proposals, improvements and experiments that have been posted
to
this list the last couple of months.
Closing RIVER-301 (which had been open/in progress for a very long time)
is
also another step towards graduation from the incubator, since one of the
main goals was to setup a testing framework.
2010/11/4 Apache Hudson Server <[email protected]>
See <https://hudson.apache.org/hudson/job/River-trunk-QA/51/changes>
