Tom Eyckmans wrote:
2009/4/1 Adam Murdoch <[email protected] <mailto:[email protected]>>
Tom Eyckmans wrote:
Hi guys,
I'm working on the native test framework execution stuff,
Excellent
I just want to let you guys know how I'm trying to implement
this so you can contribute ideas/ better ways of doing things.
Global overview:
- recurse through compiled test classes, filter out class
files that contain a $ sign => testClassFilesQueue (
BlockingQueue )
- testClassFilesQueue => scan for test classes (first found
test class determines test framework) => testInfosQueue (
BlockingQueue )
- testServer / testClient
- server controls the forked test process:
- test process requests work (which test to execute
next (dequeued from testInfosQueue -> work/memory throthle),
no more work -> terminate)
- client sends progress events to the server =>
progressEventQueue => these events get duplicated to
BuildListener notifications and Test Report output, which both
need to get executed serially.
Smarter test detection. This is done on the compiled test
classes. Only test classes that don't contain a $ sign are
queued on a blocking queue for processing.
I'm pretty sure it's quite possible for a test class to have a $
in it's name - it's just a bit unusual. We should probably support
this. What was the thinking behind excluding classes with a $ sign?
I had a problem with javassist, it threw an exception and couldn't
read the files with a $ sign in them, I didn't pay much attention to
it at the time, but they can indeed contain tests in them, as
suggested by Peter I'll try ASM for the detection perhaps ASM doesn't
have this limitation.
Interesting. I've used javassist to load up files with $ in their name
before. But, if ASM works and is fast enough, then that is probably a
better option, given that we drag it in to the classpath with groovy anyway.
When a class is identified as an actual test class, an
implementation of TestInfo is queued testInfosQueue, this
object is a reference to the test class and the type of
implementation determines the way the test class needs to be
processed. This is needed to support executing JUnit TestCases
and TestSuites or TestNG Test classes and xml suite files.
When the first test class is identified the TestFramework that
identified the test class is used. This causes only that
framework to be used for test detection from that point on.
This is too dependent on the environment, when the test source
contains both types of tests. For example, on some file systems we
may find the test-ng tests first, and on some file systems we may
find the junit tests first. We should do something which behaves
the same way everywhere, something like:
- have the build file specify which framework to use for a given
test suite, with a default (ie what we do now)
- or, scan for all types of tests and assert that there is exactly
1 test framework detected.
Because of Peters feedback I don't think we should provide auto
framework detection because than we have to run detection during build
file evaluation, which seems extremely tricky to me (sourceDirs set on
end of build file??), then we don't know the testframework and don't
know which option object is going to/should be used.
It would be nice if when I don't want to configure the test execution in
any way, that my tests just get executed without me doing anything in
the build file, and regardless of whether I'm using junit or testng or both.
To control the ports used by the server processes (over
multiple gradle builds running at the same time) I'm currently
using a single file in ~/.gradle/internal/testing/ports.used
that is exclusively locked while a new port is being
determined or when a port is no longer used. Currently I'm
starting to use ports from 2000 on, I've just picked this port
so this may not be the best starting point. Ideally I want to
detect free ports so there is really no need for configuration
for ports.
A simpler option would be to let the OS select the port when
creating the listen socket in the build process,
How does this work?
Usually you can bind to port 0, and the OS will assign a port. For
example, you can create a ServerSocket with port 0, and then query which
port is actually used by calling getLocalPort().
I'm currently undecided where to do the output processing.
What do you mean by 'output processing'?
Generating the test result XML files.
I think the build process is the best place to do this, as you suggest.
I'm currently in favour of doing this in the gradle build
process as to limit the classpath of the forked vm. I think we
can re-use some of the Ant JUnit output code but not sure if
this is something that we want.
I'd love to receive feedback on this.
Thx,
Tom
---------------------------------------------------------------------
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email
