Brett,
I think you are right on with the two important issues being how to identify
the tests and the execution model for running the tests. One of the
difficulties in real-time reporting for NMaven is that the nunit-console
either spits to the console or can redirect all of the output to file. In
the later case we wouldn't be able to report until after all tests are done,
which doesn't add much value. In the case of extracting the info to report
from the console, parsing particular strings seems a bit fragile. It seems
that we need some sort of bridge, essentially a NUnit-Runner for java.
It seems as long as we rely on running nunit-console to execute our nunit
tests, it will be difficult to neatly plug into surefire.
Thoughts?
Evan
On 6/5/07, Brett Porter <[EMAIL PROTECTED]> wrote:
Yeah, it's going to take a close look and possibly surefire changes
(though this is the best time to make them) if it is feasible at all.
At first glance, it looks possible to me.
The key things are really that:
a) it can identify where the test sets are from java.
b) how the forking of the test execution is going to work.
Currently the forking is (just) out of alignment with where you'd
want it for NMaven. You'd want it to do that and only run the tests -
but currently the reporting happens in the forked instance too.
This isn't critical, as we could get around this by forcing the
surefire plugin to "never" fork and just fork inside the test
executor - if that works out we could look at shifting the design in
surefire and that would give nunit tests the ability to do fork once,
fork always (instead of what I assume will be fork always otherwise).
wdyt?
- Brett
On 06/06/2007, at 10:40 AM, Evan Worley wrote:
> So I looked into this briefly and it seems that the current surefire
> providers rely heavily on java reflection. We will be working with
> nunit
> tests on which we cannot use java reflection. Does anyone have any
> idea how
> we might wrap nunit as a surefire provider? Currently the nunit
> plugin
> invokes a command line which runs the nunit binaries and logs/
> analyzes the
> results.
>
> Thanks for any help,
> Evan
>
> On 6/4/07, Evan Worley <[EMAIL PROTECTED]> wrote:
>>
>> Interesting question, I have just been manually transforming the
>> nunit
>> output to look like the junit. However if we could squeeze nunit
>> into a
>> surefire provider, that would be that much better.
>>
>> When you are building the same component in java and C#, these
>> inconsistencies are very noticeable, so it would be great to have
>> both
>> platforms build output consistent.
>>
>> I will look into a nunit surefire provider.
>>
>> Thanks,
>> Evan
>>
>> On 6/4/07, Brett Porter <[EMAIL PROTECTED] > wrote:
>> >
>> > I question I'd been meaning to ask - is it possible to fit nunit in
>> > as a surefire provider instead of a separate plugin?
>> >
>> > This automatically buys this plus reporting integration.
>> >
>> > - Brett
>> >
>> > On 05/06/2007, at 1:32 PM, Evan Worley wrote:
>> >
>> > > Hi All,
>> > >
>> > > I was thinking there would be some value in doing some work on
>> the
>> > > nunit
>> > > plugin to add some output similar to the junit plugin. Currently
>> > > when nunit
>> > > tests run, all the output is logged to file. It is not too much
>> > > fun when
>> > > your tests run for a few minutes, you see nothing. Here is a
>> junit
>> > > output
>> > > vs nunit output comparison
>> > >
>> > > -- JUNIT --
>> > > Running package1.TestClass1
>> > > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
>> > > 0.016 sec
>> > > Running package2.TestClass2
>> > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
>> > > 0.031 sec
>> > > .
>> > > .
>> > > .
>> > > Results :
>> > >
>> > > Tests run: 139, Failures: 0, Errors: 0, Skipped: 0
>> > >
>> > > -- NUNIT --
>> > > NMAVEN-040-000: Executed command: Commandline = nunit-console
>> C:\dev
>> > > \project
>> > > \main\component\target\test-assemblies\Namespace.Artifact.dll /
>> out
>> > > {SOME_OUTPUT_FILE} /err {SOME_OUTPUT_FILE}, Result = 0
>> > >
>> > > So I propose not logging the nunit stdout/stderr but rather
>> > > reformatting and
>> > > displaying it like the junit plugin does.
>> > >
>> > > Thoughts?
>> > > Evan
>> >
>>
>>
