Re: JUnit vs. jtreg (Was: Build failed in Hudson: River-trunk #163)

Tue, 28 Apr 2009 14:41:50 -0700 

Peter Firmstone wrote:

Thanks Tom,

I like your advise,  will respond later.

Cheers,

Peter.

Tom Hobbs wrote:

Aha, I see.  I suppose I should pay more attention to the entire
conversation.  :-)  I hope I didn't come across as being to
aggressive/dismissive/stupid (pick all that apply). Please be patient whilst I amend my advice. 

If you're in the process of writing a unit test, then check if there's a
similar jtreg test which covers what you're doing. - If so, I would say modify/copy that since it'll probably be less work. 

- If not, then go with JUnit, as Jeff Ramsdale said, JUnit has better
tool support than jtreg (which I've never heard of either).  Also,
(probably) more developers are familiar with it which removes a future
barrier to entry for them.

Some advice I got a few months back when trying to get involved was to
make a decision and get stuck in.  Waiting for others to help make
decisions, do some analysis and so on will most likely mean waiting a
long time.  If JUnit suits your purposes/skills right now, go with that,
especially in this case where there has been no clear advice from people
who know more than us that this is a wrong approach.
I wouldn't worry about fully understanding the implications of
everything before making a move.  The best way to fully explore whether
you're using the right tool or not is to try and use it for real.  Write
and submit a JUnit test, if it works then there's your analysis done
already.
If it makes you feel any better, I've got some ideas up my sleeve which
I hope to start submitting soon and they'll be JUnit tested (rather than
jtreg).
It does, actually, I was worried I might be the only one utilising junit, I was wondering why there hadn't been any unit tests created for River to date. 
I've written JUnit tests for service code before so I know it
works, I've not looked at any depth at the ClassDep stuff so I can't say
if it's going to be equally good for that.

Speaking of which, which version of JUnit are you using (since that'll
be quite important for the build scripts) and are you using a mocking
tool, if so, which one?  Let me know and I'll make sure I use the same
versions.
JUnit version 4.5, I've also got versions 4.1, 3.8.2 and 3.8.1 on my system, I'll go with whatever you choose. 

Whats a mocking Tool?



Cheers,

Tom

-----Original Message-----
From: Peter Firmstone [mailto:[email protected]] Sent: 28 April 2009 01:45 
To: [email protected]
Subject: Re: JUnit vs. jtreg (Was: Build failed in Hudson: River-trunk
#163)

Here's some background:
I was writing junit tests for the new ClassDep implementation on the understanding that River didn't have any existing unit testing. Then reading the river-dev conversation between Jonathan and Peter, I discover that River has existing unit and regression tests. So I posed the question: Should I be writing the unit tests using jtreg instead of junit?
I've decided for the time being that until I feel more comfortable, being the person who has just requested a test directory be created for the junit tests, that we fully understand the implications of doing this
before just jumping in. I'm not saying we don't do it, just too much too fast for me that's all. 

Cheers,

Peter.


Tom Hobbs wrote:

I'm not proposing to reimplement or remove anything.  Lets just put
these tests as part of the build, the theory being that they /should/
all pass.

Then adding/changing River behaviour should then provoke a detailed
critique of any jtregs that might then fail.  That critique would then
conclude to modify/remove the affected jtreg only and it would be done
on a very specific basis.

There is no reason why /new/ code and /new/ bugs cannot be covered

using

a more common (junit?) tool.  Of course, this assumes that the

community

agrees to adopt the more common tool over /extending/ the jtreg
coverage.

To be honest, I haven't had a chance to read any of the jtreg code

yet.

I'm just adverse to creating more work (reimplementing them) or

creating
danger (deleting them). I don't really understand what the discussion about them is, can't we 
just include them in the build script and see what happens?  What am I
missing?

Why one or the other?  Why not both with the understanding the jtreg

is

historic and junit is new stuff?

Cheers,

Tom

-----Original Message-----
From: Peter Firmstone [mailto:[email protected]] Sent: 26 April 2009 11:00 
To: [email protected]
Subject: Re: JUnit vs. jtreg (Was: Build failed in Hudson: River-trunk
#163)

Good to have you back Tom,

What I'm afraid of, is an uninformed decision being made, we need to
do 
our homework here, so please dive in and check out the code.  Thanks
to
Jonathan asking the right questions, and someone with the knowledge replying (thank you Peter Jones for responding) we've now found out there are existing Regression and Unit tests that rely on jtreg, in addition to the Integration test kit that is now runnable from an ant target. 

Regression tests are created every time a bug is identified in River's



API, the conditions under which it occurs are recreated in one or more
regression tests. Later when that part of the API is reimplemented, (assuming also that we're not deliberately breaking backward compatibility) the regression tests confirm that a previously 
identified
bug hasn't been reintroduced. This is important, some of the finer implementation details may not be apparent to the future developer,
the 
regression test allows us to ensure the bug doesn't get reintroduced
at 
a later date.  This is obviously very important for security bugs too.

A problem with reimplementing the regression tests with another tool
is
that we would need the old code that included the bug to confirm that the new regression test implementation does in fact reproduce the bug, 


otherwise our efforts would be for naught.

I have a feeling that if we decide to no longer use jtreg for unit

tests
that we may still need to use it for Regression testing, however I'd like to reserve any further judgment on my part until my understanding 


improves further.

Tom hobbs wrote:

Hi guys,

Sorry for my recent silence.
These jtreg doo-hickeys sound like the QA tests to me. They're something we're inherited and not really sure what to do with them, 
so

I suggest we treat them like the QA tests.
Assuming it's not too difficult, lets get running the jtregs as part of the build. As River functionality is added/changed and individual 


jtregs start breaking whoever made the change should assume that it's
their responsibility to modify or remove the jtreg (and QA test) as appropriate. 

Meanwhile, new code gets junit tested.

Thoughts?

Also, +1 Java 5.  (Java 6 I can take or leave).

Cheers,

Tom



On Sat, 25 Apr 2009 11:50:10 -0000, Peter Firmstone

<[email protected]>

wrote:


I hear you.
How about we step back, postpone making a decision until we better understand just what we've inherited then weigh up the pro's and con's? I haven't seen anyone in a hurry to write junit tests yet, a little wait while we investigate jtreg can't do any harm? ;) It took
me about 1 hour to download, install and get jtreg running, having never heard of it before, the tests themselves seem simple, remember 


I'm one of your new developers.
I'm going to sit on the fence for now have a play and write test cases for ClassDep using both tools for my own understanding.
We're going to have to get to know jtreg to make an informed decision, here's some info: 

All tests are placed in the directory tree under qa/jtreg/

Ant can be used to run all tests, I'll post the build.xml file after



I write it.

The following I've copied from the jtreg website:

Ant Examples:

<jtreg dir="test/tools/javac"/>
Run all the tests in the test/tools/javac directory, using the same JDK being used to run Ant, the default work and report directories, and running each test in a separate VM. 

<jtreg dir="test/tools/javac" workDir="myWorkDir"/>

As before, but specifying a work directory for the results.

<jtreg dir="test/tools/javac" workDir="myWorkDir">
    <include name="api/*.java" />
</jtreg>

Run just the api/*.java tests.

<jtreg>
    <arg line="-w myWorkDir -jdk /java/jdk/1.5.0 test/tools/javac">
</jtreg>
Run the test/tools/javac tests using /java/jdk/1.5.0 writing the results to myWorkDir. 

2. Writing a JDK Regression Test
2.1. How do I write a test?
The simplest test is an ordinary Java program with the usual static main method. If the test fails, it should throw an exception; if it succeeds, it should return normally. 

Here's an example:

                            /* @test 1.1 97/10/12
                               @bug 1234567
                               @summary Make sure that 1 != 0
                            */

                            public class OneZero {
public static void main(String[] args) throws Exception { 
                                    if (1 == 0) {
                                        throw new Exception("1 ==

0");

                                    }
                                }

                            }
The @test tag identifies a source file that defines a test. the harness will automatically run any .java, .sh, and .html file containing an @test tag within the appropriate comment; it ignores any file not containing such a tag or not utilizing one of the expected extensions.
If necessary the harness will compile the source file, if the class files are older than the corresponding source files. Other files which the test depends on must be specified with the @run build 
action.
The arguments to the @test tag are ignored by the harness. For identification it's useful to put the usual SCCS ID keywords (I and E, each letter surrounded by %) after the @test tag.
While not part of the tag specification, some tests use the string "/nodynamiccopyright" after @test to indicate that that the file should not be subject to automated copyright processing that might affect the operation of the test, for example, by affecting the line 


numbers of the test source code.
2.3. What do the other tags mean?

The other tags shown above are optional.

The @bug tag should be followed by one or more bug numbers,

separated

by spaces. The bug number is useful in diagnosing test failures.

It's

OK to write tests that don't have bug numbers, but if you're writing



a test for a specific bug please include its number in an @bug tag.

The @summary tag describes the condition that is checked by the

test.

It is especially useful for non-regression tests, which by

definition

don't have bug numbers, but even if there's a bug number it's

helpful
to include a summary. Note that a test summary is generally not the same thing as a Bugtraq synopsis, since the latter describes the bug 


rather than the condition that the bug violates.



Jeff Ramsdale wrote:

Not to mention tool support--IDEs, etc...
-j
On Fri, Apr 24, 2009 at 6:13 AM, Jukka Zitting <[email protected]>wrote: 



Hi,

On Fri, Apr 24, 2009 at 10:06 AM, Peter Firmstone

<[email protected]>

wrote:


It appears to me that I should perhaps be writing jtreg style

unit

tests


for


ClassDep rather than JUnit?


I'm not so sure about that. I can't tell which approach is better
technically as I don't know jtreg, but from a community building
perspective it would be much better if we used something that the

vast

majority of the Java world is already familiar with.

BR,

Jukka Zitting



www.sucden.co.uk
Sucden (UK) Limited, 5 London Bridge Street, London SE1 9SG
Telephone +44 20 7940 9400
Registered in England no. 1095841 
VAT registration no. GB 446 9061 33
Authorised and Regulated by the Financial Services Authority (FSA) and

entered in the FSA register under no. 114239

This email, including any files transmitted with it, is confidential

and may be privileged. It may be read, copied and used only by the
intended recipient. If you are not the intended recipient of this
message, please notify [email protected] immediately and delete it
from your computer system.

We believe, but do not warrant, that this email and its attachments
are virus-free, but you should check. 
Sucden (UK) Ltd may monitor traffic data of both business and personal

emails. By replying to this email, you consent to Sucden's monitoring
the content of any emails you send to or receive from Sucden. Sucden is
not liable for any opinions expressed by the sender where this is a
non-business email.

The contents of this e-mail do not constitute advice and should not be

regarded as a recommendation to buy, sell or otherwise deal with any
particular investment.

This message has been scanned for viruses by Mimecast.


www.sucdenfinancial.com
Sucden Financial Limited, Plantation Place South, 60 Great Tower Street, London EC3R 5AZ 
Telephone +44 203 207 5000

Registered in England no. 1095841
VAT registration no. GB 446 9061 33
Authorised and Regulated by the Financial Services Authority (FSA) and entered in the FSA register under no. 114239
This email, including any files transmitted with it, is confidential and may be privileged. It may be read, copied and used only by the intended recipient. If you are not the intended recipient of this message, please notify [email protected] immediately and delete it from your computer system.
We believe, but do not warrant, that this email and its attachments are virus-free, but you should check.
Sucden Financial Limited may monitor traffic data of both business and personal emails. By replying to this email, you consent to Sucden Financial 's monitoring the content of any emails you send to or receive from Sucden Financial . Sucden Financial is not liable for any opinions expressed by the sender where this is a non-business email.
The contents of this e-mail do not constitute advice and should not be regarded as a recommendation to buy, sell or otherwise deal with any particular investment. 

This message has been scanned for viruses by Mimecast.

Reply via email to