Re: [classlib][test] fail statements omitted in many exception catching test cases
Thanks Richard.
I've looked at luni module and raised HARMONY-1618 about some bugs of those
kind.
On 9/28/06, Richard Liang [EMAIL PROTECTED] wrote:
Great job, Robert. ;-)
I will have a look at sql.
On 9/28/06, Rui Hu [EMAIL PROTECTED] wrote:
Great Mark!
I've merged your code into my script. The script can be downloaded at
[1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/attachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly
stricter
on matching context around 'catch'/'fail', by handling comments
slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java
+208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
--
Robert Hu
China Software Development Lab, IBM
[OT] svnfind tool (was: Re: [classlib][test] fail statements omitted in many exception catching test cases)
Testing this script reminded me that I'd not mentioned the small script
I wrote after getting feed up typing:
find path -name .svn -prune -o ...
The script takes:
svnfind path [path2 ...] ...
and converts it to the previous form so that .svn directories are ignored.
Others might find it useful so I've included it below.
-Mark.
#!/bin/sh
while [ -n $1 ]; do
case $1 in
-*)
break
;;
*)
[EMAIL PROTECTED]$1
;;
esac
shift
done
while [ -n $1 ]; do
[EMAIL PROTECTED]$1
shift
done
if [ -z ${args[0]} ]; then
args[0]=-print
fi
exec find [EMAIL PROTECTED] -name .svn -prune -o [EMAIL PROTECTED]
On 27 September 2006 at 9:44, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly stricter
on matching context around 'catch'/'fail', by handling comments slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java +208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~ /\bfail\s*\(/)
{
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
On 27 September 2006 at 15:02, Robert Hu [EMAIL PROTECTED] wrote:
--090601000506020908060004
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=GB2312
Hi All,
In our unit test of classlib, there are huge number of test cases about exc
ep
tion catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the excepti
on
-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of our H
ar
mony classlib. So I set some rules to find out all lines of code related wi
th
it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules in
r
450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of above
s
tyle.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.jav
a:
652\658\664\670\676\685\698\704\711(line number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTes
t.
java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\File
In
putStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\File
Ou
tputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind withou
t
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those bugs?
Pl
s. share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
--090601000506020908060004
Content-Type: text/plain;
name=result.txt
Content-Disposition: inline;
filename=result.txt
Content-Transfer-Encoding: quoted-printable
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarF
=
ileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl
=
aterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl
=
aterTest.java:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\10
=
99\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipF
=
ileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipI
=
nputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallba
=
ckTest.java:60\147
Re: [classlib][test] fail statements omitted in many exception catching test cases
Fixed.
Rui Hu wrote:
Thanks Richard.
I've looked at luni module and raised HARMONY-1618 about some bugs of those
kind.
On 9/28/06, Richard Liang [EMAIL PROTECTED] wrote:
Great job, Robert. ;-)
I will have a look at sql.
On 9/28/06, Rui Hu [EMAIL PROTECTED] wrote:
Great Mark!
I've merged your code into my script. The script can be downloaded at
[1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/attachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly
stricter
on matching context around 'catch'/'fail', by handling comments
slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java
+208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
--
Tim Ellison ([EMAIL PROTECTED])
IBM Java technology centre, UK.
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib][test] fail statements omitted in many exception catching test cases
May be you can look at my JIRA too? Thanks :)
HARMONY-1604
2006/9/28, Tim Ellison [EMAIL PROTECTED]:
Fixed.
Rui Hu wrote:
Thanks Richard.
I've looked at luni module and raised HARMONY-1618 about some bugs of those
kind.
On 9/28/06, Richard Liang [EMAIL PROTECTED] wrote:
Great job, Robert. ;-)
I will have a look at sql.
On 9/28/06, Rui Hu [EMAIL PROTECTED] wrote:
Great Mark!
I've merged your code into my script. The script can be downloaded at
[1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/attachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly
stricter
on matching context around 'catch'/'fail', by handling comments
slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java
+208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
--
Alexei Zakharov,
Intel Middleware Product Division
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [OT] svnfind tool (was: Re: [classlib][test] fail statements omitted in many exception catching test cases)
can you stuff this somewhere like the wiki or patch for website or
stuff somewhere in SVN? :)
geir
On Sep 28, 2006, at 7:53 AM, Mark Hindess wrote:
Testing this script reminded me that I'd not mentioned the small
script
I wrote after getting feed up typing:
find path -name .svn -prune -o ...
The script takes:
svnfind path [path2 ...] ...
and converts it to the previous form so that .svn directories are
ignored.
Others might find it useful so I've included it below.
-Mark.
#!/bin/sh
while [ -n $1 ]; do
case $1 in
-*)
break
;;
*)
[EMAIL PROTECTED]$1
;;
esac
shift
done
while [ -n $1 ]; do
[EMAIL PROTECTED]$1
shift
done
if [ -z ${args[0]} ]; then
args[0]=-print
fi
exec find [EMAIL PROTECTED] -name .svn -prune -o [EMAIL PROTECTED]
On 27 September 2006 at 9:44, Mark Hindess
[EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly
stricter
on matching context around 'catch'/'fail', by handling comments
slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/
TimeTest.java +208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~ /
\bfail\s*\(/)
{
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
On 27 September 2006 at 15:02, Robert Hu [EMAIL PROTECTED]
wrote:
--090601000506020908060004
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=GB2312
Hi All,
In our unit test of classlib, there are huge number of test cases
about exc
ep
tion catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because
the excepti
on
-throwing checking is disabled.
I've found that the fail statement is omitted in many test
cases of our H
ar
mony classlib. So I set some rules to find out all lines of code
related wi
th
it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the
5 rules in
r
450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are
not of above
s
tyle.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font
\TextLayoutTest.jav
a:
652\658\664\670\676\685\698\704\711(line number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java
\lang\EnumTes
t.
java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests
\java\io\File
In
putStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests
\java\io\File
Ou
tputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this
kind withou
t
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more
bugs.
Anybody has better search rules or better solution to find out
those bugs?
Pl
s. share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
--090601000506020908060004
Content-Type: text/plain;
name=result.txt
Content-Disposition: inline;
filename=result.txt
Content-Transfer-Encoding: quoted-printable
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util
\jar\JarF
=
ileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util
\zip\Defl
=
aterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util
\zip\Defl
=
aterTest.java:188\619\724\785\792\859\1006\1013\1070\1077\1091
\1092\1098\10
=
99\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util
\zip\ZipF
=
ileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util
\zip\ZipI
=
nputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback
\ConfirmationCallba
=
Re: [classlib][test] fail statements omitted in many exception catching test cases
I'll take a look.
-Mark.
On 28 September 2006 at 17:31, Alexei Zakharov [EMAIL PROTECTED] wrote:
May be you can look at my JIRA too? Thanks :)
HARMONY-1604
2006/9/28, Tim Ellison [EMAIL PROTECTED]:
Fixed.
Rui Hu wrote:
Thanks Richard.
I've looked at luni module and raised HARMONY-1618 about some bugs of tho
se
kind.
On 9/28/06, Richard Liang [EMAIL PROTECTED] wrote:
Great job, Robert. ;-)
I will have a look at sql.
On 9/28/06, Rui Hu [EMAIL PROTECTED] wrote:
Great Mark!
I've merged your code into my script. The script can be downloaded at
[1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/at
tachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly
stricter
on matching context around 'catch'/'fail', by handling comments
slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.jav
a
+208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
--
Alexei Zakharov,
Intel Middleware Product Division
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
-
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [classlib][test] fail statements omitted in many exception catching test cases
I have raised some JIRAs for SQL. Could any one have a look at them?
Thanks a lot.
http://issues.apache.org/jira/browse/HARMONY-1616
http://issues.apache.org/jira/browse/HARMONY-1619
http://issues.apache.org/jira/browse/HARMONY-1642
http://issues.apache.org/jira/browse/HARMONY-1643
Best regards,
Richard
On 9/28/06, Richard Liang [EMAIL PROTECTED] wrote:
Great job, Robert. ;-)
I will have a look at sql.
On 9/28/06, Rui Hu [EMAIL PROTECTED] wrote:
Great Mark!
I've merged your code into my script. The script can be downloaded at [1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/attachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly stricter
on matching context around 'catch'/'fail', by handling comments slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java +208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
On 27 September 2006 at 15:02, Robert Hu [EMAIL PROTECTED] wrote:
--090601000506020908060004
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=GB2312
Hi All,
In our unit test of classlib, there are huge number of test cases about
excep
tion catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the
exception
-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of
our Har
mony classlib. So I set some rules to find out all lines of code related
with
it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules
in r
450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of
above s
tyle.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:
652\658\664\670\676\685\698\704\711(line number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.
java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileIn
putStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOu
tputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind
without
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those
bugs? Pl
s. share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
--090601000506020908060004
Content-Type: text/plain;
name=result.txt
Content-Disposition: inline;
filename=result.txt
Content-Transfer-Encoding: quoted-printable
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarF=
ileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterTest.java
:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\10=
[classlib][test] fail statements omitted in many exception catching test cases
Hi All,
In our unit test of classlib, there are huge number of test cases about
exception catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the
exception-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of our
Harmony classlib. So I set some rules to find out all lines of code related
with it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules in
r450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of above
style.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:652\658\664\670\676\685\698\704\711(line
number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileInputStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOutputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind without
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those bugs? Pls.
share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarFileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\DeflaterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\DeflaterTest.java:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\1099\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipFileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipInputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallbackTest.java:60\147
.\auth\src\test\java\common\javax\security\auth\callback\LanguageCallbackTest.java:60
.\auth\src\test\java\common\javax\security\auth\kerberos\ServicePermissionTest.java:302
.\auth\src\test\java\common\javax\security\auth\PolicyTest.java:132
.\auth\src\test\java\common\javax\security\auth\PrivateCredentialPermissionTest.java:267\386
.\auth\src\test\java\common\javax\security\auth\SubjectTest.java:239\248\257\269\992\1109\1301\1314\1423\1851\2049\2061\2089\2101\2127\2139
.\auth\src\test\java\common\javax\security\auth\x500\X500PrincipalTest.java:2404
.\auth\src\test\java\common\javax\security\auth\x500\X500PrivateCredentialTest.java:169\179\189
.\auth\src\test\java\common\org\apache\harmony\auth\internal\SecurityTest.java:807\855\950\971\987\995\1003\1011\1065\1089\1090\1101\1116\1125\1133\1141\1142\1151\1152\1161\1162
.\auth\src\test\java\common\org\apache\harmony\auth\login\DefaultConfigParserTest.java:79\182
.\awt\src\test\api\java\common\java\awt\color\ICC_ProfileRTest.java:37
.\awt\src\test\api\java\common\java\awt\ComponentTest.java:751
.\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:652\658\664\670\676\685\698\704\711
.\awt\src\test\api\java\windows\org\apache\harmony\awt\tests\java\awt\WinFontTest.java:427
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextServicesSupportTest.java:739
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextSupportTest.java:234\251\1388
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\IntrospectorTest.java:1266\1304
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\StatementTest.java:93
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLDecoderTest.java:92
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLEncoderTest.java:334
.\concurrent\standard\src\test\java\AbstractExecutorServiceTest.java:215\235\253\306\311\340\341\353\389\403\418\436\454\473\488\521\541\562\578\595\610\628\646\665\680\697\730\750\771
.\concurrent\standard\src\test\java\AbstractQueuedSynchronizerTest.java:76\92\162\365\514\531\631\645\660\677\692\709\724\741\756\976\1007\1039\1236\1260
.\concurrent\standard\src\test\java\AbstractQueueTest.java:62\74\94\115\128\140\154\169\183
Re: [classlib][test] fail statements omitted in many exception catching test cases
Maybe we have to review the tests one by one. ;-)
On 9/27/06, Robert Hu [EMAIL PROTECTED] wrote:
Hi All,
In our unit test of classlib, there are huge number of test cases about
exception catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the
exception-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of our
Harmony classlib. So I set some rules to find out all lines of code related with it. If a
line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules in
r450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of above
style.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:652\658\664\670\676\685\698\704\711(line
number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileInputStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOutputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind without any
victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those bugs? Pls.
share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarFileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\DeflaterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\DeflaterTest.java:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\1099\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipFileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipInputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallbackTest.java:60\147
.\auth\src\test\java\common\javax\security\auth\callback\LanguageCallbackTest.java:60
.\auth\src\test\java\common\javax\security\auth\kerberos\ServicePermissionTest.java:302
.\auth\src\test\java\common\javax\security\auth\PolicyTest.java:132
.\auth\src\test\java\common\javax\security\auth\PrivateCredentialPermissionTest.java:267\386
.\auth\src\test\java\common\javax\security\auth\SubjectTest.java:239\248\257\269\992\1109\1301\1314\1423\1851\2049\2061\2089\2101\2127\2139
.\auth\src\test\java\common\javax\security\auth\x500\X500PrincipalTest.java:2404
.\auth\src\test\java\common\javax\security\auth\x500\X500PrivateCredentialTest.java:169\179\189
.\auth\src\test\java\common\org\apache\harmony\auth\internal\SecurityTest.java:807\855\950\971\987\995\1003\1011\1065\1089\1090\1101\1116\1125\1133\1141\1142\1151\1152\1161\1162
.\auth\src\test\java\common\org\apache\harmony\auth\login\DefaultConfigParserTest.java:79\182
.\awt\src\test\api\java\common\java\awt\color\ICC_ProfileRTest.java:37
.\awt\src\test\api\java\common\java\awt\ComponentTest.java:751
.\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:652\658\664\670\676\685\698\704\711
.\awt\src\test\api\java\windows\org\apache\harmony\awt\tests\java\awt\WinFontTest.java:427
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextServicesSupportTest.java:739
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextSupportTest.java:234\251\1388
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\IntrospectorTest.java:1266\1304
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\StatementTest.java:93
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLDecoderTest.java:92
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLEncoderTest.java:334
.\concurrent\standard\src\test\java\AbstractExecutorServiceTest.java:215\235\253\306\311\340\341\353\389\403\418\436\454\473\488\521\541\562\578\595\610\628\646\665\680\697\730\750\771
.\concurrent\standard\src\test\java\AbstractQueuedSynchronizerTest.java:76\92\162\365\514\531\631\645\660\677\692\709\724\741\756\976\1007\1039\1236\1260
.\concurrent\standard\src\test\java\AbstractQueueTest.java:62\74\94\115\128\140\154\169\183
Re: [classlib][test] fail statements omitted in many exception catching test cases
This perl script does a marginally better job by being slightly stricter
on matching context around 'catch'/'fail', by handling comments slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java +208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~ /\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
On 27 September 2006 at 15:02, Robert Hu [EMAIL PROTECTED] wrote:
--090601000506020908060004
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=GB2312
Hi All,
In our unit test of classlib, there are huge number of test cases about excep
tion catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the exception
-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of our Har
mony classlib. So I set some rules to find out all lines of code related with
it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules in r
450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of above s
tyle.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:
652\658\664\670\676\685\698\704\711(line number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.
java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileIn
putStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOu
tputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind without
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those bugs? Pl
s. share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
--090601000506020908060004
Content-Type: text/plain;
name=result.txt
Content-Disposition: inline;
filename=result.txt
Content-Transfer-Encoding: quoted-printable
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarF=
ileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterTest.java:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\10=
99\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipF=
ileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipI=
nputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallba=
ckTest.java:60\147
.\auth\src\test\java\common\javax\security\auth\callback\LanguageCallbackTe=
st.java:60
.\auth\src\test\java\common\javax\security\auth\kerberos\ServicePermissionT=
est.java:302
.\auth\src\test\java\common\javax\security\auth\PolicyTest.java:132
.\auth\src\test\java\common\javax\security\auth\PrivateCredentialPermission=
Test.java:267\386
.\auth\src\test\java\common\javax\security\auth\SubjectTest.java:239\248\25=
7\269\992\1109\1301\1314\1423\1851\2049\2061\2089\2101\2127\2139
.\auth\src\test\java\common\javax\security\auth\x500\X500PrincipalTest.java=
:2404
.\auth\src\test\java\common\javax\security\auth\x500\X500PrivateCredentialT=
est.java:169\179\189
.\auth\src\test\java\common\org\apache\harmony\auth\internal\SecurityTest.j=
ava:807\855\950\971\987\995\1003\1011\1065\1089\1090\1101\1116\1125\1133\11=
41\1142\1151\1152\1161\1162
.\auth\src\test\java\common\org\apache\harmony\auth\login\DefaultConfigPars=
Re: [classlib][test] fail statements omitted in many exception catching test cases
Hi Robert,
Nice work! I've checked the entries for classes from the beans module.
It seems you found at least three bugs in tests for beans:
Bugs:
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextServicesSupportTest.java:739
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextSupportTest.java:1388
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLEncoderTest.java:449
Valid:
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextSupportTest.java:234\251
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\IntrospectorTest.java:1266\1304
needs more investigation
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\StatementTest.java:93
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLDecoderTest.java:92
.\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLEncoderTest.java:334
Thank you for this. I will file corresponding JIRA.
Regards,
2006/9/27, Robert Hu [EMAIL PROTECTED]:
Hi All,
In our unit test of classlib, there are huge number of test cases about
exception catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the
exception-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of our
Harmony classlib. So I set some rules to find out all lines of code related with it. If a
line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules in
r450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of above
style.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:652\658\664\670\676\685\698\704\711(line
number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileInputStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOutputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind without any
victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those bugs? Pls.
share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarFileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\DeflaterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\DeflaterTest.java:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\1099\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipFileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipInputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallbackTest.java:60\147
.\auth\src\test\java\common\javax\security\auth\callback\LanguageCallbackTest.java:60
.\auth\src\test\java\common\javax\security\auth\kerberos\ServicePermissionTest.java:302
.\auth\src\test\java\common\javax\security\auth\PolicyTest.java:132
.\auth\src\test\java\common\javax\security\auth\PrivateCredentialPermissionTest.java:267\386
.\auth\src\test\java\common\javax\security\auth\SubjectTest.java:239\248\257\269\992\1109\1301\1314\1423\1851\2049\2061\2089\2101\2127\2139
.\auth\src\test\java\common\javax\security\auth\x500\X500PrincipalTest.java:2404
.\auth\src\test\java\common\javax\security\auth\x500\X500PrivateCredentialTest.java:169\179\189
.\auth\src\test\java\common\org\apache\harmony\auth\internal\SecurityTest.java:807\855\950\971\987\995\1003\1011\1065\1089\1090\1101\1116\1125\1133\1141\1142\1151\1152\1161\1162
.\auth\src\test\java\common\org\apache\harmony\auth\login\DefaultConfigParserTest.java:79\182
.\awt\src\test\api\java\common\java\awt\color\ICC_ProfileRTest.java:37
.\awt\src\test\api\java\common\java\awt\ComponentTest.java:751
.\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:652\658\664\670\676\685\698\704\711
.\awt\src\test\api\java\windows\org\apache\harmony\awt\tests\java\awt\WinFontTest.java:427
Re: [classlib][test] fail statements omitted in many exception catching test cases
Great Mark!
I've merged your code into my script. The script can be downloaded at [1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/attachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly stricter
on matching context around 'catch'/'fail', by handling comments slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java +208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
On 27 September 2006 at 15:02, Robert Hu [EMAIL PROTECTED] wrote:
--090601000506020908060004
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=GB2312
Hi All,
In our unit test of classlib, there are huge number of test cases about
excep
tion catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the
exception
-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of
our Har
mony classlib. So I set some rules to find out all lines of code related
with
it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules
in r
450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of
above s
tyle.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:
652\658\664\670\676\685\698\704\711(line number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.
java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileIn
putStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOu
tputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind
without
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those
bugs? Pl
s. share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
--090601000506020908060004
Content-Type: text/plain;
name=result.txt
Content-Disposition: inline;
filename=result.txt
Content-Transfer-Encoding: quoted-printable
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarF=
ileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterTest.java
:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\10=
99\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipF=
ileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipI=
nputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallba=
ckTest.java:60\147
.\auth\src\test\java\common\javax\security\auth\callback\LanguageCallbackTe=
st.java:60
.\auth\src\test\java\common\javax\security\auth\kerberos\ServicePermissionT=
est.java:302
.\auth\src\test\java\common\javax\security\auth\PolicyTest.java:132
.\auth\src\test\java\common\javax\security\auth\PrivateCredentialPermission=
Test.java:267\386
Re: [classlib][test] fail statements omitted in many exception catching test cases
Thanks for checking the result and working on it. I put the result on Harmony wiki[1], anyone is welcome to check the result to find some bugs. [1]:http://wiki.apache.org/harmony/failstatementsomitted On 9/27/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi Robert, Nice work! I've checked the entries for classes from the beans module. It seems you found at least three bugs in tests for beans: Bugs: .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextServicesSupportTest.java:739 .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextSupportTest.java:1388 .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLEncoderTest.java:449 Valid: .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\beancontext\BeanContextSupportTest.java:234\251 .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\IntrospectorTest.java:1266\1304 needs more investigation .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\StatementTest.java:93 .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLDecoderTest.java:92 .\beans\src\test\java\org\apache\harmony\beans\tests\java\beans\XMLEncoderTest.java:334 Thank you for this. I will file corresponding JIRA. Regards, snip -- Robert Hu China Software Development Lab, IBM
Re: [classlib][test] fail statements omitted in many exception catching test cases
Great job, Robert. ;-)
I will have a look at sql.
On 9/28/06, Rui Hu [EMAIL PROTECTED] wrote:
Great Mark!
I've merged your code into my script. The script can be downloaded at [1].
Usage:
perl failFinder.pl root_of_module
e.g. Search out all related lines in luni module and redirect it to
result.txt
perl failFinder.pl trunk/modules/luni result.txt
e.g. Search out all related lines in all modules and redirect it to
result.txt
perl failFinder.pl trunk/modules/ result.txt
Anyone can find out the related lines of any modules.
[1]:
http://wiki.apache.org/harmony-data/attachments/failstatementsomitted/attachments/failFinder.pl
On 9/27/06, Mark Hindess [EMAIL PROTECTED] wrote:
This perl script does a marginally better job by being slightly stricter
on matching context around 'catch'/'fail', by handling comments slightly
better and by handling 'catch (...) { }' appearing on a single line.
It also finds a few more hits such as:
sql/src/test/java/org/apache/harmony/sql/tests/java/sql/TimeTest.java +208
which is a false positive but which uses assertTrue(false); which
should be fixed anyway.
-Mark.
#!/usr/bin/perl -w
use strict;
my $previous_line = ;
my $possible_line_number;
while () {
next if (/^\s*$/); # skip blank lines
if ($possible_line_number) {
if (m!^\s*(//|/\*|})!) {
print $ARGV, ' +', $possible_line_number, \n;
}
undef $possible_line_number;
}
if (!m!^\s*(/?\*|//)! /\bcatch\s*\(/ $previous_line !~
/\bfail\s*\(/) {
$possible_line_number = $.;
if (/catch\s*\([^\)]+\)\s*{\s*}/) {
print $ARGV, ' +', $possible_line_number, \n;
undef $possible_line_number;
}
}
$previous_line = $_;
}
On 27 September 2006 at 15:02, Robert Hu [EMAIL PROTECTED] wrote:
--090601000506020908060004
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=GB2312
Hi All,
In our unit test of classlib, there are huge number of test cases about
excep
tion catching. The typical style of those cases is like that:
try {
someStatementShouldThrowAnException;
* fail(Expected an exception);*
} catch (SomeException e){
// Expected
}
If we omit the fail statement, the test case is wrong because the
exception
-throwing checking is disabled.
I've found that the fail statement is omitted in many test cases of
our Har
mony classlib. So I set some rules to find out all lines of code related
with
it. If a line of code comform all the 5 rules, it may be a bug:
1.in a *Test.java file
2.does not start with //
3.contains catch
4.its previous line does not contains fail
5.its next line contains // or }
Then I found out 1711 lines of code in 309 files comform all the 5 rules
in r
450321. (Attachment file is the result.)
Of course not all of them are bug, because some test cases are not of
above s
tyle.
And I also find out some real bugs, we can fix them easilly:
trunk\modules\awt\src\test\api\java\common\java\awt\font\TextLayoutTest.java:
652\658\664\670\676\685\698\704\711(line number)
trunk\modules\luni\src\test\java\org\apache\harmony\tests\java\lang\EnumTest.
java:57
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileIn
putStreamTest.java:36
trunk\modules\luni\src\test\java\org\apache\harmony\luni\tests\java\io\FileOu
tputStreamTest.java:35
more..
*I must say frankly that it's hard to find out all bugs of this kind
without
any victims automatically, we must find out real bugs ourselves.*
Hope the result in attachment file can help us to find out more bugs.
Anybody has better search rules or better solution to find out those
bugs? Pl
s. share with us, thanks a lot.
--
Robert Hu
China Software Development Lab, IBM
--090601000506020908060004
Content-Type: text/plain;
name=result.txt
Content-Disposition: inline;
filename=result.txt
Content-Transfer-Encoding: quoted-printable
current position is trunk\modules
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\jar\JarF=
ileTest.java:66\79\190
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterOutputStreamTest.java:220\230
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\Defl=
aterTest.java
:188\619\724\785\792\859\1006\1013\1070\1077\1091\1092\1098\10=
99\1105\1106\1113\1114\1120\1121\1127\1128\1134\1135\1143\1179
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipF=
ileTest.java:67\291
.\archive\src\test\java\org\apache\harmony\archive\tests\java\util\zip\ZipI=
nputStreamTest.java:200
.\auth\src\test\java\common\javax\security\auth\callback\ConfirmationCallba=
ckTest.java:60\147
.\auth\src\test\java\common\javax\security\auth\callback\LanguageCallbackTe=
st.java:60
