Re: [winetest] Show missing tests in single/group results
Jakob, Paul, we should have a regular site for uploading screenshots like these... so that google can find them too... On 8/6/07, Jakob Eriksson [EMAIL PROTECTED] wrote: Paul Vriens wrote: Comments/remarks etc.. are welcome. It is excellent that you are doing this, the reports are clearer for it. I will try to submit test results regularly. regards, Jakob Eriksson -- Robert Q Kim, Wireless Internet Provider http://evdo-coverage.com/satellite-wireless-internet.html http://iptv-coverage.com http://wimax-coverage.com http://znitch.com unpaid overtime 2611 S. Pacific Coast Highway 101 Suite 203 Cardiff by the Sea, CA 92007 206 984 0880
Re: [winetest] Show missing tests in single/group results
Paul Vriens [EMAIL PROTECTED] writes: 'http://www.xs4all.nl/~pvriens/200708031000.new.single/' shows the new results page for single tests. Instead of a single '.' there is a failure shown (red box) and a skip border (as these tests have not run). There is still a '.' which could maybe better be a '?'. There is also a 'onMouseOver' hint when you go over the '.'. Hi Paul, Sorry for being so late. As a quick note, onMouseOver should contain JavaScript code, not a freeform string. Isn't the title attribute enough? -- Regards, Feri.
Re: [winetest] Show missing tests in single/group results
Jakob Eriksson wrote: Paul Vriens wrote: Comments/remarks etc.. are welcome. It is excellent that you are doing this, the reports are clearer for it. I will try to submit test results regularly. regards, Jakob Eriksson Hi Jakob, I think you ran a few win95 tests yesterday? What was wrong with for example advapi32crypt32/d3d8/d3d9 (none of them ran so it appeared)? Can you produce a list of things winetest complained about while extracting? If it just says Can't parse subtests output of *.dll, could you go to the wct directory (in %TEMP%) and execute the single *_test.exe? That should give a bit more information about why it can't get the subtests list. Cheers. Paul.
Re: [winetest] Show missing tests in single/group results
Paul Vriens wrote: Comments/remarks etc.. are welcome. It is excellent that you are doing this, the reports are clearer for it. I will try to submit test results regularly. regards, Jakob Eriksson
[winetest] Show missing tests in single/group results
Hi, (I've just remembered that there is a limit on the email size, so I'm doing it a bit differently now. The other email, including the screenshots, will probably come through as well) Before sending this to wine-patches I would like people to have a look first. This patch makes sure that missing tests are shown in the test results, both when there is only one system in a group (f.e. Win98) or multiple systems in a group. On 'http://www.xs4all.nl/~pvriens/200708031000.old.single/' you can see the current state of the test results. When a test is missing a '.' is shown (this is of course different from a missing test where we don't have results at all but that's shown as a '.' as well). On 'http://www.xs4all.nl/~pvriens/200708031000.old.group/' you can see the current state when a group of systems is involved. When tests are missing for a group, a '.' is shown with a link to the group overview. There is however no missing test output in the group overview (for example gdiplus and pdh) 'http://www.xs4all.nl/~pvriens/200708031000.new.single/' shows the new results page for single tests. Instead of a single '.' there is a failure shown (red box) and a skip border (as these tests have not run). There is still a '.' which could maybe better be a '?'. There is also a 'onMouseOver' hint when you go over the '.'. 'http://www.xs4all.nl/~pvriens/200708031000.new.group/' shows the new results when a group is involved. The main summary is still the same as in the old one, but the group results now also show the missing tests (see gdiplus and pdh again). Again there is a 'onMouseOver' hint in the group results. Remember that this is still without winetest producing output about missing dll's. Next patches will: - add a missing dll as one of the reasons for missing tests - add some better clue's on the main summary for multiple systems in a group (not a single '.' but maybe the same 'failed and skipped'). As said, one of the thoughts I had was to replace the '.' with a '?' in case of missing tests for an unknown reason but I'm not sure about that yet. Comments/remarks etc.. are welcome. I'd also like to hear about other idea's for this page to make it useful for everybody. Apart from the fact that we probably should have more exposure to the fact that we have such a page/tool. We saw an immediate rise (slowed down again) in test results as soon as winetest was mentioned on 'http://tweakers.net/meuktracker/15817/Wine-0.9.42.html' (in Dutch). There it was also mentioned that some machines crashed because of the tests etc.. (maybe a wine-winetest component on bugzilla could help with that which we could reference on the main test.winehq.org/data/*/index.html pages). Cheers, Paul.
Re: [winetest] Show missing tests in single/group results
Paul Vriens wrote:
Hi,
(I've just remembered that there is a limit on the email size, so I'm
doing it a
bit differently now. The other email, including the screenshots, will
probably come through as well)
Before sending this to wine-patches I would like people to have a look
first.
This patch makes sure that missing tests are shown in the test results,
both
when there is only one system in a group (f.e. Win98) or multiple
systems in a
group.
On 'http://www.xs4all.nl/~pvriens/200708031000.old.single/' you can see the
current state of the test results. When a test is missing a '.' is shown
(this is of course different from a missing test where we don't have
results at all but that's shown as a '.' as well).
On 'http://www.xs4all.nl/~pvriens/200708031000.old.group/' you can see the
current state when a group of systems is involved. When tests are
missing for a
group, a '.' is shown with a link to the group overview. There is
however no
missing test output in the group overview (for example gdiplus and pdh)
'http://www.xs4all.nl/~pvriens/200708031000.new.single/' shows the new
results
page for single tests. Instead of a single '.' there is a failure shown
(red
box) and a skip border (as these tests have not run). There is still a
'.' which
could maybe better be a '?'. There is also a 'onMouseOver' hint when you
go over
the '.'.
'http://www.xs4all.nl/~pvriens/200708031000.new.group/' shows the new
results
when a group is involved. The main summary is still the same as in the
old one,
but the group results now also show the missing tests (see gdiplus and pdh
again). Again there is a 'onMouseOver' hint in the group results.
Remember that this is still without winetest producing output about
missing dll's.
Next patches will:
- add a missing dll as one of the reasons for missing tests
- add some better clue's on the main summary for multiple systems in a
group
(not a single '.' but maybe the same 'failed and skipped').
As said, one of the thoughts I had was to replace the '.' with a '?' in
case of
missing tests for an unknown reason but I'm not sure about that yet.
Comments/remarks etc.. are welcome.
I'd also like to hear about other idea's for this page to make it useful
for
everybody. Apart from the fact that we probably should have more
exposure to the
fact that we have such a page/tool.
We saw an immediate rise (slowed down again) in test results as soon as
winetest
was mentioned on
'http://tweakers.net/meuktracker/15817/Wine-0.9.42.html' (in
Dutch). There it was also mentioned that some machines crashed because
of the
tests etc.. (maybe a wine-winetest component on bugzilla could help with
that
which we could reference on the main test.winehq.org/data/*/index.html
pages).
Cheers,
Paul.
Forgot to mention that only the index.html file is on the mentioned pages. No
result files (to cut down on size for me to push).
I also forgot to attach the patch. So here it is.
Cheers,
Paul
--- /wine/tools/winetest/gather 2007-08-03 17:20:09.0 +0200
+++ gather 2007-08-05 14:02:42.0 +0200
@@ -121,6 +121,22 @@
push @{$group-{tests}}, $testref;
}
+# Find missing tests. After this exercise all testresults are available
+# for all systems.
+foreach my $group (@groups) {
+next unless exists $group-{tests};
+foreach my $test (@{$group-{tests}}) {
+foreach my $testname (sort keys %alltests) {
+if (!exists $test-{results}-{$testname}) {
+# Make sure missing tests are shown in the group results
+$group-{digests}-{$testname} = differ;
+# Mark this test as missing for an unknown reason
+$test-{results}-{$testname} = [test missing, -, -,
-];
+}
+}
+}
+}
+
# Write out the tables
my ($header); # same in thead and tfoot
@@ -201,28 +217,30 @@
# Output a single cell of a test
sub singletest {
my ($test, $testname, $groupname) = @_;
-
-if (!exists $test-{results}-{$testname}) {
-print OUT td class=\note\./td\n;
+my $file = $test-{dir}/$testname.txt;
+my ($count, $todo, $error, $skipped) = @{$test-{results}-{$testname}};
+if ($count eq failed) {
+my $msg = $todo eq -?crash:
+ $todo == 258?timeout:failed;
+my $fail = -r $datadir/$build/$file?a
href=\$file\$msg/a:$msg;
+print OUT td class=\note\$fail/td\n;
+} elsif ($count eq test missing) {
+print OUT EOF;
+ td class=skip_faila
+title=Test did not run for an unknown reason
+onMouseOver=Test did not run for an unknown reason;
+./a/td
+EOF
} else {
-my $file = $test-{dir}/$testname.txt;
-my ($count, $todo, $error, $skipped) =
@{$test-{results}-{$testname}};
-if ($count eq failed) {
-my $msg = $todo eq -?crash:
- $todo == 258?timeout:failed;
-my $fail = -r
