subject:"\[webkit\-dev\] LayoutTests results fallback graph"

On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


 In addition, we do actually have a 'chromium-mac'; we don't have a
 'chromium-mac-snowleopard'. I think I mixed that in my mind while
 typing this with the apple mac ports, where there are mac-leopard,
 mac-sl, and mac ports (the latter representing lion/future).

 Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard 
 dir.

 -- Dirk


 Okay, I pulled together a slightly more comprehensive report ... in
 short, we pull things from everywhere. Maybe this is useful to someone
 if they want to try and treeify the fallbacks :)

 The format should be fairly self-explanatory. It is a rollup report
 for all of the baselines, grouped on the combination of ports,
 platforms, and type of baselines. The first column is the
 port/platform configuration. The second is the location of the test
 (generic means not in a platform/* directory). The third is the type
 of baseline for the test, the fourth is the location of the baseline
 used, and the fifth is the total # of such baselines in that location.

To confirm my understanding:

This row means that the Chromium Mac port running on Snow Leopard gets
at least 5567 -expected.png files from the LayoutTests/platform/mac
directory?

chromium-mac-snowleopard,generic,png,mac,5567

This is great data!  If you're interested in crunching numbers, it
might be interested to hack up the deduplicate-tests script to figure
out how much of the possible sharing we're realizing with our current
fallback graph.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-13 Thread Dirk Pranke

On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


 In addition, we do actually have a 'chromium-mac'; we don't have a
 'chromium-mac-snowleopard'. I think I mixed that in my mind while
 typing this with the apple mac ports, where there are mac-leopard,
 mac-sl, and mac ports (the latter representing lion/future).

 Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard 
 dir.

 -- Dirk


 Okay, I pulled together a slightly more comprehensive report ... in
 short, we pull things from everywhere. Maybe this is useful to someone
 if they want to try and treeify the fallbacks :)

 The format should be fairly self-explanatory. It is a rollup report
 for all of the baselines, grouped on the combination of ports,
 platforms, and type of baselines. The first column is the
 port/platform configuration. The second is the location of the test
 (generic means not in a platform/* directory). The third is the type
 of baseline for the test, the fourth is the location of the baseline
 used, and the fifth is the total # of such baselines in that location.

 To confirm my understanding:

 This row means that the Chromium Mac port running on Snow Leopard gets
 at least 5567 -expected.png files from the LayoutTests/platform/mac
 directory?

 chromium-mac-snowleopard,generic,png,mac,5567


That is correct.

 This is great data!  If you're interested in crunching numbers, it
 might be interested to hack up the deduplicate-tests script to figure
 out how much of the possible sharing we're realizing with our current
 fallback graph.

 Adam


I'm not sure I follow what you have in mind here ...

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Wed, Jul 13, 2011 at 1:36 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org 
 wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


 In addition, we do actually have a 'chromium-mac'; we don't have a
 'chromium-mac-snowleopard'. I think I mixed that in my mind while
 typing this with the apple mac ports, where there are mac-leopard,
 mac-sl, and mac ports (the latter representing lion/future).

 Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard 
 dir.

 -- Dirk


 Okay, I pulled together a slightly more comprehensive report ... in
 short, we pull things from everywhere. Maybe this is useful to someone
 if they want to try and treeify the fallbacks :)

 The format should be fairly self-explanatory. It is a rollup report
 for all of the baselines, grouped on the combination of ports,
 platforms, and type of baselines. The first column is the
 port/platform configuration. The second is the location of the test
 (generic means not in a platform/* directory). The third is the type
 of baseline for the test, the fourth is the location of the baseline
 used, and the fifth is the total # of such baselines in that location.

 To confirm my understanding:

 This row means that the Chromium Mac port running on Snow Leopard gets
 at least 5567 -expected.png files from the LayoutTests/platform/mac
 directory?

 chromium-mac-snowleopard,generic,png,mac,5567


 That is correct.

 This is great data!  If you're interested in crunching numbers, it
 might be interested to hack up the deduplicate-tests script to figure
 out how much of the possible sharing we're realizing with our current
 fallback graph.

 I'm not sure I follow what you have in mind here ...

No worries.  I'll figure it out myself.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Wed, Jul 13, 2011 at 1:56 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:36 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org 
 wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


 In addition, we do actually have a 'chromium-mac'; we don't have a
 'chromium-mac-snowleopard'. I think I mixed that in my mind while
 typing this with the apple mac ports, where there are mac-leopard,
 mac-sl, and mac ports (the latter representing lion/future).

 Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard 
 dir.

 -- Dirk


 Okay, I pulled together a slightly more comprehensive report ... in
 short, we pull things from everywhere. Maybe this is useful to someone
 if they want to try and treeify the fallbacks :)

 The format should be fairly self-explanatory. It is a rollup report
 for all of the baselines, grouped on the combination of ports,
 platforms, and type of baselines. The first column is the
 port/platform configuration. The second is the location of the test
 (generic means not in a platform/* directory). The third is the type
 of baseline for the test, the fourth is the location of the baseline
 used, and the fifth is the total # of such baselines in that location.

 To confirm my understanding:

 This row means that the Chromium Mac port running on Snow Leopard gets
 at least 5567 -expected.png files from the LayoutTests/platform/mac
 directory?

 chromium-mac-snowleopard,generic,png,mac,5567


 That is correct.

 This is great data!  If you're interested in crunching numbers, it
 might be interested to hack up the deduplicate-tests script to figure
 out how much of the possible sharing we're realizing with our current
 fallback graph.

 I'm not sure I follow what you have in mind here ...

 No worries.  I'll figure it out myself.

There are approximately 1500 redundant test results that we aren't
able to collapse using our current fallback strategy.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-13 Thread Eric Seidel

I believe what Adam means by this (it wasn't immediately clear to me),
is that we have 1500 redundant result files with duplicate git hashes
to some other file.  This could be calculated by the
deduplicate_results.py script by removing any of the current fallback
logic and just look at raw duplicates.

Please correct me if I've misunderstood.

-eric

On Wed, Jul 13, 2011 at 2:06 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:56 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:36 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke dpra...@chromium.org 
 wrote:
 On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org 
 wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


 In addition, we do actually have a 'chromium-mac'; we don't have a
 'chromium-mac-snowleopard'. I think I mixed that in my mind while
 typing this with the apple mac ports, where there are mac-leopard,
 mac-sl, and mac ports (the latter representing lion/future).

 Once Lion ships, chromium will undoubtedly add a 
 chromium-mac-snowleopard dir.

 -- Dirk


 Okay, I pulled together a slightly more comprehensive report ... in
 short, we pull things from everywhere. Maybe this is useful to someone
 if they want to try and treeify the fallbacks :)

 The format should be fairly self-explanatory. It is a rollup report
 for all of the baselines, grouped on the combination of ports,
 platforms, and type of baselines. The first column is the
 port/platform configuration. The second is the location of the test
 (generic means not in a platform/* directory). The third is the type
 of baseline for the test, the fourth is the location of the baseline
 used, and the fifth is the total # of such baselines in that location.

 To confirm my understanding:

 This row means that the Chromium Mac port running on Snow Leopard gets
 at least 5567 -expected.png files from the LayoutTests/platform/mac
 directory?

 chromium-mac-snowleopard,generic,png,mac,5567


 That is correct.

 This is great data!  If you're interested in crunching numbers, it
 might be interested to hack up the deduplicate-tests script to figure
 out how much of the possible sharing we're realizing with our current
 fallback graph.

 I'm not sure I follow what you have in mind here ...

 No worries.  I'll figure it out myself.

 There are approximately 1500 redundant test results that we aren't
 able to collapse using our current fallback strategy.

 Adam
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

That's correct.  Theoretically, there might exist a fallback strategy
that could remove this redundancy, but it might be arbitrarily
complicated.  Although one could argue that our current strategy is
approaching arbitrary complexity.  :)

Adam


On Wed, Jul 13, 2011 at 2:21 PM, Eric Seidel e...@webkit.org wrote:
 I believe what Adam means by this (it wasn't immediately clear to me),
 is that we have 1500 redundant result files with duplicate git hashes
 to some other file.  This could be calculated by the
 deduplicate_results.py script by removing any of the current fallback
 logic and just look at raw duplicates.

 Please correct me if I've misunderstood.

 -eric

 On Wed, Jul 13, 2011 at 2:06 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:56 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:36 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Wed, Jul 13, 2011 at 1:28 PM, Adam Barth aba...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 1:21 PM, Dirk Pranke dpra...@chromium.org wrote:
 On Tue, Jul 12, 2011 at 10:01 PM, Dirk Pranke dpra...@chromium.org 
 wrote:
 On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org 
 wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, 
 then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


 In addition, we do actually have a 'chromium-mac'; we don't have a
 'chromium-mac-snowleopard'. I think I mixed that in my mind while
 typing this with the apple mac ports, where there are mac-leopard,
 mac-sl, and mac ports (the latter representing lion/future).

 Once Lion ships, chromium will undoubtedly add a 
 chromium-mac-snowleopard dir.

 -- Dirk


 Okay, I pulled together a slightly more comprehensive report ... in
 short, we pull things from everywhere. Maybe this is useful to someone
 if they want to try and treeify the fallbacks :)

 The format should be fairly self-explanatory. It is a rollup report
 for all of the baselines, grouped on the combination of ports,
 platforms, and type of baselines. The first column is the
 port/platform configuration. The second is the location of the test
 (generic means not in a platform/* directory). The third is the type
 of baseline for the test, the fourth is the location of the baseline
 used, and the fifth is the total # of such baselines in that location.

 To confirm my understanding:

 This row means that the Chromium Mac port running on Snow Leopard gets
 at least 5567 -expected.png files from the LayoutTests/platform/mac
 directory?

 chromium-mac-snowleopard,generic,png,mac,5567


 That is correct.

 This is great data!  If you're interested in crunching numbers, it
 might be interested to hack up the deduplicate-tests script to figure
 out how much of the possible sharing we're realizing with our current
 fallback graph.

 I'm not sure I follow what you have in mind here ...

 No worries.  I'll figure it out myself.

 There are approximately 1500 redundant test results that we aren't
 able to collapse using our current fallback strategy.

 Adam
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-13 Thread Ryosuke Niwa

On Wed, Jul 13, 2011 at 2:06 PM, Adam Barth aba...@webkit.org wrote:

 There are approximately 1500 redundant test results that we aren't
 able to collapse using our current fallback strategy.


Can we make fallback explicitly specified by some file?  e.g. we can have a
fallbackOverride file in each platform directory that has entries like:

editing/selection/my-selection-test.html mac
fast/dom/ mac-leopard

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Wed, Jul 13, 2011 at 2:35 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Wed, Jul 13, 2011 at 2:06 PM, Adam Barth aba...@webkit.org wrote:
 There are approximately 1500 redundant test results that we aren't
 able to collapse using our current fallback strategy.

 Can we make fallback explicitly specified by some file?  e.g. we can have a
 fallbackOverride file in each platform directory that has entries like:
 editing/selection/my-selection-test.html mac
 fast/dom/ mac-leopard

That sounds like a maintenance nightmare.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-12 Thread Dirk Pranke

On Mon, Jul 11, 2011 at 11:52 AM, Dirk Pranke dpra...@chromium.org wrote:
On Mon, Jul 11, 2011 at 10:46 AM, Adam Barth aba...@webkit.org wrote:
On Mon, Jul 11, 2011 at 12:30 AM, Maciej Stachowiak m...@apple.com wrote:
On Jul 10, 2011, at 12:11 PM, Adam Barth wrote:
On Sun, Jul 10, 2011 at 12:01 PM, James Robinson jam...@google.com wrote:
On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:
Hi webkit-dev,

In trying to understand how our LayoutTest results system works, I've
created a digram of the fallback graph among the various
platform-specific directories:
https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US

Unfortunately, the fallback graph is not a tree, as one might imagine
initially. I'd like to propose two small changes, which will
hopefully make the system more sensible globally. I'm happy to do all
the work required to make these changes:

1) The win port should fall back either to all (the platform
independent results) or to mac, but not to mac-snowleopard, as it
does currently. (I slightly prefer all, but mac would also be
fine with me.)

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

A) The resulting fallback graph will be a tree, making the fallback
graph easier to understand for both humans and automated tools.
B) The Chromium port will behave more like the other ports (e.g., GTK
and Qt), rather than being a parasite on Apple-maintained ports.

These changes might increase the number of image baselines we store in
the tree for chromium-mac-derived ports (because there will be fewer
redundant fallback paths), but I expect that cost to be relatively
small because essentially every port has different image baselines
anyway

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

That seems possible:

abarth@quadzen:~/git/webkit/LayoutTests/platform/chromium-mac$ find .
-name *.png | wc -l
900
abarth@quadzen:~/git/webkit/LayoutTests/platform/mac$ find . -name
*.png | wc -l
6688
abarth@quadzen:~/git/webkit/LayoutTests/platform/chromium-win$ find .
-name *.png | wc -l
5988
abarth@quadzen:~/git/webkit/LayoutTests/platform/chromium-linux$ find
. -name *.png | wc -l
5731

Adding thousands more PNGs would be unfortunate, especially if a
significant number of them are going to be rolled over frequently.

I wouldn't expect a significant number of them to change frequently.

What are the concrete benefits of the fallback graph being a tree?

There are two main benefits:

1) A tree is much easier to understand than a rats nest of interwoven
fallback paths. Today, the graph is close to a tree so it's possible
for experts to understand how things work, but if we continue to add
non-transitive paths, the situation will quickly spiral out of
control. For folks who aren't experts about how the system works
today, I doubt they could correctly predict the consequence of certain
actions. One of my short-term goals is to make managing the expecte
results easier, which is why I'm interested in having the system be
understandable to non-experts.

2) Without a tree structure, it's difficult to compute the optimum
assignments of expected results to directories. With a tree
structure, it's very easy. If all the children of a node have the
same result, you can delete the result at the children and promote it
to the parent. That reasoning is false for our current fallback
graph. The correct rule for when you can promote a result instead
requires reasoning over all paths (of which, of course, there are
exponentially many).

I'm not sure I understand in what sense you're using the word
optimum here. Normally I would define it as fewest overall
baselines in the tree. Using my definition, it seems possible that
having multiple parents could result in fewer baselines. Example: if
chromium-mac is allowed to fallback on chromium, then mac, then all
(rather than just chromium then all or mac then all) you may need
fewer total baselines

Of course, I'm not sure that fewest overall baselines is in fact the
goal we should be shooting for. I definitely agree that a tree is
easier to understand. This of course is the same debate as single
inheritance vs. multiple ... multiple may reduce the total lines of
code, but be harder to understand

NRWT has nearly all of the code needed to make it easy to evaluate
what the total impact of changing the paths would be. I think I had
some scripts to do this long ago but I've lost them. I should be able
to reconstruct them this afternoon.

Okay, following up ... I think I'm actually responsible for making the
fallback path not be a tree, from when I introduced the 'chromium'
directory, so perhaps

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-12 Thread Dirk Pranke

On Tue, Jul 12, 2011 at 8:05 PM, Dirk Pranke dpra...@chromium.org wrote:
On Mon, Jul 11, 2011 at 11:52 AM, Dirk Pranke dpra...@chromium.org wrote:
On Mon, Jul 11, 2011 at 10:46 AM, Adam Barth aba...@webkit.org wrote:
On Mon, Jul 11, 2011 at 12:30 AM, Maciej Stachowiak m...@apple.com wrote:
On Jul 10, 2011, at 12:11 PM, Adam Barth wrote:
On Sun, Jul 10, 2011 at 12:01 PM, James Robinson jam...@google.com
wrote:
On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:
Hi webkit-dev,

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

That seems possible:

Adding thousands more PNGs would be unfortunate, especially if a
significant number of them are going to be rolled over frequently.

I wouldn't expect a significant number of them to change frequently.

What are the concrete benefits of the fallback graph being a tree?

There are two main benefits:

Okay, following up ... I think I'm actually responsible for making the
fallback path

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-12 Thread Ryosuke Niwa

On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.


Can we create chromium-mac and move everything that's shared between
chromium-mac-leopard and chromium-mac-snowleopard there?

It seems wrong for chromium-mac-leopard to fallback to
chromium-mac-snowleopard.

- Ryosuke
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-12 Thread Adam Barth

On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

This somewhat surprising fallback strategy is common across ports.
The why is explained on this wiki page:

http://trac.webkit.org/wiki/LayoutTestsSearchPath

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-12 Thread Dirk Pranke

On Tue, Jul 12, 2011 at 9:01 PM, Adam Barth aba...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:34 PM, Ryosuke Niwa rn...@webkit.org wrote:
 On Tue, Jul 12, 2011 at 8:19 PM, Dirk Pranke dpra...@chromium.org wrote:

 Hum. I take it back ... it still wouldn't be a tree, since
 chromium-mac-leopard would fall back to chromium-mac-snowleopard, then
 mac-leopard, but chromium-mac-snow-leopard would fall back to
 mac-snowleopard (giving chromium-mac-snowleopard two parents). And it
 looks like chromium-mac-leopard picks up 3,494 baselines from
 mac-leopard :(.

 Can we create chromium-mac and move everything that's shared between
 chromium-mac-leopard and chromium-mac-snowleopard there?
 It seems wrong for chromium-mac-leopard to fallback to
 chromium-mac-snowleopard.

 This somewhat surprising fallback strategy is common across ports.
 The why is explained on this wiki page:

 http://trac.webkit.org/wiki/LayoutTestsSearchPath


In addition, we do actually have a 'chromium-mac'; we don't have a
'chromium-mac-snowleopard'. I think I mixed that in my mind while
typing this with the apple mac ports, where there are mac-leopard,
mac-sl, and mac ports (the latter representing lion/future).

Once Lion ships, chromium will undoubtedly add a chromium-mac-snowleopard dir.

-- Dirk
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-11 Thread Maciej Stachowiak

On Jul 10, 2011, at 12:11 PM, Adam Barth wrote:

On Sun, Jul 10, 2011 at 12:01 PM, James Robinson jam...@google.com wrote:
On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:
Hi webkit-dev,

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

That seems possible:

Adding thousands more PNGs would be unfortunate, especially if a significant
number of them are going to be rolled over frequently.

What are the concrete benefits of the fallback graph being a tree?

Regards,
Maciej

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-11 Thread Adam Barth

On Mon, Jul 11, 2011 at 12:30 AM, Maciej Stachowiak m...@apple.com wrote:
On Jul 10, 2011, at 12:11 PM, Adam Barth wrote:
On Sun, Jul 10, 2011 at 12:01 PM, James Robinson jam...@google.com wrote:
On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:
Hi webkit-dev,

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

That seems possible:

Adding thousands more PNGs would be unfortunate, especially if a significant
number of them are going to be rolled over frequently.

I wouldn't expect a significant number of them to change frequently.

What are the concrete benefits of the fallback graph being a tree?

There are two main benefits:

There will be a transition cost, but the long-term benefits will
likely outweigh the transition costs.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-11 Thread David Levin

On Mon, Jul 11, 2011 at 10:46 AM, Adam Barth aba...@webkit.org wrote:

 There are two main benefits:

 1) A tree is much easier to understand than a rats nest of interwoven
 fallback paths.


I find it impossible to know what the fallback path is. Even with Adam's
graph, it is hard to figure it out as Mark demonstrated and I can understand
why. Largely this seems to be because it is a complex directed acyclic graph
as opposed to a tree.

I think this would be very beneficial.

dave
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-11 Thread Leandro Pereira

On Sun, Jul 10, 2011 at 2:52 PM, Adam Barth aba...@webkit.org wrote:
 These changes might increase the number of image baselines we store in
 the tree for chromium-mac-derived ports (because there will be fewer
 redundant fallback paths), but I expect that cost to be relatively
 small because essentially every port has different image baselines
 anyway.

I like this idea, in general. However, since some ports (GTK+ and EFL,
for instance) shares the same backends (Cairo, FontConfig, Pango,
etc.), pixel tests are often the same.

So, to minimize the size and maintenance effort of the LayoutTests, I
think it would be a good idea to have baselines for the backends, with
fallbacks for the ports. This of course wouldn't be a fallback tree,
but would be simpler than what we currently have.

OTOH, I'm not sure if this would help ports other than the ones
mentioned. Comments?

    Leandro
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-11 Thread Martin Robinson

On Mon, Jul 11, 2011 at 1:34 PM, Leandro Pereira lean...@profusion.mobi wrote:
 I like this idea, in general. However, since some ports (GTK+ and EFL,
 for instance) shares the same backends (Cairo, FontConfig, Pango,
 etc.), pixel tests are often the same.

It's been my experience that pixel results for Cairo ports do not
match unless versions of many dependencies are exactly the same. These
include Fontconfig, FreeType, Pango, Cairo, etc. I think it would be
worthwhile to standardize the dependency versions across all Unixy
Cairo ports.

--Martn
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] LayoutTests results fallback graph

Hi webkit-dev,

In trying to understand how our LayoutTest results system works, I've
created a digram of the fallback graph among the various
platform-specific directories:

https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US

Unfortunately, the fallback graph is not a tree, as one might imagine
initially.  I'd like to propose two small changes, which will
hopefully make the system more sensible globally.  I'm happy to do all
the work required to make these changes:

1) The win port should fall back either to all (the platform
independent results) or to mac, but not to mac-snowleopard, as it
does currently.  (I slightly prefer all, but mac would also be
fine with me.)

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

A) The resulting fallback graph will be a tree, making the fallback
graph easier to understand for both humans and automated tools.
B) The Chromium port will behave more like the other ports (e.g., GTK
and Qt), rather than being a parasite on Apple-maintained ports.

These changes might increase the number of image baselines we store in
the tree for chromium-mac-derived ports (because there will be fewer
redundant fallback paths), but I expect that cost to be relatively
small because essentially every port has different image baselines
anyway.

Thoughts?
Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

2011-07-10 Thread James Robinson

On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:

 Hi webkit-dev,

 In trying to understand how our LayoutTest results system works, I've
 created a digram of the fallback graph among the various
 platform-specific directories:


https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US

 Unfortunately, the fallback graph is not a tree, as one might imagine
 initially.  I'd like to propose two small changes, which will
 hopefully make the system more sensible globally.  I'm happy to do all
 the work required to make these changes:

 1) The win port should fall back either to all (the platform
 independent results) or to mac, but not to mac-snowleopard, as it
 does currently.  (I slightly prefer all, but mac would also be
 fine with me.)

 2) The chromium port should fall back directly to all rather than
 taking a detour through various Apple-maintained ports, as it does
 currently.

 These changes have the following virtues:

 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.
 B) The Chromium port will behave more like the other ports (e.g., GTK
 and Qt), rather than being a parasite on Apple-maintained ports.

 These changes might increase the number of image baselines we store in
 the tree for chromium-mac-derived ports (because there will be fewer
 redundant fallback paths), but I expect that cost to be relatively
 small because essentially every port has different image baselines
 anyway

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

- James


 Thoughts?
 Adam
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Sun, Jul 10, 2011 at 12:01 PM, James Robinson jam...@google.com wrote:
On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:
Hi webkit-dev,

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

That seems possible:

However, I would expect those savings to disappear once we finish
moving chromium-mac to Skia.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Sun, Jul 10, 2011 at 12:11 PM, Adam Barth aba...@webkit.org wrote:
On Sun, Jul 10, 2011 at 12:01 PM, James Robinson jam...@google.com wrote:
On Jul 10, 2011 10:53 AM, Adam Barth aba...@webkit.org wrote:
Hi webkit-dev,

2) The chromium port should fall back directly to all rather than
taking a detour through various Apple-maintained ports, as it does
currently.

These changes have the following virtues:

Could you measure this? I suspect that not falling back on the mac pixel
results will mean checking in a few thousand more pngs, but that's just a
guess.

That seems possible:

However, I would expect those savings to disappear once we finish
moving chromium-mac to Skia.

More numbers for scale:

abarth@quadzen:~/git/webkit/LayoutTests/platform/qt$ find . -name
*.png | wc -l
3852
abarth@quadzen:~/git/webkit/LayoutTests/platform$ find chromium* -name
*.png | wc -l
14653
abarth@quadzen:~/git/webkit/LayoutTests/platform$ find . -name *.png | wc -l
35350

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Sun, Jul 10, 2011 at 12:38 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 10:52, Adam Barth wrote:
 Hi webkit-dev,

 In trying to understand how our LayoutTest results system works, I've
 created a digram of the fallback graph among the various
 platform-specific directories:

 https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US

 Unfortunately, the fallback graph is not a tree, as one might imagine
 initially.  I'd like to propose two small changes, which will
 hopefully make the system more sensible globally.  I'm happy to do all
 the work required to make these changes:

 1) The win port should fall back either to all (the platform
 independent results) or to mac, but not to mac-snowleopard, as it
 does currently.  (I slightly prefer all, but mac would also be
 fine with me.)

 I'd argue that falling back to mac doesn't make any sense.  The regression 
 tests are run against a WebKit using the latest shipping Safari's version of 
 the underlying dependencies.  That almost always corresponds to the latest 
 shipping Mac OS X release's components, and not to the components from future 
 versions of Mac OS X.

There appears to be almost zero practical different between win
falling back to mac versus falling back to mac-snowleopard:

abarth@quadzen:~/git/webkit/LayoutTests/platform$ find mac -name *.png | wc -l
6688
abarth@quadzen:~/git/webkit/LayoutTests/platform$ find
mac-snowleopard/ -name *.png | wc -l
   3

 I also think that falling back to all would only be advisable if we were to 
 also switch Windows DRT away from the legacy Mac-style form controls and 
 rebaseline all of the Windows test results. We've shied away from this in the 
 past because it would result in a big increase in the number of 
 Windows-specific results.

I suspect there's actually an invisible apple port (which is a peer
to chromium, gtk, and qt) that should hold those common
baselines.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph


On 2011-07-10, at 12:46, Adam Barth wrote:

 On Sun, Jul 10, 2011 at 12:38 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 10:52, Adam Barth wrote:
 Hi webkit-dev,
 
 In trying to understand how our LayoutTest results system works, I've
 created a digram of the fallback graph among the various
 platform-specific directories:
 
 https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US
 
 Unfortunately, the fallback graph is not a tree, as one might imagine
 initially.  I'd like to propose two small changes, which will
 hopefully make the system more sensible globally.  I'm happy to do all
 the work required to make these changes:
 
 1) The win port should fall back either to all (the platform
 independent results) or to mac, but not to mac-snowleopard, as it
 does currently.  (I slightly prefer all, but mac would also be
 fine with me.)
 
 I'd argue that falling back to mac doesn't make any sense.  The regression 
 tests are run against a WebKit using the latest shipping Safari's version of 
 the underlying dependencies.  That almost always corresponds to the latest 
 shipping Mac OS X release's components, and not to the components from 
 future versions of Mac OS X.
 
 There appears to be almost zero practical different between win
 falling back to mac versus falling back to mac-snowleopard:
 
 abarth@quadzen:~/git/webkit/LayoutTests/platform$ find mac -name *.png | wc 
 -l
6688
 abarth@quadzen:~/git/webkit/LayoutTests/platform$ find
 mac-snowleopard/ -name *.png | wc -l
   3

I'm not sure what that is meant to convey.  What you're proposing is changing 
the fallback path for the Windows port from one that is conceptually correct 
with one that is incorrect.  Neither your emails so far nor the diagram in your 
initial message have provided any explanation as to why this is.  Perhaps you'd 
like to expand on what purpose this proposed change has and what benefits it 
would provide.

- Mark

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Sun, Jul 10, 2011 at 1:07 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 12:46, Adam Barth wrote:
 On Sun, Jul 10, 2011 at 12:38 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 10:52, Adam Barth wrote:
 Hi webkit-dev,

 In trying to understand how our LayoutTest results system works, I've
 created a digram of the fallback graph among the various
 platform-specific directories:

 https://docs.google.com/drawings/d/1z65SKkWrD4Slm6jobIphHwwRADyUtjOAxwGBVKBY8Kc/edit?hl=en_US

 Unfortunately, the fallback graph is not a tree, as one might imagine
 initially.  I'd like to propose two small changes, which will
 hopefully make the system more sensible globally.  I'm happy to do all
 the work required to make these changes:

 1) The win port should fall back either to all (the platform
 independent results) or to mac, but not to mac-snowleopard, as it
 does currently.  (I slightly prefer all, but mac would also be
 fine with me.)

 I'd argue that falling back to mac doesn't make any sense.  The 
 regression tests are run against a WebKit using the latest shipping 
 Safari's version of the underlying dependencies.  That almost always 
 corresponds to the latest shipping Mac OS X release's components, and not 
 to the components from future versions of Mac OS X.

 There appears to be almost zero practical different between win
 falling back to mac versus falling back to mac-snowleopard:

 abarth@quadzen:~/git/webkit/LayoutTests/platform$ find mac -name *.png | 
 wc -l
    6688
 abarth@quadzen:~/git/webkit/LayoutTests/platform$ find
 mac-snowleopard/ -name *.png | wc -l
       3

 I'm not sure what that is meant to convey.  What you're proposing is changing 
 the fallback path for the Windows port from one that is conceptually correct 
 with one that is incorrect.  Neither your emails so far nor the diagram in 
 your initial message have provided any explanation as to why this is.  
 Perhaps you'd like to expand on what purpose this proposed change has and 
 what benefits it would provide.

Sure.  I'll highlight the relevant section of my original email:

On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:

 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph


On 2011-07-10, at 13:20, Adam Barth wrote:

 Sure.  I'll highlight the relevant section of my original email:
 
 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:
 
 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.

I don't see how Windows falling back to mac-snowleopard has any effect on that. 
 It's no different than mac-leopard in that regard.  Then again, maybe the 
diagram is trying to convey something that I'm missing due to having no idea 
what the difference is between the myriad of different line styles in the 
diagram.

- Mark

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:

 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:

 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.

 I don't see how Windows falling back to mac-snowleopard has any effect on 
 that.  It's no different than mac-leopard in that regard.  Then again, maybe 
 the diagram is trying to convey something that I'm missing due to having no 
 idea what the difference is between the myriad of different line styles in 
 the diagram.

Notice that the circle for win has two arrows emanating from it.
One of those arrows goes to mac and the other goes to
mac-snowleopard.  That means that of the fallback paths that transit
win, one path flows through mac-snowlepard where as the remainder
flow through mac.  If we change win to fall back to mac, then
the graph becomes more tree-like.  (If make change (2) as well, then
the graph globally becomes a tree.)

Having the fallback graph not be a tree causes some strange and
confusing anomalies, which I'd be happy to explain if you don't see
the value in using a fallback tree rather than a fallback DAG.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Jul 10, 2011, at 13:57, Adam Barth aba...@webkit.org wrote:

 On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:
 
 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:
 
 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.
 
 I don't see how Windows falling back to mac-snowleopard has any effect on 
 that.  It's no different than mac-leopard in that regard.  Then again, maybe 
 the diagram is trying to convey something that I'm missing due to having no 
 idea what the difference is between the myriad of different line styles in 
 the diagram.
 
 Notice that the circle for win has two arrows emanating from it.
 One of those arrows goes to mac and the other goes to
 mac-snowleopard.  That means that of the fallback paths that transit
 win, one path flows through mac-snowlepard where as the remainder
 flow through mac.  If we change win to fall back to mac, then
 the graph becomes more tree-like.  (If make change (2) as well, then
 the graph globally becomes a tree.)

Can you please clarify what the edges in your diagram, along with what the 
different line styles, represent?

- Mark

Sent from my iPhone


 
 Having the fallback graph not be a tree causes some strange and
 confusing anomalies, which I'd be happy to explain if you don't see
 the value in using a fallback tree rather than a fallback DAG.
 
 Adam
 ___
 webkit-dev mailing list
 webkit-dev@lists.webkit.org
 http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

On Sun, Jul 10, 2011 at 2:06 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 13:57, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:

 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:

 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.

 I don't see how Windows falling back to mac-snowleopard has any effect on 
 that.  It's no different than mac-leopard in that regard.  Then again, 
 maybe the diagram is trying to convey something that I'm missing due to 
 having no idea what the difference is between the myriad of different line 
 styles in the diagram.

 Notice that the circle for win has two arrows emanating from it.
 One of those arrows goes to mac and the other goes to
 mac-snowleopard.  That means that of the fallback paths that transit
 win, one path flows through mac-snowlepard where as the remainder
 flow through mac.  If we change win to fall back to mac, then
 the graph becomes more tree-like.  (If make change (2) as well, then
 the graph globally becomes a tree.)

 Can you please clarify what the edges in your diagram, along with what the 
 different line styles, represent?

Sure.  The solid arrows at the normal transitive fallback path.  For
example, mac-leopard falls back to mac-snowleopard, which falls back
to mac, which falls back to all (the platform independent results).

Some directories have non-transitive fallback paths, which a indicated
in dashed lines.  For example, chromium-mac-snowleopard falls back to
chromium-mac, which falls back to chromium.  However, the next step in
the fallback path depends on where the path started.  If the path
started at chromium-mac-leopard, the next step in the fallback is to
mac-snowleopard whereas if the path started at chromium-mac, the next
step in the path is to mac-snowleopard.

Similarly, if we arrive at chromium by falling back via chromium-win
(regardless of how we got to chromium-win), the next steps in the
fallback path is win, mac, and, finally, all.

IMHO, these non-transitive fallback paths are nutty and not worth
their complexity.

Now, suppose we disabuse the chromium fallback paths of their
nuttiness and have every fallback path that transits chromium have
all as its next step, then we'll have a reasonably understandable
fallback system, with the sole exception of win being strangely
captive on mac-snowleopard.

In practice, there's extremely little actual difference (e.g., just
one image baseline) between mac and mac-snowleopard, so changing
win to fallback to mac has very little operational cost.

Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

We seem to be talking past one another. Why are there two edges originating at 
win, but not mac-leopard?

Sent from my iPhone

On Jul 10, 2011, at 15:23, Adam Barth aba...@webkit.org wrote:

 On Sun, Jul 10, 2011 at 2:50 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 14:27, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 2:06 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 13:57, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:
 
 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:
 
 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.
 
 I don't see how Windows falling back to mac-snowleopard has any effect 
 on that.  It's no different than mac-leopard in that regard.  Then 
 again, maybe the diagram is trying to convey something that I'm missing 
 due to having no idea what the difference is between the myriad of 
 different line styles in the diagram.
 
 Notice that the circle for win has two arrows emanating from it.
 One of those arrows goes to mac and the other goes to
 mac-snowleopard.  That means that of the fallback paths that transit
 win, one path flows through mac-snowlepard where as the remainder
 flow through mac.  If we change win to fall back to mac, then
 the graph becomes more tree-like.  (If make change (2) as well, then
 the graph globally becomes a tree.)
 
 Can you please clarify what the edges in your diagram, along with what the 
 different line styles, represent?
 
 Sure.
 
 Thanks. My confusion here comes from the idea that Windows falling back on 
 SnowLeopard causes some sort of non-tree-like complexity that other 
 platforms falling back via SnowLeopard aren't also subject to. The behaviour 
 of Leopard and Windows seems incredibly similar in this regard so I'm very 
 unclear as to why only Windows is problematic.
 
 Being a tree is a global property, not a local property.  There are
 two edges emanating from win.  In order for the graph to be a tree
 one of them must be removed.  Neither one, in isolation, makes the
 graph not a tree.
 
 There's an additional confusing element here:  Only a subset of 
 Lion-specific results are currently checked in. The difference between mac 
 and mac-snowleopard results is likely much bigger than you realise.
 
 Ah, well, I, of course, can't see invisible results.
 
 Adam
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

Because the LayoutTest fallback graph is a mess, hence this email thread.  :)

More proximately, because when the chromium-mac-leopard (for
example) fallback path flows through mac-leopard, it flows to
mac-snowleopard alongside the fallback path that originates with
mac-leopard.  Now, in the case of win, when the chromium-win
(for example) fallback path flows through win, it flows thereafter
to mac directly whereas the fallback path that originates with win
takes a detour by way of mac-snowleopard.  The fact that these two
fallback paths diverge at this point is one of the reasons the
fallback graph is not a tree.

Adam


On Sun, Jul 10, 2011 at 3:32 PM, Mark Rowe mr...@apple.com wrote:
 We seem to be talking past one another. Why are there two edges originating 
 at win, but not mac-leopard?

 Sent from my iPhone

 On Jul 10, 2011, at 15:23, Adam Barth aba...@webkit.org wrote:

 On Sun, Jul 10, 2011 at 2:50 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 14:27, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 2:06 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 13:57, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:

 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org wrote:
 These changes have the following virtues:

 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.

 I don't see how Windows falling back to mac-snowleopard has any effect 
 on that.  It's no different than mac-leopard in that regard.  Then 
 again, maybe the diagram is trying to convey something that I'm missing 
 due to having no idea what the difference is between the myriad of 
 different line styles in the diagram.

 Notice that the circle for win has two arrows emanating from it.
 One of those arrows goes to mac and the other goes to
 mac-snowleopard.  That means that of the fallback paths that transit
 win, one path flows through mac-snowlepard where as the remainder
 flow through mac.  If we change win to fall back to mac, then
 the graph becomes more tree-like.  (If make change (2) as well, then
 the graph globally becomes a tree.)

 Can you please clarify what the edges in your diagram, along with what 
 the different line styles, represent?

 Sure.

 Thanks. My confusion here comes from the idea that Windows falling back on 
 SnowLeopard causes some sort of non-tree-like complexity that other 
 platforms falling back via SnowLeopard aren't also subject to. The 
 behaviour of Leopard and Windows seems incredibly similar in this regard so 
 I'm very unclear as to why only Windows is problematic.

 Being a tree is a global property, not a local property.  There are
 two edges emanating from win.  In order for the graph to be a tree
 one of them must be removed.  Neither one, in isolation, makes the
 graph not a tree.

 There's an additional confusing element here:  Only a subset of 
 Lion-specific results are currently checked in. The difference between mac 
 and mac-snowleopard results is likely much bigger than you realise.

 Ah, well, I, of course, can't see invisible results.

 Adam

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

Ok, the fact that chromium-win's fallback behaviour uses win results but 
doesn't match win's fallback behaviour is what I was missing. Couldn't we also 
address that by changing the behaviour of chromium-win?

- Mark

Sent from my iPhone

On Jul 10, 2011, at 15:55, Adam Barth aba...@webkit.org wrote:

 Because the LayoutTest fallback graph is a mess, hence this email thread.  :)
 
 More proximately, because when the chromium-mac-leopard (for
 example) fallback path flows through mac-leopard, it flows to
 mac-snowleopard alongside the fallback path that originates with
 mac-leopard.  Now, in the case of win, when the chromium-win
 (for example) fallback path flows through win, it flows thereafter
 to mac directly whereas the fallback path that originates with win
 takes a detour by way of mac-snowleopard.  The fact that these two
 fallback paths diverge at this point is one of the reasons the
 fallback graph is not a tree.
 
 Adam
 
 
 On Sun, Jul 10, 2011 at 3:32 PM, Mark Rowe mr...@apple.com wrote:
 We seem to be talking past one another. Why are there two edges originating 
 at win, but not mac-leopard?
 
 Sent from my iPhone
 
 On Jul 10, 2011, at 15:23, Adam Barth aba...@webkit.org wrote:
 
 On Sun, Jul 10, 2011 at 2:50 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 14:27, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 2:06 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 13:57, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:
 
 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org 
 wrote:
 These changes have the following virtues:
 
 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.
 
 I don't see how Windows falling back to mac-snowleopard has any effect 
 on that.  It's no different than mac-leopard in that regard.  Then 
 again, maybe the diagram is trying to convey something that I'm 
 missing due to having no idea what the difference is between the 
 myriad of different line styles in the diagram.
 
 Notice that the circle for win has two arrows emanating from it.
 One of those arrows goes to mac and the other goes to
 mac-snowleopard.  That means that of the fallback paths that transit
 win, one path flows through mac-snowlepard where as the remainder
 flow through mac.  If we change win to fall back to mac, then
 the graph becomes more tree-like.  (If make change (2) as well, then
 the graph globally becomes a tree.)
 
 Can you please clarify what the edges in your diagram, along with what 
 the different line styles, represent?
 
 Sure.
 
 Thanks. My confusion here comes from the idea that Windows falling back on 
 SnowLeopard causes some sort of non-tree-like complexity that other 
 platforms falling back via SnowLeopard aren't also subject to. The 
 behaviour of Leopard and Windows seems incredibly similar in this regard 
 so I'm very unclear as to why only Windows is problematic.
 
 Being a tree is a global property, not a local property.  There are
 two edges emanating from win.  In order for the graph to be a tree
 one of them must be removed.  Neither one, in isolation, makes the
 graph not a tree.
 
 There's an additional confusing element here:  Only a subset of 
 Lion-specific results are currently checked in. The difference between mac 
 and mac-snowleopard results is likely much bigger than you realise.
 
 Ah, well, I, of course, can't see invisible results.
 
 Adam
 
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph

Yes.  As I said before:

On Sun, Jul 10, 2011 at 3:23 PM, Adam Barth aba...@webkit.org wrote:
 Being a tree is a global property, not a local property.  There are
 two edges emanating from win.  In order for the graph to be a tree
 one of them must be removed.  Neither one, in isolation, makes the
 graph not a tree.

Adam


On Sun, Jul 10, 2011 at 5:09 PM, Mark Rowe mr...@apple.com wrote:
 Ok, the fact that chromium-win's fallback behaviour uses win results but 
 doesn't match win's fallback behaviour is what I was missing. Couldn't we 
 also address that by changing the behaviour of chromium-win?

 - Mark

 Sent from my iPhone

 On Jul 10, 2011, at 15:55, Adam Barth aba...@webkit.org wrote:

 Because the LayoutTest fallback graph is a mess, hence this email thread.  :)

 More proximately, because when the chromium-mac-leopard (for
 example) fallback path flows through mac-leopard, it flows to
 mac-snowleopard alongside the fallback path that originates with
 mac-leopard.  Now, in the case of win, when the chromium-win
 (for example) fallback path flows through win, it flows thereafter
 to mac directly whereas the fallback path that originates with win
 takes a detour by way of mac-snowleopard.  The fact that these two
 fallback paths diverge at this point is one of the reasons the
 fallback graph is not a tree.

 Adam


 On Sun, Jul 10, 2011 at 3:32 PM, Mark Rowe mr...@apple.com wrote:
 We seem to be talking past one another. Why are there two edges originating 
 at win, but not mac-leopard?

 Sent from my iPhone

 On Jul 10, 2011, at 15:23, Adam Barth aba...@webkit.org wrote:

 On Sun, Jul 10, 2011 at 2:50 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 14:27, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 2:06 PM, Mark Rowe mr...@apple.com wrote:
 On Jul 10, 2011, at 13:57, Adam Barth aba...@webkit.org wrote:
 On Sun, Jul 10, 2011 at 1:26 PM, Mark Rowe mr...@apple.com wrote:
 On 2011-07-10, at 13:20, Adam Barth wrote:
 Sure.  I'll highlight the relevant section of my original email:

 On Sun, Jul 10, 2011 at 10:52 AM, Adam Barth aba...@webkit.org 
 wrote:
 These changes have the following virtues:

 A) The resulting fallback graph will be a tree, making the fallback
 graph easier to understand for both humans and automated tools.

 I don't see how Windows falling back to mac-snowleopard has any 
 effect on that.  It's no different than mac-leopard in that regard.  
 Then again, maybe the diagram is trying to convey something that I'm 
 missing due to having no idea what the difference is between the 
 myriad of different line styles in the diagram.

 Notice that the circle for win has two arrows emanating from it.
 One of those arrows goes to mac and the other goes to
 mac-snowleopard.  That means that of the fallback paths that transit
 win, one path flows through mac-snowlepard where as the remainder
 flow through mac.  If we change win to fall back to mac, then
 the graph becomes more tree-like.  (If make change (2) as well, then
 the graph globally becomes a tree.)

 Can you please clarify what the edges in your diagram, along with what 
 the different line styles, represent?

 Sure.

 Thanks. My confusion here comes from the idea that Windows falling back 
 on SnowLeopard causes some sort of non-tree-like complexity that other 
 platforms falling back via SnowLeopard aren't also subject to. The 
 behaviour of Leopard and Windows seems incredibly similar in this regard 
 so I'm very unclear as to why only Windows is problematic.

 Being a tree is a global property, not a local property.  There are
 two edges emanating from win.  In order for the graph to be a tree
 one of them must be removed.  Neither one, in isolation, makes the
 graph not a tree.

 There's an additional confusing element here:  Only a subset of 
 Lion-specific results are currently checked in. The difference between 
 mac and mac-snowleopard results is likely much bigger than you realise.

 Ah, well, I, of course, can't see invisible results.

 Adam


___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

Re: [webkit-dev] LayoutTests results fallback graph