Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-05-21 Thread Adam Roben
On Feb 23, 2010, at 6:42 PM, Maciej Stachowiak wrote: > The challenge with reftests is coming up with different markup that's > supposed to render exactly the same, but for unrelated reasons. While it's > possible, for example, to compare a table layout to absolute positioned > boxes, for some

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-21 Thread Ojan Vafai
On Tue, Apr 13, 2010 at 1:49 PM, Simon Fraser wrote: > dumpAsText also turns off pixel results, which seems like a loss of useful > testing. What's the plan there? > The hope is that the dumpRenderTree tests will be sufficient testing for pixel results. On Tue, Apr 13, 2010 at 2:55 PM, Dirk Pran

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread Dirk Pranke
On Tue, Apr 13, 2010 at 11:54 AM, Ojan Vafai wrote: > On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak wrote: >> >> On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: >> >> On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak wrote: >>> >>> On the one hand, it's good to shift the default. On the other

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread Simon Fraser
On Apr 13, 2010, at 11:54 AM, Ojan Vafai wrote: > On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak wrote: > On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: >> On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak wrote: >> On the one hand, it's good to shift the default. On the other hand, many >>

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread David Hyatt
There are also a fair number of directories that test rendering and layout, although they are usually obvious by their names... many of the subdirectories in "fast" cannot be converted to pure text tests (clipping, margin collapsing, backgrounds, etc.). dave On Apr 13, 2010, at 1:54 PM, Ojan V

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-04-13 Thread Ojan Vafai
On Mon, Mar 1, 2010 at 7:53 PM, Maciej Stachowiak wrote: > On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: > > On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak wrote: > >> On the one hand, it's good to shift the default. On the other hand, many >> render tree dumping tests do not currently need t

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-09 Thread Jeremy Orlow
On Tue, Mar 9, 2010 at 9:19 AM, Robert O'Callahan wrote: > > Mozilla has been using this technique for years. Perhaps we can pick their >> brains for some good tricks. Or, dare I say it, share some tests. >> >> >> > Hi, hope I'm not crashing the party, and sorry I'm late :-). Let me just > say a f

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-09 Thread Robert O'Callahan
> Mozilla has been using this technique for years. Perhaps we can pick their > brains for some good tricks. Or, dare I say it, share some tests. > > Hi, hope I'm not crashing the party, and sorry I'm late :-). Let me just say a few things about reftests... Maciej mentioned that a reftest can asser

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Maciej Stachowiak
On Mar 1, 2010, at 6:46 PM, Ojan Vafai wrote: On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak wrote: On the one hand, it's good to shift the default. On the other hand, many render tree dumping tests do not currently need to run any JavaScript script, while many text-only tests do. I wo

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Ojan Vafai
On Mon, Mar 1, 2010 at 6:40 PM, Maciej Stachowiak wrote: > On Mar 1, 2010, at 5:49 PM, Ojan Vafai wrote: > > On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel wrote: > >> On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler wrote: >> > But in practice pixel results are often ignored entirely. I think that >

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Maciej Stachowiak
On Mar 1, 2010, at 5:49 PM, Ojan Vafai wrote: On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel wrote: On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler wrote: > But in practice pixel results are often ignored entirely. I think that reftest-style tests if done right could be a great addition. Also,

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-03-01 Thread Ojan Vafai
On Tue, Feb 23, 2010 at 3:26 PM, Eric Seidel wrote: > On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler wrote: > > But in practice pixel results are often ignored entirely. I think that > reftest-style tests if done right could be a great addition. > > Also, there are zillions of render-tree-dump tes

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Maciej Stachowiak
On Feb 23, 2010, at 8:14 AM, Darin Adler wrote: On Feb 23, 2010, at 8:08 AM, Ariya Hidayat wrote: When the test is run, the browser loads both files, takes snapshots, and does a pixel comparison. Thus font differences between platforms become less of an issue. Isn't it our existing pixel

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Eric Seidel
On Tue, Feb 23, 2010 at 2:19 PM, Darin Adler wrote: > But in practice pixel results are often ignored entirely. I think that > reftest-style tests if done right could be a great addition. Pixel tests are run for every build by chromium, and regressions tracked there. :) Also, there are zillion

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Cameron McCormack
Dimitri Glazkov: > I agree. We should organize cross-browser-test-fest and pick each > other's brains. I believe some Mozilla people are interested in working on a cross browser test suite framework, based in part on reftests. Jonathan Watt and fantasai are people to coordinate with on this, IIRC

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Adler
On Feb 23, 2010, at 11:06 AM, Andras Becsi wrote: > I like this concept of reftests because it seems easier to maintain and > scales better on multiple ports than the current form of testing. A reftest-style test seems like a good alternative to a pure text test. It can handle testing a lot mor

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Fisher
The first step is probably to get the infrastructure in place so that new tests can be written as ref tests when possible. -Darin On Tue, Feb 23, 2010 at 11:24 AM, Dimitri Glazkov wrote: > I agree. We should organize cross-browser-test-fest and pick each > other's brains. > > I like the idea of

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Dimitri Glazkov
I agree. We should organize cross-browser-test-fest and pick each other's brains. I like the idea of introducing ref tests and I think they are teh hot. My only concern would be to the amount effort of converting existing tests to that, so I think we should start small. :DG< On Tue, Feb 23, 2010

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread David Levin
On Tue, Feb 23, 2010 at 11:06 AM, Andras Becsi wrote: > > BTW, does the python based layout testing framework, which was checked in > recently, support pixel tests? AFAIK it can run multiple DRT's in parallel > and speed up the testing process which wouldn't be bad a bad thing in this > case eithe

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Andras Becsi
Hi, I like this concept of reftests because it seems easier to maintain and scales better on multiple ports than the current form of testing. On the other hand it has to be noted that it would apparently be much work to create reference tests for the current tests which doesn't appear to be a

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Sam Weinig
On Tue, Feb 23, 2010 at 12:58 PM, Dirk Schulze wrote: > Am Dienstag, den 23.02.2010, 08:34 -0800 schrieb Simon Fraser: > > > It could be an image, or it could be a configuration of elements, > or a table, or something else that can be configured to look exactly the > same as the CSS border prope

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Adler
On Feb 23, 2010, at 9:58 AM, Dirk Schulze wrote: > But how do we write tests for box-shadows, gradients or patterns on a text > (like in SVG)? Possibly with . -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.or

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Dirk Schulze
Am Dienstag, den 23.02.2010, 08:34 -0800 schrieb Simon Fraser: > It could be an image, or it could be a configuration of elements, or a > table, or something else that can be configured to look exactly the same as > the CSS border property being tested. > > Simon I like the idea of reftests.

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
On 23/2/10 17:34 , Simon Fraser wrote: On Feb 23, 2010, at 8:21 AM, Tor Arne Vestbø wrote: On 23/2/10 17:02 , Simon Fraser wrote: I think the correct longterm solution to this problem is to use reftests. A reftest consists of two files; the test file, and a "reference" file that should give t

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Simon Fraser
On Feb 23, 2010, at 8:21 AM, Tor Arne Vestbø wrote: > On 23/2/10 17:02 , Simon Fraser wrote: >> On Feb 23, 2010, at 5:00 AM, Tor Arne Vestbø wrote: >> >>> Hey all, >>> >>> A reoccurring problem when trying to maintain layout-test results >>> is differences in font and theme metrics for tests tha

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
On 23/2/10 17:02 , Simon Fraser wrote: On Feb 23, 2010, at 5:00 AM, Tor Arne Vestbø wrote: Hey all, A reoccurring problem when trying to maintain layout-test results is differences in font and theme metrics for tests that dump the render tree. Often a test does not actually test font loading/r

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Darin Adler
On Feb 23, 2010, at 8:08 AM, Ariya Hidayat wrote: >> When the test is run, the browser loads both files, takes snapshots, and >> does a pixel comparison. Thus font differences between platforms become less >> of an issue. > > Isn't it our existing pixel test infrastructure? Or do you mean somet

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Ariya Hidayat
> When the test is run, the browser loads both files, takes snapshots, and does > a pixel comparison. > Thus font differences between platforms become less of an issue. Isn't it our existing pixel test infrastructure? Or do you mean something else? -- Ariya Hidayat http://www.linkedin.com/in/a

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Simon Fraser
On Feb 23, 2010, at 5:00 AM, Tor Arne Vestbø wrote: > Hey all, > > A reoccurring problem when trying to maintain layout-test results is > differences in font and theme metrics for tests that dump the render tree. > Often a test does not actually test font loading/rendering or theming, but > ha

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Tor Arne Vestbø
On 23/2/10 14:15 , Evan Martin wrote: On Tue, Feb 23, 2010 at 2:00 PM, Tor Arne Vestbø wrote: Lately we've been playing with the idea of using SVG fonts for the Qt port to get the same set of expected results for qt-mac, qt-linux and qt-win, by injecting new @font-face rules using a user-style

Re: [webkit-dev] Increasing the number of cross-platform/port expected results

2010-02-23 Thread Evan Martin
On Tue, Feb 23, 2010 at 2:00 PM, Tor Arne Vestbø wrote: > Lately we've been playing with the idea of using SVG fonts for the Qt port > to get the same set of expected results for qt-mac, qt-linux and qt-win, by > injecting new @font-face rules using a user-stylesheet and preventing > platform-font