Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-14 Thread Dirk Pranke
On Wed, Feb 13, 2013 at 11:33 PM, Benjamin Poulain benja...@webkit.org wrote: On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke dpra...@chromium.org wrote: Those changes are not harmless. There are people monitoring tests results full time in order to keep WebKit in good shape. No other part

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-14 Thread Dongsung Huang
I'm curious, what would you imagine the ref test contains? If I am not mistaken, the composition operations are parallel with the ones of SVG and Canvas (aren't they?). I would have attempted comparing the 3 implementations as it seems to me the pixels values should be the same. That

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-13 Thread Dongsung Huang
I like this idea. I cannot find any harm if we have this functionality. When a pixel test is more succinct, we can make a pixel test. When a 'getPixel on Javascript' test is more succinct, we can make a 'getPixel' test. As http://trac.webkit.org/wiki/Writing%20Layout%20Tests%20for%20DumpRenderTree

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang luxte...@company100.netwrote: I like this idea. I cannot find any harm if we have this functionality. Those changes are not harmless. There are people monitoring tests results full time in order to keep WebKit in good shape. No other part of

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-13 Thread noam . rosenthal
-dev@lists.webkit.org Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly On Wed, Feb 13, 2013 at 9:38 PM, Dongsung Huang luxte...@company100.netmailto:luxte...@company100.net wrote: I like this idea. I cannot find any harm if we have

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 11:16 PM, Dirk Pranke dpra...@chromium.org wrote: Those changes are not harmless. There are people monitoring tests results full time in order to keep WebKit in good shape. No other part of WebKit require continuous attention. I'm sorry, but either I don't

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-13 Thread Benjamin Poulain
On Wed, Feb 13, 2013 at 11:27 PM, noam.rosent...@nokia.com wrote: Why wasn't a ref-test a better solution in this particular case? Because gaussian blurs on the GPU are not accurate, and look slightly different on different GPUs, but usually close enough. We need a way to measure close

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-13 Thread noam . rosenthal
...@company100.net, webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly On Wed, Feb 13, 2013 at 11:27 PM, noam.rosent

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-10 Thread Ryosuke Niwa
On Sat, Feb 9, 2013 at 5:35 PM, Dirk Pranke dpra...@chromium.org wrote: Perhaps -- and this is a tangent to this thread -- we could also investigate ways of writing tests that did render pngs but told NRWT to ignore them (e.g., dumpAsTextAndIgnoredImage() or something), or conveyed which

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread noam . rosenthal
@lists.webkit.orgmailto:webkit-dev@lists.webkit.org webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly On Fri, Feb 8, 2013 at 3:16 PM, noam.rosent...@nokia.commailto:noam.rosent...@nokia.com

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread Benjamin Poulain
On Sat, Feb 9, 2013 at 1:24 AM, noam.rosent...@nokia.com wrote: I did not say they cannot be tested with the two methods suggested :) It's more about a preference to move some of those decisions from the test infrastructure to the test itself, but potentially those bugs could be tested in

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread noam . rosenthal
From: ext Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org This is the situation I am afraid with a solution where pixels would be evaluated from JavaScript. You can test, but you lack visibility why something is correct or incorrect. Text tests, ref-tests and pixel tests have a

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread noam . rosenthal
@lists.webkit.orgmailto:webkit-dev@lists.webkit.org webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly On Sat, Feb 9, 2013 at 1:24 AM, noam.rosent...@nokia.commailto:noam.rosent

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread Simon Fraser
10:57 AM To: Noam Rosenthal noam.rosent...@nokia.com Cc: rn...@webkit.org rn...@webkit.org, webkit-dev@lists.webkit.org webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly On Sat, Feb 9, 2013 at 1:24 AM

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread noam . rosenthal
Oops, forgot to reply to the list. From: Noam Rosenthal noam.rosent...@nokia.commailto:noam.rosent...@nokia.com From: ext Simon Fraser simon.fra...@apple.commailto:simon.fra...@apple.com The existing pauseAnimation DRT API can stop an animation mid-flight, to allow for reliable snapshotting. Why

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-09 Thread Dirk Pranke
On Sat, Feb 9, 2013 at 1:57 AM, Benjamin Poulain benja...@webkit.org wrote: On Sat, Feb 9, 2013 at 1:24 AM, noam.rosent...@nokia.com wrote: I did not say they cannot be tested with the two methods suggested :) It's more about a preference to move some of those decisions from the test

[webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread noam . rosenthal
Hellos we don't currently have a solution in webkit's test infrastructure for testing animations on the fly, or in general for testing multiple image results in the same test. (If this was discussed before and I'm unaware of that discussion, please stop me here...) This is because ImageDiff

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Robert Kroeger
Questions inline. On Fri, Feb 8, 2013 at 8:35 AM, noam.rosent...@nokia.com wrote: Hellos we don't currently have a solution in webkit's test infrastructure for testing animations on the fly, or in general for testing multiple image results in the same test. I have been wishing for such a

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread noam . rosenthal
From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] on behalf of ext Robert Kroeger [rjkro...@chromium.org] How would the test wait for the animation to happen? Tests that use timeouts seem more likely to be flaky from what

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Michelangelo De Simone
I'd like to propose a solution, and would welcome some feedback on whether it's a good one... The idea is that you would be able to programatically retrieve the current snapshot into a canvas ImageData, and then compare the pixel results with JavaScript in the LayoutTest. Something like:

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Ryosuke Niwa
On Fri, Feb 8, 2013 at 5:35 AM, noam.rosent...@nokia.com wrote: we don't currently have a solution in webkit's test infrastructure for testing animations on the fly, or in general for testing multiple image results in the same test. (If this was discussed before and I'm unaware of that

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa rn...@webkit.org wrote: I had similar thoughts but my counter proposal is to let DRT/WTR generate multiple actual results either in the form of multiple layers in PNG or multiple PNG images. The advantage of this latter approach is that we can

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Dirk Pranke
On Fri, Feb 8, 2013 at 11:12 AM, Ryosuke Niwa rn...@webkit.org wrote: On Fri, Feb 8, 2013 at 5:35 AM, noam.rosent...@nokia.com wrote: we don't currently have a solution in webkit's test infrastructure for testing animations on the fly, or in general for testing multiple image results in the

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread noam . rosenthal
From: ext Ryosuke Niwa rn...@webkit.orgmailto:rn...@webkit.org I'd like to propose a solution, and would welcome some feedback on whether it's a good one... The idea is that you would be able to programatically retrieve the current snapshot into a canvas ImageData, and then compare the pixel

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 3:16 PM, noam.rosent...@nokia.com wrote: The problem with dynamic features of the web like animations/interactions is that they're non-deterministic, or at least a lot less deterministic than static features of the web like layouts. Ref tests, pixel tests etc. are

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Gregg Tavares
Can you expose a time or setTime function in DRT and some option that says let JS control the clock? Then a test could do something like layoutTestController.overridePreference(JsControlledClock, 1); // Render 5 frames over 1 second var frame =0; function renderFrame() {

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Benjamin Poulain
On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares g...@google.com wrote: Can you expose a time or setTime function in DRT and some option that says let JS control the clock? Unfortunately, that only works in the simple cases. We could cheat the WebCore clock, but that would ultimately be wrong

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Gregg Tavares
On Fri, Feb 8, 2013 at 4:16 PM, Benjamin Poulain benja...@webkit.orgwrote: On Fri, Feb 8, 2013 at 4:12 PM, Gregg Tavares g...@google.com wrote: Can you expose a time or setTime function in DRT and some option that says let JS control the clock? Unfortunately, that only works in the simple

Re: [webkit-dev] Testing feature suggestion: animation/interaction pixel-results on the fly

2013-02-08 Thread Philip Rogers
I'm not sure if this is generally known: we use [svg document].setCurrentTime(time) in the svg/animations tests for both script and pixel tests. The SVG animation test harness leaves a lot to be desired, but it may be a good place to start for a more general DRT approach: