[webkit-dev] layout tests: how are some compared against image, and others only text?
What is it that causes some tests to require baseline images (and not text files) for comparison, while others require text and not image baselines? (I know that I can specifically SKIP comparison against IMAGE and/or TEXT using test_expectations.txt... but even without the use of test_expectations, I believe that some tests are compared against only text or only image.) As an example, I see that this test has only image baselines and no text baselines: $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.png | wc -l 10 $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt | wc -l ls: cannot access LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt: No such file or directory 0 while this test has only text baselines and no image baselines: $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png | wc -l ls: cannot access LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png: No such file or directory 0 $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.txt | wc -l 5 Is there something inherent in each test that indicates whether its results will be compared against image and/or text baselines? Or is it simply a matter of what baseline files are found to compare against? ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] layout tests: how are some compared against image, and others only text?
This is probably more appropriate for webkit-help but... In general, tests that call layoutTestController.dumpAsTest() or layoutTestController.dumpAsTest(false) and ref tests don't have expected.png files. I'm not aware of tests that don't require expected.txt however. Maybe they're specific to some canvas tests? - Ryosuke On Wed, Jan 4, 2012 at 3:39 PM, Elliot Poger wrote: > What is it that causes some tests to require baseline images (and not text > files) for comparison, while others require text and not image baselines? > > (I know that I can specifically SKIP comparison against IMAGE and/or TEXT > using test_expectations.txt... but even without the use of > test_expectations, I believe that some tests are compared against only text > or only image.) > > As an example, I see that this test has only image baselines and no text > baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.png | wc -l > 10 > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt: No such file > or directory > 0 > > while this test has only text baselines and no image baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png: No such file or > directory > 0 > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.txt | wc -l > 5 > > Is there something inherent in each test that indicates whether its > results will be compared against image and/or text baselines? Or is it > simply a matter of what baseline files are found to compare against? > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] layout tests: how are some compared against image, and others only text?
On Wed, Jan 4, 2012 at 3:39 PM, Elliot Poger wrote: > What is it that causes some tests to require baseline images (and not text > files) for comparison, while others require text and not image baselines? > > (I know that I can specifically SKIP comparison against IMAGE and/or TEXT > using test_expectations.txt... but even without the use of > test_expectations, I believe that some tests are compared against only text > or only image.) > > As an example, I see that this test has only image baselines and no text > baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.png | wc -l > 10 > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt: No such file > or directory > 0 > Wrong, you've missed some: $ find LayoutTests/ -name canvas-text-baseline* LayoutTests/platform/chromium-linux/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/gtk/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/qt/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/efl/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/chromium-win/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/chromium-gpu-linux/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/chromium-gpu-win/fast/canvas/canvas-text-baseline-expected.png LayoutTests/platform/mac/fast/canvas/canvas-text-baseline-expected.png LayoutTests/fast/canvas/canvas-text-baseline.html LayoutTests/fast/canvas/canvas-text-baseline-expected.txt < this one applies > while this test has only text baselines and no image baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png: No such file or > directory > 0 > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.txt | wc -l > 5 > This test calls layoutTestController.dumpAsText() (in js-test-pre.js), so there's no .png. - James > > Is there something inherent in each test that indicates whether its > results will be compared against image and/or text baselines? Or is it > simply a matter of what baseline files are found to compare against? > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] layout tests: how are some compared against image, and others only text?
On Wed, Jan 4, 2012 at 6:39 PM, Elliot Poger wrote: > What is it that causes some tests to require baseline images (and not text > files) for comparison, while others require text and not image baselines? > > (I know that I can specifically SKIP comparison against IMAGE and/or TEXT > using test_expectations.txt... but even without the use of > test_expectations, I believe that some tests are compared against only text > or only image.) > > As an example, I see that this test has only image baselines and no text > baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.png | wc -l > 10 > $ ls LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-text-baseline*.txt: No such file > or directory > Tests with platform-independent results will often put the .txt files in the same directory as the .html. (the final fallback path for layout test results is the source directory.) Try: $ ls LayoutTests/fast/canvas/canvas-text-baseline*.txt LayoutTests/fast/canvas/canvas-text-baseline-expected.txt > while this test has only text baselines and no image baselines: > > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png | wc -l > ls: cannot access > LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.png: No such file or > directory > 0 > $ ls LayoutTests/platform/*/fast/canvas/canvas-lineWidth*.txt | wc -l > 5 > > Is there something inherent in each test that indicates whether its > results will be compared against image and/or text baselines? Or is it > simply a matter of what baseline files are found to compare against? > As rniwa@ mentioned, this is controlled by the test's JS code itself with layoutTestController.dumpAsText(). Stephen > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
