Re: [webkit-dev] Running pixel tests on build.webkit.org
And one very quick, short-term solution: 3. Generate new pixel results to match the current behavior, and check them in as hypothetically correct. And of course if someone notices an existing problem and fixes it, they check in corrected images then. It doesn't help find current problems, but those are being missed now anyway. It does let the tests be run again approximately immediately, even faster than waiting for test expectations functionality, so we can catch regressions moving forward. - Pam On Thu, Jan 7, 2010 at 5:01 PM, Ojan Vafai o...@chromium.org wrote: On Thu, Jan 7, 2010 at 10:22 AM, Darin Adler da...@apple.com wrote: On Jan 7, 2010, at 10:19 AM, Dimitri Glazkov wrote: Are we planning to run pixel tests on the build bots? If we can get them green, we should. It’s a lot of work. We need a volunteer to do that work. We’ve tried before. Two possible long-term solutions come to mind: 1. Turn the bots orange on pixel failures. They still need fixing, but are not as severe as text diff failures. I'm not a huge fan of this, but it's an option. 2. Add in a concept of expected failures and only turn the bots red for *unexpected* failurs. More details on this below. In chromium-land, there's an expectations file that lists expected failures and allows for distinguishing different types of failures (e.g. IMAGE vs. TEXT). It's like Skipped lists, but doesn't necessarily skip the test. Fixing the expected failures still needs doing of course, but can be done asynchronously. The primary advantage of this approach is that we can turn on pixel tests, keep the bots green and avoid further regressions. Would something like that make sense for WebKit as a whole? To be clear, we would be nearly as loathe to add tests to this file as we are about adding them to the Skipped lists. This just provides a way forward. While it's true that the bots used to be red more frequently with pixel tests turned on, for the most part, there weren't significant pixel regressions. Now, if you run the pixel tests on a clean build, there are a number of failures and a very large number of hash-mismatches that are within the failure tolerance level. -Ojan For reference, the format of the expectations file is something like this: // Fails the image diff but not the text diff. fast/forms/foo.html = IMAGE // Fails just the text diff. fast/forms/bar.html = TEXT // Fails both the image and text diffs. fast/forms/baz.html = IMAGE+TEXT // Skips this test (e.g. because it hangs run-webkit-tests or causes other tests to fail). SKIP : fast/forms/foo1.html = IMAGE ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Selection highlight painting (gaps?)
On Mon, Oct 19, 2009 at 1:43 PM, Peter Kasting pkast...@google.com wrote: On Mon, Oct 19, 2009 at 1:40 PM, Darin Fisher da...@chromium.org wrote: I think there is a good use case for copying a selection of HTML from any web page and pasting that into the rich text editor of a web mail program. I agree, but that case does not degrade badly when you only copy the text, whereas if there is any case where you _don't_ want to copy the images etc., the current behavior makes those use cases impossible. I don't follow this argument. If I want the text, tables, images, etc., then if copy only grabs the text, what I want is impossible. If I only want the text, at least I can remove the other stuff after pasting: inconvenient, but possible. I must not understand what you're describing. - Pam Note that within Chrome we put in ctrl-shift-v to paste as plain text precisely because of issues like this. Most other programs don't have that option though (and even in Chrome it's hard to discover). PK ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] currently failing regression tests on Mac OS X Leopard
On Fri, Mar 6, 2009 at 7:47 AM, Darin Adler da...@apple.com wrote: After fixing the test I could easily deal with myself, the following tests are failing for me: fast/block/float/t0905-c414-flt-fit-01-d-g.html fast/block/float/t0905-c5525-fltblck-00-d-ag.html fast/block/float/t0905-c5526-flthw-00-c-g.html It looks like the check-in to deal with the YinYang character included some incorrect test results, or at least results that don't work on my computer. We need to correct the test results. It may not be the only problem, but those tests are missing some necessary resources. None of the images in support/ that they look for are there. - Pam fast/repaint/transform-absolute-in-positioned-container.html Simon, you added this test originally so maybe you can figure out why it's giving me a result with an element 20 pixels wide instead of 40 now. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
Re: [webkit-dev] Getting more buildbots green
On Sat, Jan 3, 2009 at 11:41 AM, Darin Adler da...@apple.com wrote: Looking at the buildbot I see a few broken regression tests: fast/text/find-case-folding.html — Looks like I broke this one when I introduced the ICU usearch-based text searching. I'll try to fix it. editing/selection/move-left-right.html — The failure here is curious. The actual test output seems to be unchanged, but all the WARNING lines mentioning moving in the wrong direction seem to be missing. Mitz, can you help? fast/dom/dom-constructors.html — This test needs a result for GTK that indicate lack of an audio element or it needs to be in the skipped list. dom — There are many failures in the GTK bot due to a full URL rather than partial URL showing up somewhere that causes the DOM test machinery to report errors that include full paths. These should be fixed or added to the GTK skipped list. fast/dom/Window/timeout-released-on-close.html — Failing on GTK, not sure why. fast/encoding/char-decoding-mac.html — Needs custom results or a skipped list since the GTK build does not support these encodings. fast/events/special-key-events-in-input-text.html — Failing on GTK due to lack of eventSender. Needs to be added to the skipped list. fast/forms/textarea-selection-preservation.html — Failing on GTK, not sure why. A recent regression? fast/loader/plain-text-document.html, fast/xsl/xslt-text.html — Failing on GTK, not sure why the frame name is being generated with the text someFloatString in it. ecma/String/15.5.4.7-1.js ecma/String/15.5.4.7-2.js ecma/String/15.5.4.7-3.js ecma_2/String/match-002.js js1_5/String/regress-107771.js — Failing on GTK. Is lastIndexOf really broken on GTK? I'm quite surprised to see platform-specific failures in the JavaScriptCore tests. Anyone have any insight? And some intermittent failures: http/tests/webarchive/test-preload-resources.html http/tests/webarchive/test-css-url-encoding-shift-jis.html http/tests/webarchive/test-css-url-encoding-utf-8.html http/tests/webarchive/test-css-url-encoding.html — Various web archive tests seem to be intermittently failing on Mac because of connections that close. The results of these tests seem to depend on keep alive and our test HTTP run of Apache doesn't seem to work consistently. http/tests/appcache/offline-access.html — Seems to have timed out once on Mac. fast/dom/Window/timeout-released-on-close.html — Seems to have failed once on Mac. ecma_3/Date/15.9.5.6.js ecma_3/Date/15.9.5.7.js — These tests seem to toggle between success and failure on the GTK and WX bots based perhaps on time of day or time zone? This is particularly troublesome because we don't have a skipped list for JavaScript tests. Maybe we can tweak the script we use to run these tests to ignore these intermittent failures? And also a lot of slaves seem to be down. There are lots of 38 pending on the waterfall, and I am not sure why. It seems strange that the Chromium slaves would have disappeared at the same time as the Mac Intel and Windows slaves. I don't know where the Mac Intel and Windows slaves are located, but the Chromium ones have been gone since a power line got knocked down in Mountain View late on December 23 (PST). I rebooted the machines on the 29th, but apparently they're not set up to rejoin on reboot, and I didn't have the login information. Anyway, I'm sure they'll be back once everyone's back at work tomorrow. - Pam I'd love to set up the skipped lists properly and fix enough bugs so that all our bots are green. Can you help? Maybe there are already bug reports about some of these. -- Darin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
[webkit-dev] New layout tests from Chromium
Part of our effort to move to WebKit tip-of-tree and participate fully in the WebKit community is to contribute the nearly 80 new layout tests we wrote before Google Chrome and Chromium were public. You can see our progress here: http://tinyurl.com/cr-tests or http://spreadsheets.google.com/ccc?key=piGkUUMLW-PNUzuhTCzm4NQ Thanks for your patience with reviews and checkins as we work through the list and improve our collective layout-test-writing skills. (If you're interested in helping out, please feel free to sign up for a test or two. Unlike many Chromium developers, most WebKit developers both have Macs and know about the current layout-test conventions and style. :) - Pam ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
