Re: [webkit-dev] Layout Tests and the W3C
From: Maciej Stachowiak m...@apple.commailto:m...@apple.com Date: Mon, 15 Apr 2013 18:29:43 -0700 To: Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org Cc: WebKit-Dev webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] Layout Tests and the W3C On Apr 15, 2013, at 5:31 PM, Dirk Pranke dpra...@chromium.orgmailto:dpra...@chromium.org wrote: Hi all, Those of you who are subscribed to blink-dev@ will see that I just sent out a note entitled Blink, Testing and the W3C, describing stuff I'm working on to get the W3C tests running regularly as part of the layout tests. The W3C has gotten to the point where they have many thousands of ref tests, a lot of which are newer versions of old test suites that we imported once upon a time (meaning that we can replace a lot of pixel tests with ref tests). That work leverages a bunch of work Rebecca Hauck has been working on that actually started pre-Blink-fork, and in fact currently only exists as a patch posted to bugs.webkit.orghttp://bugs.webkit.org/ ( https://bugs.webkit.org/show_bug.cgi?id=111513 ). I would like to get something resembling the same process working for both WebKit and Blink, since we use basically the same tools. There will probably be some differences, of course, e.g., in how we mirror repos and track issues, but I would hope we'd all agree the more tests we can share (both between WebKit and Blink and with other vendors like FF and IE) the happier everyone will be, right? We haven't really worked out the processes for how this'll work yet; Rebecca and I were starting to work on it when the fork happened. Assuming there is still interest, I'll start sketching out how things might work for WebKit and report back. Also, if we actually start to make progress on this, I'd be happy to talk about it at the WebKit committer's meeting if there's interest there as well. I think it would be great to run W3C reftests as part of WebKit's layout tests. Extra awesome would be to have the setup to import new versions of test suites and/or new test suites regularly. I would welcome such work and I expect others would as well. Yes! I think it should go in a few phases – 1) work out the mechanics of the import, 2) identify import the first suite of tests manually (probably the most stable suite – CSS2.1 for example), 3) come up with a strategy for refreshing and importing new tests (could be partly considered in the mechanics of the import script), 4) turn on the script to automatically import/refresh once all the wrinkles from 1-3 are sorted out. I also think this would make a great contributor's meeting topic, either as a regular session or a hackathon. Just added this to the wiki (thanks to Alan Stearns – it wasn't cooperating with me when I tried). I'm happy to do an overview of the issues, what little bit of work I've started on this, and open it up to discussion on how to proceed. One thing I would hope for is if we are able to request time slots at all – if we can have this in an early-ish slot on Thursday, that'd be great. I won't be able to attend on Friday. Cheers, -Rebecca ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] New script to import W3C CSS tests
Hi Dirk Everyone, Dirk, your question about this in IRC the other day was timely indeed. I've recently written a batch of tests for CSS Regions that I intend to submit to both the W3C and Webkit (and have planned more to come). Rather than push two slightly different versions of the same tests to each repo, and to address how these tests are to be maintained over time, I took a stab automating the import and just submitted a patch [1]. I know this conversation was started a while ago as I attended the WK Contributor's meeting last spring where my colleagues Jacob Goldstein and Alan Stearns led the discussion. I was sort of new onto the scene then and was pretty much a lurker in the meeting, so you probably don't remember me. I recall that much of the debate was about handling failing tests, approved vs. submitted, duplicates, etc. Since all I've done is automate the mechanics of the import itself, I have not solved a lot of those issues, but the conversion of tests certainly had its own set of things to consider. I figured I would put this one piece in place, get feedback on it, and hopefully resurrect the discussion on fitting this into the bigger picture. A great goal would be to have this script run on an automated schedule and to extend it to deal with test results, expected failures, etc. All of the dirty details are in the ChangeLog of the patch and in the script headers, but I basically tried to cover #8 from last spring's meeting notes [2]. I think the script covers all of those bullets, with that last bullet as sort of. I also got a little inspiration from the import-w3c-performance-wg-tests that already exists. I followed a few of their steps, but had to add a few layers to handle the added complexity of the CSS test suites. Dirk, when you have a chance, take a look. Review it with gusto as you say. And anyone else who's interested, your feedback is welcome. I fairly certain this will go a few rounds before it's settled. :) Cheers, -Rebecca [1] https://bugs.webkit.org/show_bug.cgi?id=111513 [2] http://trac.webkit.org/wiki/ImportingThirdPartyTests ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] New script to import W3C CSS tests
On 3/7/13 3:26 PM, Dirk Pranke dpra...@chromium.org wrote: On Thu, Mar 7, 2013 at 3:04 PM, Tony Gentilcore to...@chromium.org wrote: I also got a little inspiration from the import-w3c-performance-wg-tests that already exists. I followed a few of their steps, but had to add a few layers to handle the added complexity of the CSS test suites. Is there any way we can merge the two scripts so there is only one import/exporter than handles both performance and CSS test suites? I thought about it. There are only a couple similar pieces between the two scripts, but since the perf importer is relatively short, there's no reason why the code couldn't be consolidated into one module with separate execution paths. I figured I'd start this out atomically and then figure out how it can all plug in. That is a good question, and something I'll be looking at as part of this :). More generally, I'm trying to look into importing and running tests from the w3c across the board as one of my next projects. Rebecca beat me to it! -- Dirk -Tony ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Updating webkit.org with Xcode 4.3 for Lion instructions
Hi, A few of us have recently discovered that Xcode 4.3 for Lion (installed now through the App Store rather than from a dmg) does not include by default the command line tools required by the webkit build scripts. I've logged a bug and submitted a patch to update the 'Installing Developer Tools' section of webkit.org here: http://www.webkit.org/building/tools.html Here's the bug: https://bugs.webkit.org/show_bug.cgi?id=80429 Can I get a review? Thanks! -Rebecca ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
