[chromium-dev] Re: running layout_tests on vista and windows 7
(This time using a reply-all) We used to do this for things like faking font metrics. It resulted in lots of "if (webkit_glue::IsLayoutTestMode())" code. We moved away from it because it made it made the code more complicated and meant we weren't shipping the code we were testing. I've heard it proposed before that instead of adding code to check for layout test mode, that maybe we can intercept the system calls and return controlled values (maybe the XP theme?) when in layout test mode. I'm not sure how feasible this is on Windows. On Mon, Jul 13, 2009 at 12:50 PM, Dirk Pranke wrote: > > Hi all, > > (If you don't ever care to run the webkit layout tests, you can skip this > note). > > As most of you are no doubt aware, we currently can only run the > webkit layout_tests on windows XP. For some of us who primarily > develop on 64-bit Vista, this is inconvenient at best, and this is > only going to get worse over time as more of migrate to 64-bit > machines and (eventually) Windows 7. > > So, I'm working on porting the layout tests to Vista. This note is a > writeup of the approach I'm thinking of taking, and I'm looking for > feedback and suggestions, especially since most of you have been on > this code base a lot longer than me. > > My basic approach is to try and get something up as quickly as > possible as a proof of concept, and then work to try and reduce the > maintenance over time. So, I've started by cloning the chromium-win > port over to vista, and modifying the test scripts to be aware of the > new set of test expectations. I will then tweak the tests to get > roughly the same list of tests passing on Vista as on Windows. The > main differences will have to do with how the theming renders scroll > bars and a few other native controls. I have most of this now, and > should have the rest of this in a day or two, but this is not a > maintainable solution without a lot of manual overhead. > > Next, we'll get a buildbot setup to run on Vista. > > While we're doing this, I'll start working on reducing the test set > duplication between XP and Vista. The best way to do this (we > currently think) will be to modify test_shell to *not* draw the native > controls, but rather stub them out in a platform-independent way for > test purposes (e.g., just painting a grey box instead of a real scroll > bar). Then we can write a few platform-specific unit tests to ensure > that the widgets do work correctly, but the bulk of the tests will > become (more) platform-independent. My hope is that we'll have > something that I can demonstrate here in a week or two, and that it > will extend trivially to Win 7. > > A stretch hope is that we can even get the rendering to be > platform-independent enough that we may even be able to leverage them > across the linux and mac ports. I don't know if this is realistic or > not, as many of the tests may differ just due to font rendering and > other minor differences. > > An alternative strategy is to start looking at more and more of the > tests and making sure they are written to be as platform-independent > as possible. First we'd this by making sure that we don't rely on > pixel-based tests where text-based tests would do. Another option > would be to switch to writing two tests just to ensure that page A > renders the same way as page B (where A and B use two different sets > of layout but should produce the same output). Both of these options > are significantly more work up front, but will payoff in much less > maintenance down the line. Also, all of this work will also overlap > with the webkit test suites, so it'll need to be coordinated with our > upstream buddies. > > Comments? Thoughts? > > -- Dirk > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: running layout_tests on vista and windows 7
On Mon, Jul 13, 2009 at 1:16 PM, Dirk Pranke wrote: > > Yup, I've already adopted that. Thanks! > > On Mon, Jul 13, 2009 at 12:55 PM, Thomas Van > Lenten wrote: > > Quick skimmed reply: Mac already has expectations per OS where we need > them, > > so you might be able to follow that basic model (and maybe small tweaks > to > > the scripts to use it). It looks for version specific and then falls > back > > to generic platform files so we only have to dup the ones that are os > > version specific. > > TVL > > > > On Mon, Jul 13, 2009 at 3:50 PM, Dirk Pranke wrote: > >> > >> Hi all, > >> > >> (If you don't ever care to run the webkit layout tests, you can skip > this > >> note). > >> > >> As most of you are no doubt aware, we currently can only run the > >> webkit layout_tests on windows XP. For some of us who primarily > >> develop on 64-bit Vista, this is inconvenient at best, and this is > >> only going to get worse over time as more of migrate to 64-bit > >> machines and (eventually) Windows 7. > >> > >> So, I'm working on porting the layout tests to Vista. This note is a > >> writeup of the approach I'm thinking of taking, and I'm looking for > >> feedback and suggestions, especially since most of you have been on > >> this code base a lot longer than me. > >> > >> My basic approach is to try and get something up as quickly as > >> possible as a proof of concept, and then work to try and reduce the > >> maintenance over time. So, I've started by cloning the chromium-win > >> port over to vista, and modifying the test scripts to be aware of the > >> new set of test expectations. I will then tweak the tests to get > >> roughly the same list of tests passing on Vista as on Windows. The > >> main differences will have to do with how the theming renders scroll > >> bars and a few other native controls. I have most of this now, and > >> should have the rest of this in a day or two, but this is not a > >> maintainable solution without a lot of manual overhead. > >> > >> Next, we'll get a buildbot setup to run on Vista. > >> > >> While we're doing this, I'll start working on reducing the test set > >> duplication between XP and Vista. The best way to do this (we > >> currently think) will be to modify test_shell to *not* draw the native > >> controls, but rather stub them out in a platform-independent way for > >> test purposes (e.g., just painting a grey box instead of a real scroll > >> bar). Then we can write a few platform-specific unit tests to ensure > >> that the widgets do work correctly, but the bulk of the tests will > >> become (more) platform-independent. My hope is that we'll have > >> something that I can demonstrate here in a week or two, and that it > >> will extend trivially to Win 7. > >> > >> A stretch hope is that we can even get the rendering to be > >> platform-independent enough that we may even be able to leverage them > >> across the linux and mac ports. I don't know if this is realistic or > >> not, as many of the tests may differ just due to font rendering and > >> other minor differences. > >> > >> An alternative strategy is to start looking at more and more of the > >> tests and making sure they are written to be as platform-independent > >> as possible. First we'd this by making sure that we don't rely on > >> pixel-based tests where text-based tests would do. > We started doing this for all of the editing tests, but got pushback from WebKit, because they were concerned that we may inadvertantly lose some of the subtle things that a pixel test can test that a text based test can't. I still think it is a good idea (less platform specific results to check out, tests run faster, etc), but just an FYI that you might hit some pushback in this area, so be sure to run it by webkit-dev before doing too much work. > Another option > >> would be to switch to writing two tests just to ensure that page A > >> renders the same way as page B (where A and B use two different sets > >> of layout but should produce the same output). Both of these options > >> are significantly more work up front, but will payoff in much less > >> maintenance down the line. Also, all of this work will also overlap > >> with the webkit test suites, so it'll need to be coordinated with our > >> upstream buddies. > >> > >> Comments? Thoughts? > >> > >> -- Dirk > >> > >> >> > > > > > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: running layout_tests on vista and windows 7
Yup, I've already adopted that. Thanks! On Mon, Jul 13, 2009 at 12:55 PM, Thomas Van Lenten wrote: > Quick skimmed reply: Mac already has expectations per OS where we need them, > so you might be able to follow that basic model (and maybe small tweaks to > the scripts to use it). It looks for version specific and then falls back > to generic platform files so we only have to dup the ones that are os > version specific. > TVL > > On Mon, Jul 13, 2009 at 3:50 PM, Dirk Pranke wrote: >> >> Hi all, >> >> (If you don't ever care to run the webkit layout tests, you can skip this >> note). >> >> As most of you are no doubt aware, we currently can only run the >> webkit layout_tests on windows XP. For some of us who primarily >> develop on 64-bit Vista, this is inconvenient at best, and this is >> only going to get worse over time as more of migrate to 64-bit >> machines and (eventually) Windows 7. >> >> So, I'm working on porting the layout tests to Vista. This note is a >> writeup of the approach I'm thinking of taking, and I'm looking for >> feedback and suggestions, especially since most of you have been on >> this code base a lot longer than me. >> >> My basic approach is to try and get something up as quickly as >> possible as a proof of concept, and then work to try and reduce the >> maintenance over time. So, I've started by cloning the chromium-win >> port over to vista, and modifying the test scripts to be aware of the >> new set of test expectations. I will then tweak the tests to get >> roughly the same list of tests passing on Vista as on Windows. The >> main differences will have to do with how the theming renders scroll >> bars and a few other native controls. I have most of this now, and >> should have the rest of this in a day or two, but this is not a >> maintainable solution without a lot of manual overhead. >> >> Next, we'll get a buildbot setup to run on Vista. >> >> While we're doing this, I'll start working on reducing the test set >> duplication between XP and Vista. The best way to do this (we >> currently think) will be to modify test_shell to *not* draw the native >> controls, but rather stub them out in a platform-independent way for >> test purposes (e.g., just painting a grey box instead of a real scroll >> bar). Then we can write a few platform-specific unit tests to ensure >> that the widgets do work correctly, but the bulk of the tests will >> become (more) platform-independent. My hope is that we'll have >> something that I can demonstrate here in a week or two, and that it >> will extend trivially to Win 7. >> >> A stretch hope is that we can even get the rendering to be >> platform-independent enough that we may even be able to leverage them >> across the linux and mac ports. I don't know if this is realistic or >> not, as many of the tests may differ just due to font rendering and >> other minor differences. >> >> An alternative strategy is to start looking at more and more of the >> tests and making sure they are written to be as platform-independent >> as possible. First we'd this by making sure that we don't rely on >> pixel-based tests where text-based tests would do. Another option >> would be to switch to writing two tests just to ensure that page A >> renders the same way as page B (where A and B use two different sets >> of layout but should produce the same output). Both of these options >> are significantly more work up front, but will payoff in much less >> maintenance down the line. Also, all of this work will also overlap >> with the webkit test suites, so it'll need to be coordinated with our >> upstream buddies. >> >> Comments? Thoughts? >> >> -- Dirk >> >> >> > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: running layout_tests on vista and windows 7
Quick skimmed reply: Mac already has expectations per OS where we need them, so you might be able to follow that basic model (and maybe small tweaks to the scripts to use it). It looks for version specific and then falls back to generic platform files so we only have to dup the ones that are os version specific. TVL On Mon, Jul 13, 2009 at 3:50 PM, Dirk Pranke wrote: > > Hi all, > > (If you don't ever care to run the webkit layout tests, you can skip this > note). > > As most of you are no doubt aware, we currently can only run the > webkit layout_tests on windows XP. For some of us who primarily > develop on 64-bit Vista, this is inconvenient at best, and this is > only going to get worse over time as more of migrate to 64-bit > machines and (eventually) Windows 7. > > So, I'm working on porting the layout tests to Vista. This note is a > writeup of the approach I'm thinking of taking, and I'm looking for > feedback and suggestions, especially since most of you have been on > this code base a lot longer than me. > > My basic approach is to try and get something up as quickly as > possible as a proof of concept, and then work to try and reduce the > maintenance over time. So, I've started by cloning the chromium-win > port over to vista, and modifying the test scripts to be aware of the > new set of test expectations. I will then tweak the tests to get > roughly the same list of tests passing on Vista as on Windows. The > main differences will have to do with how the theming renders scroll > bars and a few other native controls. I have most of this now, and > should have the rest of this in a day or two, but this is not a > maintainable solution without a lot of manual overhead. > > Next, we'll get a buildbot setup to run on Vista. > > While we're doing this, I'll start working on reducing the test set > duplication between XP and Vista. The best way to do this (we > currently think) will be to modify test_shell to *not* draw the native > controls, but rather stub them out in a platform-independent way for > test purposes (e.g., just painting a grey box instead of a real scroll > bar). Then we can write a few platform-specific unit tests to ensure > that the widgets do work correctly, but the bulk of the tests will > become (more) platform-independent. My hope is that we'll have > something that I can demonstrate here in a week or two, and that it > will extend trivially to Win 7. > > A stretch hope is that we can even get the rendering to be > platform-independent enough that we may even be able to leverage them > across the linux and mac ports. I don't know if this is realistic or > not, as many of the tests may differ just due to font rendering and > other minor differences. > > An alternative strategy is to start looking at more and more of the > tests and making sure they are written to be as platform-independent > as possible. First we'd this by making sure that we don't rely on > pixel-based tests where text-based tests would do. Another option > would be to switch to writing two tests just to ensure that page A > renders the same way as page B (where A and B use two different sets > of layout but should produce the same output). Both of these options > are significantly more work up front, but will payoff in much less > maintenance down the line. Also, all of this work will also overlap > with the webkit test suites, so it'll need to be coordinated with our > upstream buddies. > > Comments? Thoughts? > > -- Dirk > > > > --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
