Re: [webkit-dev] C++ unit tests for WebKit?
On Wed, Jul 11, 2012 at 2:23 PM, Hans Muller hmul...@adobe.com wrote: Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's thoughts. That has been discussed before. We have some C++ tests, see Tools/TestWebKitAPI/Tests/ You can probably integrate yours in there. Cheers, Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] C++ unit tests for WebKit?
Don't we have gunit tests in Tools/TestWebKitAPI already? From: webkit-dev-boun...@lists.webkit.org [webkit-dev-boun...@lists.webkit.org] on behalf of Hans Muller [hmul...@adobe.com] Sent: Wednesday, July 11, 2012 5:23 PM To: webkit-dev Subject: [webkit-dev] C++ unit tests for WebKit? Bear Travis, Alexandru Chiculita, and I have been working on the WebKit CSS Exclusions implementation. I've been working on a set of C++ classes that are used to compute the layout of inline content around or within exclusion shapes. When I got started, before attempting to integrate the code with WebKit, I wrote a pile of unit tests. This was helpful for the usual reasons but there doesn't appear to be a place for low level unit tests in WebKit. Naturally what I've done will be tested in terms of the Exclusions public JavaScript and CSS APIs, so adding lower level unit tests will not drastically change our confidence about the overall feature working correctly. On the other hand, unit tests can be useful for isolating problems and for implicitly specifying interface semantics that wouldn't otherwise be documented. Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's thoughts. Thanks, - Hans - This transmission (including any attachments) may contain confidential information, privileged material (including material protected by the solicitor-client or other applicable privileges), or constitute non-public information. Any use of this information by anyone other than the intended recipient is prohibited. If you have received this transmission in error, please immediately reply to the sender and delete this information from your system. Use, dissemination, distribution, or reproduction of this transmission by unintended recipients is not authorized and may be unlawful. ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] C++ unit tests for WebKit?
It looks like Tools/TestWebKitAPI/Tests/WebCore just contains one small KURL test. I could certainly add more for the Exclusions shape classes, but I got that feeling that maybe you get about being the first to take a table at an empty restaurant. Perhaps it's just that most of the C++ classes in WebCore are too interdependent to merit unit testing? - Hans On 7/11/12 2:26 PM, Benjamin Poulain benja...@webkit.org wrote: On Wed, Jul 11, 2012 at 2:23 PM, Hans Muller hmul...@adobe.com wrote: Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's thoughts. That has been discussed before. We have some C++ tests, see Tools/TestWebKitAPI/Tests/ You can probably integrate yours in there. Cheers, Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] C++ unit tests for WebKit?
Generally speaking, WebKit's testing philosophy is to test at API boundaries, typically either a given port's WebKit API or the web platform API. The benefit of that approach is that it makes it easier for us to refactor the internals of WebCore without being constrained by fragile tests---only by API commitments. In the case of CSS Exclusions, it's likely valuable to think about how you can test the feature via the web platform. Developing a rich set of tests that use web platform APIs has a number of follow-on benefits that WebCore-internal testing doesn't, such as the ability to run the tests in other browsers and to contribute them to the W3C as part of the standards process. Of course, this sort of one-size-fits-all testing strategy works well in come cases in less well in others. I understand that other projects have different testing strategies that have worked well for them, but I would encourage you to try our the community's conventions and customs for a while to get a feel for which ones work well and which ones ought to be improved. Adam On Wed, Jul 11, 2012 at 3:44 PM, Hans Muller hmul...@adobe.com wrote: It looks like Tools/TestWebKitAPI/Tests/WebCore just contains one small KURL test. I could certainly add more for the Exclusions shape classes, but I got that feeling that maybe you get about being the first to take a table at an empty restaurant. Perhaps it's just that most of the C++ classes in WebCore are too interdependent to merit unit testing? - Hans On 7/11/12 2:26 PM, Benjamin Poulain benja...@webkit.org wrote: On Wed, Jul 11, 2012 at 2:23 PM, Hans Muller hmul...@adobe.com wrote: Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's thoughts. That has been discussed before. We have some C++ tests, see Tools/TestWebKitAPI/Tests/ You can probably integrate yours in there. Cheers, Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] C++ unit tests for WebKit?
Thanks for the thoughtful reply. It was exactly the community's conventions and customs that I was trying to get a handle on. Not to put too fine a point on it, but I assume that adding unit tests to TestWebKitAPI or writing tests that depend on APIs defined in Internals.idl wouldn't be customary or conventional?I'd been thinking about doing the latter (Internals.idl), just because it would enable testing some low level classes before everything we're working on had been integrated top to bottom. Doing so could just be a private temporary change, since eventually it will be possible to exercise the code via public APIs. - Hans From: Adam Barth aba...@webkit.orgmailto:aba...@webkit.org To: Hans Muller hmul...@adobe.commailto:hmul...@adobe.com Cc: Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org, Konrad Piascik kpias...@rim.commailto:kpias...@rim.com, webkit-dev webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] C++ unit tests for WebKit? Generally speaking, WebKit's testing philosophy is to test at API boundaries, typically either a given port's WebKit API or the web platform API. The benefit of that approach is that it makes it easier for us to refactor the internals of WebCore without being constrained by fragile tests---only by API commitments. In the case of CSS Exclusions, it's likely valuable to think about how you can test the feature via the web platform. Developing a rich set of tests that use web platform APIs has a number of follow-on benefits that WebCore-internal testing doesn't, such as the ability to run the tests in other browsers and to contribute them to the W3C as part of the standards process. Of course, this sort of one-size-fits-all testing strategy works well in come cases in less well in others. I understand that other projects have different testing strategies that have worked well for them, but I would encourage you to try our the community's conventions and customs for a while to get a feel for which ones work well and which ones ought to be improved. Adam On Wed, Jul 11, 2012 at 3:44 PM, Hans Muller hmul...@adobe.commailto:hmul...@adobe.com wrote: It looks like Tools/TestWebKitAPI/Tests/WebCore just contains one small KURL test. I could certainly add more for the Exclusions shape classes, but I got that feeling that maybe you get about being the first to take a table at an empty restaurant. Perhaps it's just that most of the C++ classes in WebCore are too interdependent to merit unit testing? - Hans On 7/11/12 2:26 PM, Benjamin Poulain benja...@webkit.orgmailto:benja...@webkit.org wrote: On Wed, Jul 11, 2012 at 2:23 PM, Hans Muller hmul...@adobe.commailto:hmul...@adobe.com wrote: Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's thoughts. That has been discussed before. We have some C++ tests, see Tools/TestWebKitAPI/Tests/ You can probably integrate yours in there. Cheers, Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.orgmailto:webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] C++ unit tests for WebKit?
On Wed, Jul 11, 2012 at 4:31 PM, Hans Muller hmul...@adobe.com wrote: Thanks for the thoughtful reply. It was exactly the community's conventions and customs that I was trying to get a handle on. Not to put too fine a point on it, but I assume that adding unit tests to TestWebKitAPI or writing tests that depend on APIs defined in Internals.idl wouldn't be customary or conventional? As the name implies, TestWebKitAPI is mostly about testing the WebKit API (meaning the API that WebKit exposes to folks who use it as a library). Your feature doesn't seem to involve any WebKit API, so I would steer clear of TestWebKitAPI. We're still feeling out when it's appropriate to test something via Internals. One good use of Internals is to set a WebCore::Setting directly rather than having to create API for N different ports. There's a risk that Internals will become a dumping group for random things, and this is something we try to look out for when reviewing changes to Internals.idl. I'd been thinking about doing the latter (Internals.idl), just because it would enable testing some low level classes before everything we're working on had been integrated top to bottom. Doing so could just be a private temporary change, since eventually it will be possible to exercise the code via public APIs. Obviously, you should feel free to do whatever is convenient in your own private branch. For development on trunk, I would encourage you to try to get a basic version of your feature working end-to-end (so that it's testable via the web platform) and then to elaborate it incrementally. That's not always possible, but it tends to be a good approach for generating test coverage. I'm happy to discuss this topic further in #webkit if that would be helpful to you. Adam From: Adam Barth aba...@webkit.org To: Hans Muller hmul...@adobe.com Cc: Benjamin Poulain benja...@webkit.org, Konrad Piascik kpias...@rim.com, webkit-dev webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] C++ unit tests for WebKit? Generally speaking, WebKit's testing philosophy is to test at API boundaries, typically either a given port's WebKit API or the web platform API. The benefit of that approach is that it makes it easier for us to refactor the internals of WebCore without being constrained by fragile tests---only by API commitments. In the case of CSS Exclusions, it's likely valuable to think about how you can test the feature via the web platform. Developing a rich set of tests that use web platform APIs has a number of follow-on benefits that WebCore-internal testing doesn't, such as the ability to run the tests in other browsers and to contribute them to the W3C as part of the standards process. Of course, this sort of one-size-fits-all testing strategy works well in come cases in less well in others. I understand that other projects have different testing strategies that have worked well for them, but I would encourage you to try our the community's conventions and customs for a while to get a feel for which ones work well and which ones ought to be improved. Adam On Wed, Jul 11, 2012 at 3:44 PM, Hans Muller hmul...@adobe.com wrote: It looks like Tools/TestWebKitAPI/Tests/WebCore just contains one small KURL test. I could certainly add more for the Exclusions shape classes, but I got that feeling that maybe you get about being the first to take a table at an empty restaurant. Perhaps it's just that most of the C++ classes in WebCore are too interdependent to merit unit testing? - Hans On 7/11/12 2:26 PM, Benjamin Poulain benja...@webkit.org wrote: On Wed, Jul 11, 2012 at 2:23 PM, Hans Muller hmul...@adobe.com wrote: Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's thoughts. That has been discussed before. We have some C++ tests, see Tools/TestWebKitAPI/Tests/ You can probably integrate yours in there. Cheers, Benjamin ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org http://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] C++ unit tests for WebKit?
(Resending from the right address) FYI, Chromium port has some unit-like tests: http://trac.webkit.org/browser/trunk/Source/WebKit/chromium/tests Apparently it is not community's responsibility to maintain these. But it might be useful for developing some low-level standalone/low-level stuff like collections and utilities if you are developing on Chromium port. My feeling is that the lack of unit test is mixture of philosophical decisions and technical limitations. As Adam mentioned, Layout Tests for Web Platform features fit former, and untested low level stuff fits later. If we have some low-level/primitive component for building larger web platform features, such primitives are worth unit-testing IMHO. And TestWebKitAPI (or cr-port test in some case) is good place for them. On Thu, Jul 12, 2012 at 8:48 AM, Adam Barth aba...@webkit.org wrote: On Wed, Jul 11, 2012 at 4:31 PM, Hans Muller hmul...@adobe.com wrote: Thanks for the thoughtful reply. It was exactly the community's conventions and customs that I was trying to get a handle on. Not to put too fine a point on it, but I assume that adding unit tests to TestWebKitAPI or writing tests that depend on APIs defined in Internals.idl wouldn't be customary or conventional? As the name implies, TestWebKitAPI is mostly about testing the WebKit API (meaning the API that WebKit exposes to folks who use it as a library). Your feature doesn't seem to involve any WebKit API, so I would steer clear of TestWebKitAPI. We're still feeling out when it's appropriate to test something via Internals. One good use of Internals is to set a WebCore::Setting directly rather than having to create API for N different ports. There's a risk that Internals will become a dumping group for random things, and this is something we try to look out for when reviewing changes to Internals.idl. I'd been thinking about doing the latter (Internals.idl), just because it would enable testing some low level classes before everything we're working on had been integrated top to bottom. Doing so could just be a private temporary change, since eventually it will be possible to exercise the code via public APIs. Obviously, you should feel free to do whatever is convenient in your own private branch. For development on trunk, I would encourage you to try to get a basic version of your feature working end-to-end (so that it's testable via the web platform) and then to elaborate it incrementally. That's not always possible, but it tends to be a good approach for generating test coverage. I'm happy to discuss this topic further in #webkit if that would be helpful to you. Adam From: Adam Barth aba...@webkit.org To: Hans Muller hmul...@adobe.com Cc: Benjamin Poulain benja...@webkit.org, Konrad Piascik kpias...@rim.com, webkit-dev webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] C++ unit tests for WebKit? Generally speaking, WebKit's testing philosophy is to test at API boundaries, typically either a given port's WebKit API or the web platform API. The benefit of that approach is that it makes it easier for us to refactor the internals of WebCore without being constrained by fragile tests---only by API commitments. In the case of CSS Exclusions, it's likely valuable to think about how you can test the feature via the web platform. Developing a rich set of tests that use web platform APIs has a number of follow-on benefits that WebCore-internal testing doesn't, such as the ability to run the tests in other browsers and to contribute them to the W3C as part of the standards process. Of course, this sort of one-size-fits-all testing strategy works well in come cases in less well in others. I understand that other projects have different testing strategies that have worked well for them, but I would encourage you to try our the community's conventions and customs for a while to get a feel for which ones work well and which ones ought to be improved. Adam On Wed, Jul 11, 2012 at 3:44 PM, Hans Muller hmul...@adobe.com wrote: It looks like Tools/TestWebKitAPI/Tests/WebCore just contains one small KURL test. I could certainly add more for the Exclusions shape classes, but I got that feeling that maybe you get about being the first to take a table at an empty restaurant. Perhaps it's just that most of the C++ classes in WebCore are too interdependent to merit unit testing? - Hans On 7/11/12 2:26 PM, Benjamin Poulain benja...@webkit.org wrote: On Wed, Jul 11, 2012 at 2:23 PM, Hans Muller hmul...@adobe.com wrote: Have the merits of C++ unit tests been debated before? I noticed that a bug representing as much has been around since 2008 (https://bugs.webkit.org/show_bug.cgi?id=21010). The last comment (2009) concludes with: I'd be interested to hear other's thoughts. Personally, I'd like to be able to include unit tests with worthy C++ classes, however I would also like to hear other's
