I've thought about this more. 1. I've been making too much of the unit tests requiring an IE security setting. I think that the first failure actually tells you how to correct this (and if not we could make it so), so this is a self correcting problem.
2. The areas the watir/firewatir test suite are strong are around windows, attach, dialogs, frames -- all the areas where Watir is more fully featured than tools like Selenium (webdriver) or HTMLUnit (Celerity). So this is why I am attached to them. Jarmo, I refactored a lot of the unit testing framework into commonwatir. I am curious what your plan is to make them work in the gems? I suppose one possibility would be for the gem build process to copy this common code into each of the gems' unit test directories. This would avoid source duplication while allow gem independence. Bret On Mon, Sep 27, 2010 at 5:41 PM, Bret Pettichord <[email protected]>wrote: > On Mon, Sep 27, 2010 at 5:06 PM, Jarmo <[email protected]> wrote: > >> On Tue, Sep 28, 2010 at 12:57 AM, Bret Pettichord <[email protected]> >> wrote: >> >> It shouldn't be harder to run tests than to actually use the >> >> lib thus having tests at some separate place and setting them >> >> (separately) up isn't good option either. >> > >> > Right, but since you have to change the security settings for IE to get >> the >> > test (only) to run, we break this rule no matter what. >> >> Ok, but if user wants to use Watir/FireWatir then they have to in the >> end change their browser settings anyway nevertheless. Why not make it >> part of an installation process in some documentation to avoid many of >> those browser misconfiguration problems in the first place? In my view >> it isn't different if user wants to use the library itself or wants to >> run the tests (which are also using the library itself) because he/she >> needs to make some configuration adjustments anyway in one way or >> another. >> > > I don't believe this is true. Watir works with IE with no configuration > changes. You only have to dig into the security settings if you want to run > the unit tests. > (Firefox is a different story). > > > The problem in the end is that our unit tests aren't really unit tests, > and > > can't really meet all of these rules. The question, then, is which rules > > should we break? > > > > Maybe the solution would be to write some true unit tests and then > package > > them with the gem? These would be tests that wouldn't actually fire up a > > browser, though, so although they would meet all of your rules, they > would > > perhaps be less valuable. > > Agreed on the less valuable part. I still think that if user can run >> unit-tests (or not THE unit tests, but the currently available tests) >> then that means that his system is configured properly to use actual >> library itself. There's no other point to have tests with the gem. If >> there are runnable unit tests then we can always point users to run >> them if there's some strange problems. >> > > This actually argues for us packaging a set of tests with the gem that do > not require the IE security setting changes. I see two approaches. > 1. Use local files like the current unit tests, but avoid javascript. > 2. Test against a public webserver. > > > >> Also, why do you think that Watirspec is a subset of Watir/FireWatir > >> tests? I can see it as a pretty good replacement of current tests. > >> Yeah, there are few functionalities which aren't tested by Watirspec > >> like #attach and #click_no_wait, > > > > Watirspec was created for Celerity. Therefore, I have assumed that it > only > > included tests for features that are supported in Celerity. This means it > > omits support for such features as these. > > If you look at Watir-Webdriver, which uses also Watirspec then there >> isn't that much of an API difference between Watir and (almost?) all >> of the public API is covered with Watirspec. You may think of it as >> all public methods are tested. >> > > I think that Watirspec is a good test of the DOM-API for Watir. It is the > other parts of the Watir API that are not covered by it. I think we agree on > this. > > > > I was also able to use the Watir test suites to find a number of > > discrepancies between Watir and FireWatir. I don't believe the Watirspec > > suite can do this. It could, but it would need quite a bit of work first. > > Why not? Just run Watirspec against Watir and FireWatir - i'm pretty >> sure there will be different failures... >> > > I'd like you to run the Watir unit tests in compatibility mode against each > of them to see what I am talking about. Once you've done that, I think you > would have a good understanding of their value. (And if you were > unimpressed, then I'd be inclined to drop the issue). > > -- > Bret Pettichord > Lead Developer, Watir, www.watir.com > > Blog, www.io.com/~wazmo/blog <http://www.io.com/%7Ewazmo/blog> > Twitter, www.twitter.com/bpettichord > > -- Bret Pettichord Lead Developer, Watir, www.watir.com Blog, www.io.com/~wazmo/blog Twitter, www.twitter.com/bpettichord
_______________________________________________ Wtr-development mailing list [email protected] http://rubyforge.org/mailman/listinfo/wtr-development
