On Mon, Sep 27, 2010 at 4:21 PM, Jarmo <[email protected]> wrote: > Maybe there just needs to be better documentation about running unit tests > then. >
True. I have one argument for having unit tests with the gem - if i install > a gem then i usually check if it has any tests with it and if there > isn't any tests then i will just delete the gem, no questions asked. > I'm not going to search where the source code itself is located or > anything like that. If there's no tests within the gem then it has to > be a bad gem - this is what i'm thinking. That's why i would like to > have tests with the gems. Even if there's going to be any unified API > or not. > Good point. > It shouldn't be harder to run tests than to actually use the > lib thus having tests at some separate place and setting them > (separately) up isn't good option either. > Right, but since you have to change the security settings for IE to get the test (only) to run, we break this rule no matter what. The problem in the end is that our unit tests aren't really unit tests, and can't really meet all of these rules. The question, then, is which rules should we break? Maybe the solution would be to write some true unit tests and then package them with the gem? These would be tests that wouldn't actually fire up a browser, though, so although they would meet all of your rules, they would perhaps be less valuable. Also, why do you think that Watirspec is a subset of Watir/FireWatir > tests? I can see it as a pretty good replacement of current tests. > Yeah, there are few functionalities which aren't tested by Watirspec > like #attach and #click_no_wait, Watirspec was created for Celerity. Therefore, I have assumed that it only included tests for features that are supported in Celerity. This means it omits support for such features as these. but other than that i'd say the suite > is pretty decent. Or has someone made an explicit analysis of > Watirspec being a superset rather than just guessing? > I was also able to use the Watir test suites to find a number of discrepancies between Watir and FireWatir. I don't believe the Watirspec suite can do this. It could, but it would need quite a bit of work first. I think you've convinced me that I need to write up some better documentation on the Watir+ test suite, describing the various options and structure of the tests. There is a lot here that probably isn't being used. Take a look here for more... http://github.com/bret/watir/tree/master/commonwatir/unittests/setup/ In the old days, we put some of the support for the test suite directly in our library, which created problems because our users thought bits of it were part of watir (which in a way it was, but this wasn't our intention) so this is why I've been growing the library support for our unit tests outside of the lib directory. Like watirspec, our unit tests really want to be their own separate project. Bret > 2010/9/28 Bret Pettichord <[email protected]>: > > There is another, separate reason, why I am in favor of removing the unit > > tests from the gem. > > > > We used to do this, but we would often get complaints from users who > > couldn't run the unit tests. The problem is that there are a couple of > > additional requirements that you need to meet in order to get the unit > tests > > working. Users tend to treat the unit tests as a way of testing whether > they > > have installed Watir correctly, but this isn't what it is designed to do > nor > > accomplishes. So instead, they run into problems with the unit tests and > > think that these are problems they need to get sorted out. But they > aren't. > > > > One of these issues is that you have to turn off one of the security > > settings in IE before running the tests. We also saw a recent thread > about > > UAC access complicating unit tests. > > > > I think what people want is just something that verifies whether Watir is > > installed correctly. Maybe we could create something that does this and > > package it as a unit test and then separately maintain the old "unit > tests" > > as a compatability/regression suite separately. > > > > Bret > > > > 2010/9/27 Bret Pettichord <[email protected]> > >> > >> Right they used to be separate. Actually they use to be one test suite, > >> but then were duplicated when firewatir was forked from watir. Now they > >> contain lots of duplication, but also lots of subtle differences. > >> > >> I was trying to remove duplication. Also I added an option to each test > >> suite that allowed it to be run against the other implementation. This > >> "compatibility" mode was helpful in finding inconsistencies between the > two > >> implementations. These features required adding interdependencies that > broke > >> the ability to run in a gem. My goal was to eventually migrate all of > the > >> tests out into a separate project that could be used to run against any > >> implementation. This turned out to be a bigger task than I expected > because > >> of the large number of minor changes that were made, separately, in each > of > >> the two test suites. > >> > >> Yes Jari's watirspec is something like a unified test suite, but it is a > >> subset of what we have in the watir/firewatir project. > >> > >> I think it is interesting to discuss what to do about this, but this has > >> been a known issue for some two years and probably not something that > can > >> reasonably be resolved in the time frame of the 1.6.6 release. This is > why I > >> suggest we simply remove the unit tests from this version of the gem. > >> > >> Bret > >> > >> On Mon, Sep 27, 2010 at 2:42 PM, Jarmo <[email protected]> wrote: > >>> > >>> Isn't this completely separate problem? > >>> > >>> I see that currently there are unittests under FireWatir and unittests > >>> under Watir. They should be executable without depending of eachother, > >>> thus should be executable within installed gem and not just from the > >>> source. I would try to solve that problem first without creating any > >>> unified tests. > >>> > >>> When talking about unified Watir-API test suite, then isn't Jari's > >>> Watirspec something like that? > >>> > >>> Again, for me these things seem to be different problems. Correct me > >>> if i'm wrong. > >>> > >>> Jarmo > >>> > >>> On Mon, Sep 27, 2010 at 10:38 PM, Bret Pettichord <[email protected] > > > >>> wrote: > >>> > The reason we have this problem is that what we really need is a > >>> > unified > >>> > Watir-API test suite that could be run against any implementation. > >>> > > >>> > I was trying to convert our existing unit tests into such a test > suite, > >>> > but > >>> > ran out of time to complete this work and thus it is in an interim > >>> > state. > >>> > > >>> > One way to fix the problem would be to roll back this work. If this > is > >>> > your > >>> > plan, then i think it is a bad idea. > >>> > > >>> > Another way to fix the problem would be to complete this work and > >>> > migrate > >>> > all of the "unit tests" out of the individual gems. If this is your > >>> > plan, > >>> > then it is a good idea, but it also will probably result in not > having > >>> > the > >>> > "unit tests" inside the individual gems. > >>> > > >>> > So I guess the question is, what is your plan? > >>> > > >>> > Bret > >>> > > >>> > On Mon, Sep 27, 2010 at 2:18 PM, Jarmo <[email protected]> wrote: > >>> >> > >>> >> Why not fix it? If i say that i have plan to do it, is that a bad > >>> >> plan? > >>> >> Why? > >>> >> > >>> >> Jarmo > >>> >> > >>> >> On Mon, Sep 27, 2010 at 9:08 PM, Bret Pettichord < > [email protected]> > >>> >> wrote: > >>> >> > Yes the problem running unit tests from the gems is not fixed and > >>> >> > there > >>> >> > is > >>> >> > no plan to fix it. > >>> >> > > >>> >> > It would be best not to ship the unit tests with the gems. > >>> >> > > >>> >> > If we want to run the unit tests on different platforms, we should > >>> >> > do > >>> >> > this > >>> >> > by checking out the source from github, not installing the gems. > >>> >> > > >>> >> > Bret > >>> >> > > >>> >> > On Mon, Sep 27, 2010 at 7:34 AM, Željko Filipin > >>> >> > <[email protected]> wrote: > >>> >> >> > >>> >> >> On Fri, Sep 24, 2010 at 11:04 PM, Charley Baker > >>> >> >> <[email protected]> > >>> >> >> wrote: > >>> >> >> > we can take any > >>> >> >> > help from documentation to running tests on various OSes. > >>> >> >> > >>> >> >> I can run unit tests on all major OSes. Is it still the case that > >>> >> >> there > >>> >> >> is > >>> >> >> something wrong with unit tests provided with the gem? > >>> >> >> > >>> >> >> Željko > >>> >> >> > >>> >> >> _______________________________________________ > >>> >> >> Wtr-development mailing list > >>> >> >> [email protected] > >>> >> >> http://rubyforge.org/mailman/listinfo/wtr-development > >>> >> > > >>> >> > > >>> >> > > >>> >> > -- > >>> >> > Bret Pettichord > >>> >> > Lead Developer, Watir, www.watir.com > >>> >> > > >>> >> > Blog, www.io.com/~wazmo/blog <http://www.io.com/%7Ewazmo/blog> > >>> >> > Twitter, www.twitter.com/bpettichord > >>> >> > > >>> >> > > >>> >> > _______________________________________________ > >>> >> > Wtr-development mailing list > >>> >> > [email protected] > >>> >> > http://rubyforge.org/mailman/listinfo/wtr-development > >>> >> > > >>> >> _______________________________________________ > >>> >> Wtr-development mailing list > >>> >> [email protected] > >>> >> http://rubyforge.org/mailman/listinfo/wtr-development > >>> > > >>> > > >>> > -- > >>> > Bret Pettichord > >>> > Lead Developer, Watir, www.watir.com > >>> > > >>> > Blog, www.io.com/~wazmo/blog <http://www.io.com/%7Ewazmo/blog> > >>> > Twitter, www.twitter.com/bpettichord > >>> > > >>> > > >>> > _______________________________________________ > >>> > Wtr-development mailing list > >>> > [email protected] > >>> > http://rubyforge.org/mailman/listinfo/wtr-development > >>> > > >>> _______________________________________________ > >>> Wtr-development mailing list > >>> [email protected] > >>> http://rubyforge.org/mailman/listinfo/wtr-development > >> > >> > >> -- > >> Bret Pettichord > >> Lead Developer, Watir, www.watir.com > >> > >> Blog, www.io.com/~wazmo/blog <http://www.io.com/%7Ewazmo/blog> > >> Twitter, www.twitter.com/bpettichord > >> > > > > > > > > -- > > Bret Pettichord > > Lead Developer, Watir, www.watir.com > > > > Blog, www.io.com/~wazmo/blog <http://www.io.com/%7Ewazmo/blog> > > Twitter, www.twitter.com/bpettichord > > > > > > _______________________________________________ > > Wtr-development mailing list > > [email protected] > > http://rubyforge.org/mailman/listinfo/wtr-development > > > _______________________________________________ > Wtr-development mailing list > [email protected] > http://rubyforge.org/mailman/listinfo/wtr-development > -- Bret Pettichord Lead Developer, Watir, www.watir.com Blog, www.io.com/~wazmo/blog Twitter, www.twitter.com/bpettichord
_______________________________________________ Wtr-development mailing list [email protected] http://rubyforge.org/mailman/listinfo/wtr-development
