Question: In Cucumber when you're writing code to satisfy steps and accessing the model objects directly, what support for asserts, responses, etc. do people use. (the equivalent of ActionController::TestCase and ActiveSupport::TestCase), Fixtures, etc.
Thanks, T On Tue, Nov 25, 2008 at 12:16 PM, Matt Wynne <[EMAIL PROTECTED]> wrote: > > On 25 Nov 2008, at 17:26, Ben Mabey wrote: > >> David Chelimsky wrote: >>> >>> On Tue, Nov 25, 2008 at 12:52 AM, Ben Mabey <[EMAIL PROTECTED]> wrote: >>> >>>> Andrew Premdas wrote: >>>> >>>>> I came across this idea of dropping unit tests for acceptance tests in >>>>> the java world. It didn't like it there and I don't like it here, but >>>>> maybe thats because I'm an old fuddy duddy or something :). I do think >>>>> that every public method of an object should be specifically unit >>>>> tested, and yes that means that if you refactor your object you should >>>>> refactor your unit tests. This isn't really that much of a burden if >>>>> you design your objects to have simple and minimal public api's in the >>>>> first place. >>>>> >>>>> >>>> +1 >>>> >>>>> What is that makes you think you can refactor code run acceptance >>>>> tests and be save without unit tests? Writing tests "that guarantee >>>>> the correct functioning of the system" isn't something you can just >>>>> do. Best you can hope for with acceptance tests is that part of the >>>>> system functions correctly most of the time in some circumstances. >>>>> >>>>> Perhaps its the BDD ideal that your only writing the code you need to >>>>> make your acceptance tests pass, that make you think your acceptance >>>>> tests cover all your code. However just because you've written minimal >>>>> code to make an acceptance test pass doesn't mean that you can't use >>>>> this code in a multitude of different ways >>>>> >>>>> Do you really think that BDD created code is just some black box that >>>>> you can tinker around with restructure and still be sure it works just >>>>> because your black box tests still work? >>>>> >>>>> I just don't believe you can get the coverage you need for an >>>>> application using acceptance testing / features alone. If you do >>>>> actually write enough features to do this you'll end up doing much >>>>> more work than writing unit tests combined with features. >>>>> >>>>> >>>> +1 again. >>>> >>>>> All best >>>>> >>>>> Andrew >>>>> >>>>> >>>> Here is how I look at the two sets of tests... >>>> >>>> Features at the application level (acceptance tests) instill more >>>> confidence >>>> >>> >>> CONFIDENCE! >>> >>> That and, as Kent Beck describes today, responsible software, are why >>> we do testing at all. >>> >>> >>>> in me about the correctness of the system's behavior. Object level code >>>> examples (unit tests) instill more confidence in me about the design of >>>> the >>>> system. >>>> With acceptance tests passing we have no guarantee about the state of >>>> the >>>> design. Remember, TDD/BDD naturally produces easy to test objects and >>>> by >>>> skipping object level examples you run the risk of creating dependent >>>> laden, >>>> highly coupled objects that are hard to test. (Just think, you can make >>>> all >>>> of your features, for a web app, pass by writing the app in PHP4 with no >>>> objects at all :p .) >>>> >>> >>> Which is not an inherently bad deal, if that's your comfort zone, and >>> if that's the comfort zone of *everybody* on your team. >>> >>> >>>> I also think that acceptance tests are too slow to be used in all >>>> refactorings and they are not fine grained enough so you'll end up doing >>>> more debugging than you would otherwise with good object level coverage. >>>> I >>>> generally try to keep each individual unit test faster than a tenth of a >>>> second, as suggested in 'Working Effectively With Legacy Code'. What >>>> results is an extremely fast suite that can be used to quickly do >>>> refactorings. I have experienced the pain of using just Cucumber >>>> features >>>> first hand-- finding bugs on this level is just not as fast object level >>>> examples. If you skip object level examples you are incurring a >>>> technical >>>> debt that you will feel down the road, IMO. >>>> >>>> Someone at the start of this thread had wondered what people had learned >>>> when they went through this process of balancing FIT tests with unit >>>> tests. >>>> >>> >>> I can speak to this a bit. Maybe more than a bit. >>> >>> When I was working with .NET FitNesse and NUnit, we had very high >>> levels of coverage in NUnit. Early on one project I told Micah Martin >>> (who co-created FitNesse with Bob Martin) that I was concerned about >>> the duplication between our FitNesse tests and NUnit tests and >>> questioned the value of keeping it. >>> >>> Micah pointed out reasons that made absolute 100% perfect sense in the >>> context of the project we were working on. The customers were >>> encouraged to own the FitNesse tests. They were stored on a file >>> system, backed up in zip files, while the NUnit tests were stored in >>> subversion with the code. The FitNesse fixtures were stored with the >>> application code, distant from the FitNesse tests. >>> >>> In order to foster confidence in the code amongst the developers, >>> having a high level of coverage in NUnit made sense, in spite of the >>> duplication with some of the FitNesse tests. >>> >>> That duplication, by the way, was only in terms of method calls at the >>> highest levels of the system. When a FitNesse test made an API call, >>> that message went all the way to the database and back. >>> >>> When an NUnit test made the same call, that message typically got no >>> further than the object in the test, using stubs and mocks to keep it >>> isolated. >>> >>> Now fast forward to our current discussion about Cucumber and RSpec. >>> As things stand today, we tend to store .feature files right in the >>> app alongside the step_definitions and the application code. >>> >>> The implications here are different from having a completely decoupled >>> acceptance testing system. I'm not saying that abandoning RSpec or >>> Test::Unit or whatever is the right thing to do. But I certainly feel >>> less concerned about removing granular code examples, especially on >>> rails/merb controllers and views, when I've got excellent coverage of >>> them from Cucumber with Webrat. Thus far I have seen a case where I >>> couldn't quickly understand a failure in a view or controller based on >>> the feedback I get from Cucumber with Webrat. >>> >>> But this is mostly because that combination of tools does a very good >>> job of pointing me to the right place. This is not always the case >>> with high level examples. If you're considering relaxing a requirement >>> for granular examples, you should really consider each case separately >>> and include the level of granularity of feedback you're going to get >>> from your toolset when you make that decision. >>> >>> Now this is how *I* see things. >>> >>> For anybody who is brand new to all this, my feeling is that whatever >>> pain there is from duplication between the two levels of examples and >>> having to change granular examples to refactor is eclipsed by the pain >>> of debugging from high level examples. >>> >>> Also, as I alluded to earlier, every team is different. If you are >>> working solo, the implications of taking risks by working >>> predominantly at higher levels is different from when you are on a >>> team. The point of testing is not to follow a specific process. The >>> point is to instill confidence so you can continue to work without >>> migraines, and deliver quality software. >>> >>> Cheers, >>> David >>> >> >> Thanks for sharing your experience and insight! Having never used >> FitNesse I didn't see that distinction at all. What you said makes a lot of >> sense. >> -Ben > > Amen to that. Thanks guys, it's been a fascinating and enlightening > discussion. > > I am looking forward to the next chance I get to talk about this with > someone (who's interested!) over a beer. > > I don't suppose any of you are going to XP Day, London, this year? > > cheers, > Matt > _______________________________________________ > rspec-users mailing list > rspec-users@rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users > _______________________________________________ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
