Hi Myron,
Thanks for your quick response. The feature you are talking about could be
very useful I would suggest that it would only provide what I would like if
it formats the expectation failures in a way that they are not just a long
string of different failures. In regards to the before(:context) hooks I
did look at this an option, and you are absolutely right about the caveats
in my case (testing an api) the get/post methods from rspec-rails are not
usable as below.
Failure/Error: get 'test'
RuntimeError:
@routes is nil: make sure you set it in your tests setup method.
My hypothetical case of 100 expectations is really just being used to
emphasize the problem in reality I have a bunch of tests similar to this
that make more like 4-5 expectations but then if you expand it to look at
invalid user, disabled user and deleted user etc. you end up with a lot
more. Honestly it isn't a huge performance hit that make my tests take
hours and hours to run, but in saying that the more I write the worse it
will get. (By no way am I saying these tests are perfect I believe checking
it respects an XML format and is valid XML is probably superfluous)
context 'valid request' do
before do
@user = FactoryGirl.create(:authenticable_user)
# Not going to put the actual request here assume its something
post :api_request, request
end
it { expect(response).to be_ok }
describe 'with a valid user' do
it 'is a valid XML structure' do
expect { parse_xml(response.body) }.not_to raise_error
end
it 'is successful' do
expect(response.body).to include("success='true'")
end
it 'respects expected XML format' do
expect(response.body).to match_response_schema('login_response')
end
it 'contains a valid authentication token' do
auth_token = Nokogiri::XML(response.body).xpath("//login_response"
).attribute("auth_token").value
expect(auth_token).to match(#A regex)
end
end
Any way if you have hints etc. let me know. Honestly being able to use
before(:all) with post/get would fix this problem perfectly but from what
you have noted this seems not possible and may require some work on
rspec-raiils itself.
Thanks,
Jesse
On Thursday, 12 February 2015 16:47:18 UTC+13, Myron Marston wrote:
>
> On Wednesday, February 11, 2015 at 7:23:13 PM UTC-8, Jesse Whitham wrote:
>>
>> So I ran into this problem with Testing our API.
>>
>> The problem is the get request is called multiple times based on
>> examples. e.g this code below will run get 'test' twice.
>>
>> require 'rails_helper'
>>
>> describe API::TestController, type: controller do
>> before do
>> get 'test'
>> end
>>
>> it { expect(response).to be_ok }
>> it { expect(response.body).to eq('test code')end
>>
>> This is a problem when you start to have more expect statements in terms
>> of performance. As far as I know there is no good workarounds for examples
>> to re use the same response. The guide herehttp://betterspecs.org/#single
>> talks
>> about putting multiple expects into the it statement, this seems to go
>> against getting good failure responses.
>>
>> Using a before(:all) you get an error like so
>>
>> Failure/Error: get 'test'
>> RuntimeError:
>> @routes is nil: make sure you set it in your tests setup method.
>>
>> Is there a way to send only one request without ruining the failure
>> responses?
>> (or if you like use memoization over multiple examples)
>>
>> I did find you could use a global variable but this seems like the worst
>> code ever.
>>
>> require 'rails_helper'
>>
>> describe API::TestController, type: controller do
>> it 'makes a single request' do
>> get 'test'
>> $stupid_global = response
>> end
>> it { expect($stupid_global).to be_ok }
>> it { expect($stupid_global.body).to eq('test code')end
>>
>>
>> I posted this here https://github.com/rspec/rspec-core/issues/1876 and
>> got this response:
>>
>> This conundrum (shared state vs performance is one of the reasons we
>> added compound matchers to RSpec 3.2, so you can now do:
>>
>>
>> it { expect(response).to be_ok.and eq 'test code' }
>>
>>
>> This isn't a complete solution of course but we don't want to advocate
>> shared state across examples.
>>
>> Incidentally Github issues are not the place to request support, please
>> use the mailing list / google group (https:
>> //groups.google.com/forum/#!forum/rspec) and/or #rspec on freenode."
>>
>>
>> I really don't see this as a even usable solution as if you have 100
>> expectations
>>
>>
>> And you compound those you end up with failure in one string like so:
>>
>>
>> Failure/Error: "we expected it to have this and and we expected it to
>> have this and we expected it to have this and we expected it to have this
>> and we expected it to have this and we expected it to have this we
>> expected it to have this we expected it to have this we expected it to
>> have this we expected it to have this we expected it to have this we
>> expected it to have this"
>>
>> you don't compound them have one useless string with lots of expectations
>>
>> Failure/Error: "we expected the response to be ok (not sure why its not)"
>>
>> or you make 100 requests (massive performance load).
>>
>> Does anyone have any suggestions for better ways? Alternative testing
>> frameworks? (maybe rspec just isn't useful for this kind of testing) or
>> even a feature for shared state? (By the sounds of it this will not be
>> supported)
>>
>>
>
> Hey Jesse,
>
> This is a great question. One solution, which has been available for
> years, is to use a before(:context) (or before(:all) — that’s the old
> RSpec 2.x form, and it still works in RSpec 3) hook. See, for example, this
> PR
> <https://github.com/rspec/rspec-support/pull/179/files#diff-ec40054ce667411396ff663c4d03bb50R65>
> where
> I’m doing a slow operation in before(:context), storing it in an instance
> variable, making it available via some attr_reader declarations, and
> using the results from multiple examples.
>
> Note, however that before(:context) hooks come with many caveats. (See
> the “Warning: before(:context)” section from our docs
> <http://rspec.info/documentation/3.2/rspec-core/RSpec/Core/Hooks.html#before-instance_method>).
>
> The basic problem is that many things that integrate with RSpec — such as
> DB transactions from DB cleaner or rspec-rails, or the rspec-mocks test
> double life cycle — have a per-example life cycle, and running logic
> *outside* of that lifecycle can cause problems. If you create DB records
> in before(:context) and are using per-example DB transactions, it would
> create the records and not clean them up afterwords, potentially affecting
> later tests. So I’d say the before(:context) solution is great as long as
> you don’t have per-example life cycle stuff going on. If you do have that
> kind of stuff going on (and it’s very common to, especially in a rails
> context) you’re better off avoiding before(:context) or at least being
> extremely careful what you do in there.
>
> I think the “one expectation per example” guideline is a useful corrective
> to a pattern many first-time testers fall into, where they do too much in
> one test or one example, and have hard-to-understand test failures, but
> it's not something I recommend following strictly. Personally, I use “one
> expectation per example” as a signal…if I’m putting multiple expectations
> in one example I may be specifying multiple behaviors. In fast, isolated
> unit tests you want to keep each example focused on one behavior. In
> slower, integrated tests that’s far less important, and the cost of the
> setup time (and different kind of test) causes me to not worry about “one
> expectation per example”. If you are doing slow integrated testing and the
> thing being is so complicated that it needs 100 expectations (as per your
> hypothetical case), that suggests to me that your logic could benefit from
> being refactored, with more of it being extracted into stand-alone ruby
> objects that don’t interact with the slow external things and can be
> quickly unit tested in isolation.
>
> One other thing I’ve been mulling over recently is a new feature in RSpec
> that would better support what you’re trying to do. I’m thinking it would
> be something like:
>
> it "returns a successful response" do
> get 'test'
> aggregate_failures do
> expect(response).to be_ok
> expect(response.body).to eq("test code")
> endend
>
> The idea is that aggregate_failures (not necessarily what we’ll call it —
> it’s the best name I’ve thought of so far, though) will change how expect
> works
> for the duration of the block so that rather than aborting on first
> failure, it collects all expectation failures until the end of the example,
> and the block, and then, if there were any failures in the block, it’ll
> abort at that point with all of the failure output.
>
> Would that do what you want?
>
> HTH,
> Myron
>
--
You received this message because you are subscribed to the Google Groups
"rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/rspec/454ab805-c421-4ac8-a335-9a1f0e737653%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
