Em 16-04-2012 09:27, Rodrigo Rosenfeld Rosas escreveu:
Thank you David and Matt for your comments.
Indeed I had considered using FakeWeb but I thought that maybe there
could be an easier way just to mock 'open' directly since I guess it
would be faster to run.
With regards to the work around to overcome the issue with mocking AR
so that I could speed up my tests I think that creating helpers in the
model that will restrict the use of the AR API sounds like a good
balance, but that wouldn't make my specific tests faster.
Here is why: my Model class shouldn't be created or updated by any
other code in my application. So I'd just have to add a new method to
the Model class that would be specific to this action and I'd need to
write a "unit" test for this new method. I quoted "unit" because it is
actually an integration test as it will touch the database or
otherwise I wouldn't be able to upgrade Rails to a new version if I'm
mocking some methods from AR and they change in a future version.
So I'd need to touch the database anyway and my 9 specs would still
take almost a full second to run while this specific controller spec
takes about 0.7s (up to 2s depending on my system load).
Well, actually, I guess RSpec has changed the way it measure spec runs
since I last used it. I'm saying that because it will report me from
0.7 to 2s when I run "rspec path/to/my_controller_spec.rb", but if I
run "rspec --profile" it will run the full suite about the same time
and this specific controller action will take 0.38s.
Anyway, I don't see how I would create hundreds of specs and expect
them to run in a few seconds and still have a reliable test suite. I
mean, I want to feel safe if I'm upgrading to a newer Rails version
even if it changed its AR API.
The only real improvements I can see for my tests running fasts
without changing anything in the specs is to use something like Hydra
so that I could use the full power of my 6-core processor.
Does RSpec has a built-in threaded option? I guess not because I don't
think it would be much effective on CRuby because it would probably
lock on database access (or maybe not, I don't know how the pg driver
is implemented). For my particular application I suspect it is not
cpu-heavy, but io-expensive instead. So maybe a multi-thread spec
runner could improve its performance when running multiple specs.
Best,
Rodrigo.
After some more investigation I could find out why it is taking so much
time for the spec to run. Even for a simple model, calling a single
"Model.create! name: 'Model name'" will take almost 100ms. So, here is
usually what happens:
root = FactoryGirl.create :model # 2 creates:
# section = Section.create! name: 'Section'
# root = Model.create! section: section, label: 'Label'
post 'action', parent_id: root.id # another Model creation in the
controller alongside with 2 other creations of a has_many association
So, without counting any assertions just for creating all records so far
about 400ms were already spent. There will also be about 2 more updates
and 2 "delete" statements in this spec and that will complete about
540ms of total spec run time.
Another interesting fact is that for measuring this I've mocked the user
and controller authentication related methods, but if I actually create
the user the time for completing the spec will be about the same, even
if in the logs it is reported to spend about 70ms for executing the
insert in the user's table.
But I don't think it should take so much time for creating/updating
those records. See the output of the logging, for example:
Completed 200 OK in 83ms (Views: 0.6ms | ActiveRecord: 14.9ms)
This is consistent with the individual timing for each query/insert
statement including validations. Even so, the first 'post' method call
in the spec takes about 100ms.
So, my question is, if AR and Views are taking about 16ms what else
could be taking about 70ms?
This is also hard to profile since PostgreSQL will need more time for
the first queries and will be much faster for subsequent ones... This
means that running all specs will take much less time than if I run each
spec separately and compare to the total time.
But I don't really think I'll be able to test this action faster. I need
to check for all created records including associations and fields set
by the before_validation and removed by after_save so that I can get
confidence in the code. And since all of this should only happen through
this action, it wouldn't matter if I moved lots of those checks to the
"unit" test of my Model as the total time would remain the same. It
would only make my specs harder to read with lots of mocks and stubbed
methods in my controller spec.
I mean, that is ok, I don't aim in having single assertions per unit
test. I just need to make sure things won't break in a future
refactoring or requirement changes. And of course I'd like my tests to
run faster, but my current understanding is that I can't get this test
to run faster because it is testing a lot of code. Of course that an
in-memory AR adapter could help me speeding up my tests but using mocks
here won't give me any confidence on my code base.
Usually slow tests are not a problem when I'm working in some controller
as I don't need to run the full suite. But when I change some model or
library it would be great to run the full suite and I'm a bit worried if
that will become super slow some time in the future.
But I think I'll start to worry about this when this future becomes
present because there is no easy replacement that will allow my specs to
run faster. I'll still be able to try Hydra before having to resort to
mocks if things get really unsustainable.
But thank you very much for your input on the subject. I was just
worried if I was doing something fundamentally wrong.
Cheers,
Rodrigo.
_______________________________________________
rspec-users mailing list
rspec-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users