Test framework and dispatching benchmarks

[Russell Keith-Magee]

Hi all,

As a result of some performance concerns, Adrian has reverted the template rendering signals that were part of the recent test framework changesets.

I was under the impression that the overhead of an dispatching a signal with no handlers would be negligible, so, to validate (or invalidate) Adrian's concerns, I've done a little benchmarking, and I have three options for moving forward.

On my machine, constructing and rendering a 'text only, empty context, no tags' template takes approx 0.16ms. Constructing and rendering a trivial template (the 'polls/template.html' from tutorial 3, using dictionaries in the context instead of QuerySets) takes 1.2ms.

If the dispatcher has no handlers registered, a call to dispatcher.send() takes on the order of 0.1ms. If you set up 100 signals, and register 5 handlers on each of those signals, dispatching a signal which has no handlers still takes on the order of 0.1ms.

So; dispatching isn't completely free, but it is fairly low impact. Keep in mind that these are pretty trivial templates, and the benchmark numbers I have presented don't take into account the time required for database processing, view processing, disk access and the like, so in practice, the difference will likely be much larger.

I can see 3 options.

Option 1 - Don't worry about the overhead. Always emit the template rendering signal; the testing system is the only mechanism that will listen to it. The time spent processing a signal that is ignored is at least an order of magnitude less than template rendering time, likely much less when the full view evaluation process is considered.

However, any loss is still a loss, and every little bit adds up when you are being Slashdotted.

Option 2 - Add a boolean condition to shortcut the signal dispatch:

def render():
    if settings.TESTING :
        dispatcher.dispatch(...)
    return <rendered template>

This reduces the overhead in the dispatcher system itself, at the cost of a single boolean comparison. However, it does make a special case of testing in general code.

Option 3: In the setup of the test target in django-admin, replace Template.render with a template rendering method that dispatches the required signal.

def test_renderer(self, context):
    dispatcher.dispatch(...)
    return self.render_internal(context)

Template.render_internal = Template.render
Template.render = test_renderer

This results in zero impact unless you are actually testing. However, it does mean that unit tests are not guaranteed to be standalone. To overcome this, the installation of the test renderer could be put into an initialization method somewhere in the test module, with the documentation describing how to configure Django for testing for those that want to use a external testing tool.

Personally, I find myself leaning towards option 3 - outside of testing, I can't see any use case for a template-rendering signal, and I don't like special cases. Instrumentation of the rendering system as part of the test framework setup seems an appropriate solution.

Comments? Questions? Alternatives? Preferences?

Yours,
Russ Magee %-)

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/django-developers
-~----------~----~----~----~------~----~------~--~---