On Thu, Sep 17, 2009 at 4:25 PM, Simon Willison <si...@simonwillison.net> wrote:
>
> I think we should add logging to Django in version 1.2, implemented as
> a light-weight wrapper around the Python logging module
> (django.core.log maybe?) plus code to write errors to the Apache error
> log under the mod_python handler and environ['wsgi.errors'] under WSGI
> (meaning mod_wsgi will write to the Apache error log as well).
>
> Benefits of logging as a core Django service
> ============================================
>
> Adding logging to Django core would provide the following benefits:
>
> 1. We'll be able to de-emphasise the current default "e-mail all
> errors to someone" behaviour, which doesn't scale at all well.
>
> 2. Having logging in the core framework will mean people start
> actually using it, which will make it easier for people to debug their
> Django apps. Right now adding "print" statements to a Django app is a
> common debugging technique, but it's messy (you have to remember to
> take them out again) and error prone - some production environments
> throw errors if an app attempts to write to stdout. It's also not
> obvious - many developers are surprised when I show them the
> technique.
>
> 3. Logging in Django core rather than a 3rd party app will encourage
> reusable applications to log things in a predictable way, standard
> way.
>
> 4. 3rd party debugging tools such as the debug toolbar will be able to
> hook in to Django's default logging behaviour. This could also lead to
> plenty of additional 3rd party innovation - imagine a tool that looks
> out for logged SQL that took longer than X seconds, or one that groups
> together similar log messages, or streams log messages to IRC...
>
> 5. Built-in support for logging reflects a growing reality of modern
> Web development: more and more sites have interfaces with external web
> service APIs, meaning there are plenty of things that could go wrong
> that are outside the control of the developer. Failing gracefully and
> logging what happened is the best way to deal with 3rd party problems
> - much better than throwing a 500 and leaving no record of what went
> wrong.
>
> 6. Most importantly from my point of view, when a sysadmin asks where
> Django logs errors in production we'll have a good answer for them!
>
> 7. As a general rule, I believe you can never have too much
> information about what's going on with your web application. I've
> never thought to myself "the problem with this bug is I've got too
> much information about it". As for large log files, disk space is
> cheap - and pluggable backends could ensure logs were sensibly
> rotated.
No disagreement here with any of these assertions.
> Places logging would be useful
> ==============================
>
> - Unhandled exceptions that make it up to the top of the Django stack
> (and would cause a 500 error to be returned in production)
> - The development web server could use logging for showing processed
> requests (where currently these are just printed to stdout).
> - Failed attempts at signing in to the admin could be logged, making
> security audits easier.
> - We could replace (or complement) django.connection.queries with a
> log of executed SQL. This would make the answer to the common question
> "how do I see what SQL is being executed" much more obvious.
> - Stuff that loads things from INSTALLED_APPS could log what is being
> loaded, making it much easier to spot and debug errors caused by code
> being incorrectly loaded.
> - Likewise, the template engine could log which templates are being
> loaded from where, making it easier to debug problems stemming from an
> incorrectly configured TEMPLATE_DIRS setting.
> - We could use logging to address the problems with the template
> engine failing silently - maybe some template errors (the ones more
> likely to be accidental than just people relying on the fail-silent
> behaviour deliberately) should be logged as warnings.
>
> Most of the above would be set to a low log level which by default
> would not be handled, displayed or stored anywhere (logging.info or
> similar). Maybe "./manage.py runserver --loglevel=info" could cause
> such logs to be printed to the terminal while the development server
> is running.
>
> Problems and challenges
> =======================
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it in our
> own higher level interface.
In the absence of specifics, this makes me a little bit nervous. The
Python logging interface may be very Java-heavy and complex, but it is
a thoroughly known quantity, and it houses a lot of features.
I've seen several attempts to wrap Java loggers in a "nicer"
interface, and every one of them ended up hobbling some of the power
features of the logger. There is also the issue of our wrapper playing
nicely with the loggers already being used in the wild.
I'm also not entirely convinced that the answer here isn't just
documentation. The documentation for log4j has historically been
pretty awful, and while Python's documentation is an improvement, it
could certainly be better IMHO. Good documentation for how to use
logging in the context of Django could go a long way.
> 3. We risk people using logging where signals would be more
> appropriate.
This may be a better way to approach the problem - more details below.
> What would it look like?
> ========================
>
> Here's what I'm thinking at the moment (having given the API very
> little thought). In your application code:
>
> from django.core import log
> # Log to the default channel:
> log.debug('Retrieving RSS feed from %s' % url)
> # Log to a channel specific to your app:
> log.debug('Retrieving RSS feed from %s' % url, channel='myapp.rss')
> try:
> feed = httpfetch(url, timeout=3)
> except socket.timeout:
> log.info('Timeout fetching feed %s' % url)
>
> In settings.py:
>
> MIDDLEWARE_CLASSES = (
> ...
> 'django.middleware.LogErrorsToWSGI', # write exceptions to
> wsgi.errors
> )
> LOGGING_MIDDLEWARE_LEVEL = 'info'
>
> # If you want custom log handlers - not sure how these would interact
> with
> # channels and log levels yet
> LOG_HANDLERS = (
> 'django.core.log.handlers.LogToDatabase',
> 'django.core.log.handlers.LogToEmail',
> )
>
> What do people think? I'd be happy to flesh this out in to a full spec
> with running code.
Details notwithstanding, I'm +1 to the idea of adding logging to the
core framework - or, at least, making it easier to use logs for
reporting internal state and error conditions instead of email).
As for likely roadblocks: I've been led to believe that Adrian has
objections to framework-level logging. I have no idea as to the nature
of his objection, but ticket #5415 indicates that he is (or has been,
historically) in favor of adding signals that could be used for
logging or debugging purposes.
Yours,
Russ Magee %-)
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
