On Sep 17, 9:25 am, Simon Willison <si...@simonwillison.net> wrote:
> Problems and challenges
> =======================
>
> 1. The Python logging module isn't very nicely designed - its Java
> heritage shines through, and things like logging.basicConfig behave in
> unintuitive ways (if you call basicConfig twice the second call fails
> silently but has no effect). This is why I suggest wrapping it in our
> own higher level interface.
Simon, I'm the author of Python's logging package. Sorry for the delay
in replying, I've been away from this list awhile. I think the "Java
heritage shines through" is just FUD. basicConfig's behaviour is fully
documented here:
http://docs.python.org/library/logging.html#logging.basicConfig
Including the fact that it sometimes (by design) has no effect.
There are a lot of people for whom logging just means writing to a
file, and that's why they have difficulty understanding why logging is
designed as it is. I would suggest you take a quick look at
http://plumberjack.blogspot.com/2009/09/python-logging-101.html
and then tell me why you think Python logging isn't well designed for
its purpose. You can do basic logging with two lines of setup (one
line if you ignore the import):
import logging
logging.basicConfig(level=logging.DEBUG,filename='/path/to/my/log',
format='%(asctime)s %(message)s')
and then
logging.getLogger(__name__).debug("Just checking this works")
Not too sure where the Java heritage is there, or where the hard part
is.
>
> 2. There may be some performance overhead, especially if we replace
> mechanisms like django.connection.queries with logging. This should be
> negligble: here's a simple benchmark:
>
> # ("hello " * 100) gives a 600 char string, long enough for a SQL
> statement>>> import timeit, logging
> >>> t = timeit.Timer('logging.info("hello " * 100)', 'import logging')
> >>> t.timeit(number=100) # one hundred statements
>
> 0.00061702728271484375>>> t.timeit(number=1000000) # one million statements
>
> 6.458014965057373
>
> That's 0.0006 of a second overhead for a page logging 100 SQL
> statements. The performance overhead will go up if you attach a
> handler, but that's fine - the whole point of a framework like
> 'logging' is that you can log as much as you like but only act on
> messages above a certain logging level.
A quick-and-dirty measurement showed me that a logging call (which
writes to file) takes on the order of 57 microseconds, which can be
reduced to around 50 microseconds if you forego collecting stack
frame, thread and process informaion. Not too shabby, though perhaps
not appropriate for extremely high-performance use cases. I hasten to
add, it's not a scientific benchmark.
>
> 3. We risk people using logging where signals would be more
> appropriate.
>
They're for entirely different purposes so I can't imagine this will
happen too often.
> 4. We might go too far, and make Django a "noisy" piece of software
> which logs almost everything that happens within it. Let's be tasteful
> about this.
>
One thing about the logging design (which perhaps makes it appear
complicated) is that developers can shape the logging in such a way
that the verbosity in different parts can be turned on and off pretty
much at will, and even without restarting the server in some cases.
So, with a little care in how things are arranged, this needn't
happen.
> 5. People might leave logging on, then find their server disk has
> filled up with log files and caused their site to crash.
>
> 6. Logging levels are confusing - what exactly is the difference
> between warn, info, error, debug, critical and fatal? We would need to
> document this and make decisions on which ones get used for what
> within the framework.
DEBUG: Detailed information, of no interest when everything is working
well but invaluable when diagnosing problems.
INFO: Affirmations that things are working as expected, e.g. "service
has started" or "indexing run complete". Often ignored.
WARNING: There may be a problem in the near future, and this gives
advance warning of it. But the application is able to proceed
normally.
ERROR: The application has been unable to proceed as expected, due to
the problem being logged.
CRITICAL: This is a serious error, and some kind of application
meltdown might be imminent.
I'll be happy to clarify further if needed.
> What would it look like?
> ========================
>
> Here's what I'm thinking at the moment (having given the API very
> little thought). In your application code:
>
> from django.core import log
I'm not sure it's a good idea to have a wrapper, as I don't believe
it's needed and may restrict the level of control you typically need
to have over logging. You'll not be convinced by my just saying so -
therefore, I'll be happy to work with you to understand what (in
specific areas, including at the module level) you're trying to
achieve, and explaining the best way to achieve it.
> # Log to the default channel:
> log.debug('Retrieving RSS feed from %s' % url)
> # Log to a channel specific to your app:
> log.debug('Retrieving RSS feed from %s' % url, channel='myapp.rss')
> try:
> feed = httpfetch(url, timeout=3)
> except socket.timeout:
> log.info('Timeout fetching feed %s' % url)
>
> In settings.py:
>
> MIDDLEWARE_CLASSES = (
> ...
> 'django.middleware.LogErrorsToWSGI', # write exceptions to
> wsgi.errors
> )
> LOGGING_MIDDLEWARE_LEVEL = 'info'
>
> # If you want custom log handlers - not sure how these would interact
> with
> # channels and log levels yet
> LOG_HANDLERS = (
> 'django.core.log.handlers.LogToDatabase',
> 'django.core.log.handlers.LogToEmail',
> )
>
> What do people think? I'd be happy to flesh this out in to a full spec
> with running code.
Please email me and I will be happy to help with this.
Regards,
Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
