On Oct 11, 7:57 pm, Hanne Moa <hanne....@gmail.com> wrote:
> On Sun, Oct 11, 2009 at 04:42, Simon Willison <si...@simonwillison.net> wrote:
> > I'm keen to receive as much feedback on the implementation as possible.
>
> It is perfectly possible to set up logging in such a way that all the
> logs for several programs wind up in the same file, or there are
> duplicate entries. The first happens/happened if/when several programs
> used the same interpreter. The second is user error. Been there been
> stumped by both. Users need to be protected from both, somehow, but
> how... Might be that a standard way for doing it in Django will
> suffice for case 2. Will a warning/docs be needed for case 1?
"Logs for several programs wind up in the same file": this could
happen if multiple programs (scripts) use a FileHandler for file
"foo.log" with mode "a" (append). Then the logs will end up in the
same file because that's what's been asked for. I'm not sure what you
mean when you say "several programs used the same interpreter" - as
normally you have one Python interpreter per process. So if you are
talking about one invocation of a Python executable, then I'm not sure
what you mean by "programs", other than the executable running several
scripts in turn, in a way such as:
for scriptname in scriptnames:
mod = __import__(scriptname)
mod.main()
In that case, since the Python logging system is per-process, it will
be there across these multiple invocations and behave as if all of the
scriptnames main() calls are part of the single process - which they
are.
Typically, duplicate messages are caused when multiple handlers are
added to loggers. For example if one handler was added to both "foo"
and the root logger, and if the "foo" logger propagates to its parent
(the default setting) then you will get messages twice - once from
"foo"'s handlers and once from the root's handlers. This often happens
because people wrongly conflate loggers and handlers and think there
has to be a one-to-one correspondence between them. Nowadays you only
occasionally see questions on comp.lang.python about duplicate
messages, so I'm not sure if this still causes confusion.
I'm not sure what you can do to protect users from this, it's similar
to how it's hard in Django to have code which is guaranteed to execute
once and only once (see Simon's separate thread about this), but in
logging it's easy to avoid. You just have to document how things work
as best you can, and go from there.
> When I first used the logging-library (2002?) I had a problem in
> threaded programs, some exceptions were swallowed that shouldn't have
> been, leading to rather mysterious behaviour. The exception-hierarchy
> have been changed (a couple of times ;) ) since then so it might no
> longer be an issue but I'd sleep better if there was tests for this
> too. Vinay?
What exception hierarchy are you talking about? Logging's policy on
swallowing exceptions has been consistent throughout, and is that
exceptions caused in logging (say, format string doesn't tie up with
arguments) are always swallowed because you don't want an application
to crash because of a logging error or misconfiguration. Exceptions
such as SystemExit and KeyboardInterrupt should never be swallowed.
Some other exceptions are handled by handleError() methods of handlers
which sometimes retry (in the case of sockets) and at other times
check the logging.raiseExceptions flag. If this is set to False (as it
should be in production) then exceptions during handling are always
swallowed - but these are exceptions due to the logging itself rather
than exceptions in the application.
Logging should be safe in threaded programs because threading.RLocks
are used to synchronize access to module shared state and I/O between
multiple threads. I don't know of any threading bugs - and remember,
use of threading just by itself can sometimes lead to "mysterious
behaviour" because it's easy to miss race conditions, deadlocks etc.
One problem with writing tests for threading is that a threaded
program can follow a whole bunch of execution paths, all of which are
correct, because of OS scheduling being affected by other stuff
running on the machine (e.g. backup jobs, network monitoring) which is
outside your control. All of which means that having a defined outcome
to compare against becomes problematic.
Of course if you have any specific thread-safety issue relating to
logging, then please raise an issue on the Python bug tracker and
attach a small script which illustrates the problem.
Regards,
Vinay Sajip
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
