On Oct 11, 7:57 pm, Hanne Moa <hanne....@gmail.com> wrote: > On Sun, Oct 11, 2009 at 04:42, Simon Willison <si...@simonwillison.net> wrote: > > I'm keen to receive as much feedback on the implementation as possible. > > It is perfectly possible to set up logging in such a way that all the > logs for several programs wind up in the same file, or there are > duplicate entries. The first happens/happened if/when several programs > used the same interpreter. The second is user error. Been there been > stumped by both. Users need to be protected from both, somehow, but > how... Might be that a standard way for doing it in Django will > suffice for case 2. Will a warning/docs be needed for case 1?
"Logs for several programs wind up in the same file": this could happen if multiple programs (scripts) use a FileHandler for file "foo.log" with mode "a" (append). Then the logs will end up in the same file because that's what's been asked for. I'm not sure what you mean when you say "several programs used the same interpreter" - as normally you have one Python interpreter per process. So if you are talking about one invocation of a Python executable, then I'm not sure what you mean by "programs", other than the executable running several scripts in turn, in a way such as: for scriptname in scriptnames: mod = __import__(scriptname) mod.main() In that case, since the Python logging system is per-process, it will be there across these multiple invocations and behave as if all of the scriptnames main() calls are part of the single process - which they are. Typically, duplicate messages are caused when multiple handlers are added to loggers. For example if one handler was added to both "foo" and the root logger, and if the "foo" logger propagates to its parent (the default setting) then you will get messages twice - once from "foo"'s handlers and once from the root's handlers. This often happens because people wrongly conflate loggers and handlers and think there has to be a one-to-one correspondence between them. Nowadays you only occasionally see questions on comp.lang.python about duplicate messages, so I'm not sure if this still causes confusion. I'm not sure what you can do to protect users from this, it's similar to how it's hard in Django to have code which is guaranteed to execute once and only once (see Simon's separate thread about this), but in logging it's easy to avoid. You just have to document how things work as best you can, and go from there. > When I first used the logging-library (2002?) I had a problem in > threaded programs, some exceptions were swallowed that shouldn't have > been, leading to rather mysterious behaviour. The exception-hierarchy > have been changed (a couple of times ;) ) since then so it might no > longer be an issue but I'd sleep better if there was tests for this > too. Vinay? What exception hierarchy are you talking about? Logging's policy on swallowing exceptions has been consistent throughout, and is that exceptions caused in logging (say, format string doesn't tie up with arguments) are always swallowed because you don't want an application to crash because of a logging error or misconfiguration. Exceptions such as SystemExit and KeyboardInterrupt should never be swallowed. Some other exceptions are handled by handleError() methods of handlers which sometimes retry (in the case of sockets) and at other times check the logging.raiseExceptions flag. If this is set to False (as it should be in production) then exceptions during handling are always swallowed - but these are exceptions due to the logging itself rather than exceptions in the application. Logging should be safe in threaded programs because threading.RLocks are used to synchronize access to module shared state and I/O between multiple threads. I don't know of any threading bugs - and remember, use of threading just by itself can sometimes lead to "mysterious behaviour" because it's easy to miss race conditions, deadlocks etc. One problem with writing tests for threading is that a threaded program can follow a whole bunch of execution paths, all of which are correct, because of OS scheduling being affected by other stuff running on the machine (e.g. backup jobs, network monitoring) which is outside your control. All of which means that having a defined outcome to compare against becomes problematic. Of course if you have any specific thread-safety issue relating to logging, then please raise an issue on the Python bug tracker and attach a small script which illustrates the problem. Regards, Vinay Sajip --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---