New submission from Ben Spiller <spiller....@gmail.com>:
I just came across a fairly serious thread-safety / race condition bug in the logging.Loggers class, which causes random log lines to be lost i.e. not get passed to some of the registered handlers, if (other, unrelated) handlers are being added/removed using add/removeHandler from another thread during logging. This potentially affects all log handler classes, though for timing reasons I've found it easiest to reproduce with the logging.FileHandler. See attached test program that reproduces this. I did some debugging and looks like although add/removeHandler are protected by _acquireLock(), they modify the self.handlers list in-place, and the callHandlers method iterates over self.handlers with no locking - so if you're unlucky you can end up with some of your handlers not being called. A trivial way to fix the bug is by editing callHandlers and copying the list before iterating over it: - for hdlr in c.handlers: + for hdlr in list(c.handlers): However since that could affect the performance of routine log statements a better fix is probably to change the implementation of add/removeHandler to avoid in-place modification of self.handlers so that (as a result of the GIL) it'll be safe to iterate over the list in callHandlers, e.g. change removeHandler like this: - self.handlers.remove(hdlr) + newhandlers = list(self.handlers) + newhandlers.remove(hdlr) + self.handlers = hdlr (and the equivalent in addHandler) ---------- components: Library (Lib) files: logger-race.py messages: 329429 nosy: benspiller priority: normal severity: normal status: open title: Logger race condition - loses lines if removeHandler called from another thread while logging type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7 Added file: https://bugs.python.org/file47914/logger-race.py _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue35185> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
