The default behavior of Python logging is (1) Logging handlers may get added (usually in main or very close to it). (2) Logging goes to those handlers, or a default handler if none were added.
With this proposal, we would have (0) A default handler gets added. (1) Logging handlers may get added (usually in main or very close to it) in addition to the default. (2) Logging goes to these handlers and the default. On Fri, Dec 13, 2019 at 12:46 PM Pablo Estrada <pabl...@google.com> wrote: > > I looked at the documentation for basicConfig: > https://docs.python.org/3/library/logging.html#logging.basicConfig > > Specifically, the following line: > > > This function does nothing if the root logger already has handlers > > configured, unless the keyword argument force is set to True. > > That would mean that anyone can override the handling later on - which the > workers do? > Best > -P. > > On Fri, Dec 13, 2019 at 10:55 AM Robert Bradshaw <rober...@google.com> wrote: >> >> Thanks for looking into this. >> >> I'm not sure unconditionally calling logging.basicConfig() on module >> import is the correct solution--this prevents modules that wish to set >> up handlers in place of the default handler from being able to do so. >> (This is why logging.basicConfig is lazily done at the first logging >> statement for the root logger, rather than earlier.) >> >> On Thu, Dec 12, 2019 at 4:34 PM Pablo Estrada <pabl...@google.com> wrote: >> > >> > Hello all, >> > It has been pointed out to me by Chad, and also by others, that my logging >> > changes have caused logs to start getting lost. >> > >> > It seems that by never logging on the root logger, initialization for a >> > root handler is skipped; and that's what causes the failures. >> > >> > I will work on a fix for this. I am thinking of providing a very simple >> > apache_beam.utils.get_logger function that does something like this: >> > >> > def get_logger(name): >> > logging.basicConfig() >> > return logging.getLogger(name) >> > >> > And specific paths that need special handling of the logs should override >> > this config by adding their own handlers (e.g. sdk_worker, fn_api_runner, >> > etc). >> > >> > I hope I can have a fix for this by tomorrow. >> > Best >> > -P.
