On Fri, Dec 13, 2019 at 1:08 PM Robert Bradshaw <[email protected]> wrote:
> The default behavior of Python logging is > > (1) Logging handlers may get added (usually in main or very close to it). > (2) Logging goes to those handlers, or a default handler if none were > added. > > With this proposal, we would have > > (0) A default handler gets added. > (1) Logging handlers may get added (usually in main or very close to > it) in addition to the default. > (2) Logging goes to these handlers and the default. > > On Fri, Dec 13, 2019 at 12:46 PM Pablo Estrada <[email protected]> wrote: > > > > I looked at the documentation for basicConfig: > https://docs.python.org/3/library/logging.html#logging.basicConfig > > > > Specifically, the following line: > > > > > This function does nothing if the root logger already has handlers > configured, unless the keyword argument force is set to True. > > > > That would mean that anyone can override the handling later on - which > the workers do? > > Best > > -P. > > > > On Fri, Dec 13, 2019 at 10:55 AM Robert Bradshaw <[email protected]> > wrote: > >> > >> Thanks for looking into this. > >> > >> I'm not sure unconditionally calling logging.basicConfig() on module > >> import is the correct solution--this prevents modules that wish to set > >> up handlers in place of the default handler from being able to do so. > >> (This is why logging.basicConfig is lazily done at the first logging > >> statement for the root logger, rather than earlier.) > >> > >> On Thu, Dec 12, 2019 at 4:34 PM Pablo Estrada <[email protected]> > wrote: > >> > > >> > Hello all, > >> > It has been pointed out to me by Chad, and also by others, that my > logging changes have caused logs to start getting lost. > Does this include losing user logs? Is there a JIRA, and should we fix it in Beam 2.18 branch? > >> > > >> > It seems that by never logging on the root logger, initialization for > a root handler is skipped; and that's what causes the failures. > >> > > >> > I will work on a fix for this. I am thinking of providing a very > simple apache_beam.utils.get_logger function that does something like this: > >> > > >> > def get_logger(name): > >> > logging.basicConfig() > >> > return logging.getLogger(name) > >> > > >> > And specific paths that need special handling of the logs should > override this config by adding their own handlers (e.g. sdk_worker, > fn_api_runner, etc). > >> > > >> > I hope I can have a fix for this by tomorrow. > >> > Best > >> > -P. >
