The Python basicConfig[1] sets up a StreamHandler[2], which by default
publishes to stderr. This configuration is applied by default whenever
someone logs anything on the root logger (e.g. logging.info('abc')). With
the module-based logging changes, the basic config is never called on the
pipeline construction side, so logging depends 100% on the user, instead of
having a 'reasonable default'.The default configuration gets setup if the user ever logs on the root logger (logging.info, ...). If they don't then, nothing is set up. I'd say that it's best to be consistent with our previous behavior, which was to have a default configuration by virtue of logging on the root logger. By calling basicConfig in Pipeline.__init__, we allow the user to set up a configuration before creating the Pipeline object, or we set up a default at that point. Thoughts? [1] https://docs.python.org/3/library/logging.html#logging.basicConfig [2] https://docs.python.org/3/library/logging.handlers.html#logging.StreamHandler On Tue, Dec 17, 2019 at 2:42 PM Luke Cwik <lc...@google.com> wrote: > In Beam Java, the expectation has always been that pipeline authors are > responsible for setting up logging correctly during pipeline construction > time and that the Beam SDK is responsible for setting up logging at > pipeline execution time. > > Is this something we can solve by documenting and telling users to > configure logging at pipeline construction time or is the expected python > user experience that logging always works out of the box? > > On Tue, Dec 17, 2019 at 2:15 PM Pablo Estrada <pabl...@google.com> wrote: > >> It should not affect debuggability at pipeline runtime, as the sdk_worker >> already does the appropriate setup of handlers, but it may affect debugging >> of e.g. ptransform expansion, and pipeline construction issues. >> Best >> -P. >> >> On Tue, Dec 17, 2019 at 1:59 PM Udi Meiri <eh...@google.com> wrote: >> >>> Pablo, does the issue affect debuggability of pipelines? >>> >>> On Mon, Dec 16, 2019 at 6:23 PM Chad Dombrova <chad...@gmail.com> wrote: >>> >>>> >>>> >>>> On Mon, Dec 16, 2019 at 5:59 PM Pablo Estrada <pabl...@google.com> >>>> wrote: >>>> >>>>> +chad...@gmail.com <chad...@gmail.com> is this consistent with >>>>> behavior that you observed? >>>>> >>>> >>>> I honestly can't recall, sorry. I just remember that while I was >>>> testing I updated sdk version and some logging stopped. I *think* I was >>>> missing the state/message stream, which would be on the client side after >>>> pipeline construction. >>>> >>>> -chad >>>> >>>>
