Hi Joey, Thanks for reaching out! I see that your changes haven't been deploed yet, so I've triggered the corresponding job and Playground will be updated soon.
Thanks, Andrey From: Joey Tran <joey.t...@schrodinger.com> Reply to: "dev@beam.apache.org" <dev@beam.apache.org> Date: Thursday, 7 March 2024 at 22:33 To: Valentyn Tymofieiev <valen...@google.com> Subject: [EXTERNAL] Re: Hiding logging for beam playground examples I ended up making this change[1] but it looks like Playground is still outputting all the log statements. Could someone help me with this? I suppose it's best to try and actually test this, but I had a hard time getting the environment set up. [1]https://github.com/apache/beam/pull/29948 """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""ghnt [1] https://github.com/apache/beam/pull/29948#event-11504996254 On Thu, Nov 16, 2023 at 7:42 AM Joey Tran <joey.t...@schrodinger.com<mailto:joey.t...@schrodinger.com>> wrote: Good idea though it ended up being a shallow trace ``` File "/opt/playground/backend/executable_files/91e3e49b-7197-4252-a8bd-93c5b252ed55/91e3e49b-7197-4252-a8bd-93c5b252ed55.py", line 57, in <module> assert False ``` I think I found where the log level is set anyways ``` https://github.com/apache/beam/blob/master/playground/infrastructure/logger.py#L39 ``` When I have some time, I'll try doing a local deployment of playground and modifying those log levels On Wed, Nov 15, 2023 at 10:52 PM Valentyn Tymofieiev <valen...@google.com<mailto:valen...@google.com>> wrote: I am also not familiar with Playground. I suspect you could try to make it crash and maybe find a stacktrace? Setting logging could like like so: https://github.com/apache/beam/blob/729c4de416b8252ec99f0a1253ac7af3023733df/sdks/python/apache_beam/examples/wordcount.py#L110 On Wed, Nov 15, 2023 at 12:06 PM Joey Tran <joey.t...@schrodinger.com<mailto:joey.t...@schrodinger.com>> wrote: The motivating example does not use LogElements, just Map(print) https://beam.apache.org/documentation/transforms/python/aggregation/combineglobally/#example-2-combining-with-a-lambda-function Some examples of the extraneous logging: ``` 2023-09-08 22:46:37,334 [INFO] ==================== <function populate_data_channel_coders at 0x7ff2665e1a20> ==================== 2023-09-08 22:46:37,336 [INFO] Creating state cache with size 104857600 2023-09-08 22:46:37,338 [INFO] Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.EmbeddedWorkerHandler object at 0x7ff2664c9870> for environment ref_Environment_default_environment_2 (beam:env:embedded_python:v1, b'') ``` The example code itself doesn't set the log level in some playground code. Does anyone have a pointer to where? I'm not familiar On Wed, Nov 15, 2023 at 2:10 PM Valentyn Tymofieiev via dev <dev@beam.apache.org<mailto:dev@beam.apache.org>> wrote: Are the examples using LogElements? https://github.com/apache/beam/blob/2012107a0fa2bb3fedf1b5aedcb49445534b2dad/sdks/python/apache_beam/transforms/util.py#L1271 Note that LogElements by default prints to stdout, but can be configured to use a different logger. We could also change the default. On Tue, Nov 14, 2023 at 9:48 AM Robert Bradshaw via dev <dev@beam.apache.org<mailto:dev@beam.apache.org>> wrote: +1 to at least setting the log level to higher than info. Some runner logging (e.g. job started/done) may be useful. On Tue, Nov 14, 2023 at 9:37 AM Joey Tran <joey.t...@schrodinger.com<mailto:joey.t...@schrodinger.com>> wrote: > > Hi all, > > I just had a workshop to demo beam for people at my company and there was a > bit of confusion about whether the beam python playground examples were even > working and it turned out they just got confused by all the runner logging > that is output. > > Is this worth keeping? It seems like it'd be a common source of confusion for > new users > > Cheers, > Joey
