Hello Beam users and community, The Beam Python SDK, and Java workers have a utility where they will print a log message whenever there's an execution thread where no state transitions happen for over five minutes.
These messages are common in two scenarios: 1. A deadlock happening in the worker (very uncommon, but possible) 2. An operation simply takes over 5 minutes (e.g. a slow RPC, waiting for an external event, etc). The old phrasing of these logs has often been a bit confusing, and led users to think that there was actual stuckness in the pipeline, when reality was more harmless: an operation was just slow. I am introducing a change[1] for the Apache Beam SDK to rephrase these logs, and make them less confusing. If you ever used these logs for your debugging, know that the string will change, but the logs will remain : ). If you didn't know about these, now you do, and hopefully they will be useful to you! : ) Thanks! -P. [1] https://github.com/apache/beam/pull/10446/files