Yifan Mai created BEAM-7340: ------------------------------- Summary: DoFn.teardown metrics are lost in Python SDK Key: BEAM-7340 URL: https://issues.apache.org/jira/browse/BEAM-7340 Project: Beam Issue Type: Bug Components: sdk-py-harness Reporter: Yifan Mai
If user code in DoFn.shutdown updates custom user metrics, those updates will not get registered e.g. counter increments are not registered. Context: In [FnApiRunner.run_stages|https://github.com/apache/beam/blob/4629e82512ef1606c78cf28a2d66402c3533e23f/sdks/python/apache_beam/runners/portability/fn_api_runner.py#L342-L364], DoFn.teardown is called in worker_handler_manager.close_all, but this is called outside of the FnApiRunner.run_stage calls, so no metrics / monitoring info is retrieved there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)