damccorm opened a new pull request, #41145: URL: https://github.com/apache/beam/pull/41145
### Summary Fixes 600s `pytest-timeout` failures in `IcebergTest::test_only` (`databases/iceberg.yaml`) and `Iceberg_Cdc_To_IcebergTest::test_only` (`e2e/iceberg_cdc_to_iceberg.yaml`) in `PostCommit YAML Xlang Direct` ([run 42720954393](https://github.com/apache/beam/actions/runs/42720954393)). ### Root Cause PR #40982 (`74f141d1931`) updated `WatchForSnapshotsSdf` so that streaming `ReadFromIcebergCDC` reads with `to_timestamp` continue polling until `to_timestamp` has passed (`ReadUtils.hasPassed(toTimestamp)`, i.e. `toTimestamp < System.currentTimeMillis()`). Both `databases/iceberg.yaml` and `e2e/iceberg_cdc_to_iceberg.yaml` hardcoded far-future timestamps (`2078352000000` in 2035 and `4102444800000` in 2100) for their streaming `ReadFromIcebergCDC` steps, causing `WatchForSnapshotsSdf` to poll indefinitely after reading the pre-populated snapshots. ### Fix - Populate `NOW_MS` (`int(time.time() * 1000)`) before each pipeline step in `integration_tests.py` and preserve native non-string values on exact `{VAR}` placeholder matches in `replace_recursive`. - Use `to_timestamp: "{NOW_MS}"` in the streaming `ReadFromIcebergCDC` steps of `databases/iceberg.yaml` and `e2e/iceberg_cdc_to_iceberg.yaml` (matching `IcebergCatalogBaseIT.testReadWriteStreaming`). ### Verification - Triggered `beam_PostCommit_Yaml_Xlang_Direct.yml` workflow run on branch: [`42736112909`](https://github.com/apache/beam/actions/runs/42736112909) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
