bvolpato opened a new pull request, #40857: URL: https://github.com/apache/beam/pull/40857
The streaming autosharding test uses plain inserts and expects an exact row count. JdbcIO allows committed writes to be replayed after a retry, so that fixture can fail on duplicate rows ([example run](https://github.com/apache/beam/actions/runs/40307594966)). Give this fixture a primary key and use PostgreSQL's `ON CONFLICT DO NOTHING`. Keep the configured workload, streaming/autosharding options, and exact row-count assertion. The workload now includes primary-key and conflict checks, so its performance baseline will change. The focused JDBC unit test and PostgreSQL integration test passed locally. A controlled JdbcIO retry after a committed transaction produced excess rows with the old fixture and the expected count with the new one. A missing-record control still failed. The full Dataflow performance run was not run locally. <!-- Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). - [x] Replace these instructions with a description of your change and why you have made it. - [ ] Mention the appropriate issue in your description (for example: `addresses #123` or `fixes #456`). - [ ] Update `CHANGES.md` with noteworthy changes. See [CONTRIBUTING.md]() for more details. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
