mxtymoshyk opened a new pull request, #39934: URL: https://github.com/apache/beam/pull/39934
BigQueryIO creates temporary tables and datasets and deletes them when it is done. A delete can succeed at BigQuery and still have its work item fail to commit afterwards. The runner then replays that work item, and the replayed delete gets a 404 because the first attempt already removed the resource. `DatasetServiceImpl.deleteTable` and `deleteDataset` passed `ALWAYS_RETRY`, so that 404 was retried `MAX_RPC_RETRIES` (9) times with exponential backoff and then rethrown as an `IOException`. In a streaming job the work item is retried forever and the job cannot drain. Both methods now pass `DONT_RETRY_NOT_FOUND` and swallow an item-not-found error, since "the resource is gone" is the outcome the caller asked for. This mirrors how `getTable` in the same class already handles a 404. Every other status code keeps its existing retry count and failure behaviour. fixes #24997 ### Notes for reviewers * The 404 is only swallowed for the two delete calls. `dryRunQuery` and `patchTableDescription` still use `ALWAYS_RETRY`, and no `getX`/`create`/`patch` path changed. * `ApiErrorExtractor.itemNotFound` walks `getCause()`, so it still recognises the 404 after `executeWithRetries` wraps it in a new `IOException` -- the same reason the existing `getTable` catch block works. * A swallowed 404 is logged at INFO with the table/dataset id, so an operator can still see that the delete found nothing. * The two new tests prepare exactly one mock response, so a retry trips the `Verify` in the shared mock request. They assert both "did not throw" and "did not retry". * Verified locally on `:sdks:java:io:google-cloud-platform`: `BigQueryServicesImplTest` passes 50/50 with the change. Reverting only the `BigQueryServicesImpl` change makes both new tests fail with a `VerifyException` from the retried request, confirming the tests cover the fix. * No prior PR referenced this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
