ahmedabu98 opened a new pull request, #24041: URL: https://github.com/apache/beam/pull/24041
#4869 added a [random string](https://github.com/apache/beam/pull/4869/files#diff-b47580f7fcc36b9a014a20be2dd4dc3207ce1e17120358bbf32330d64097f32fR170) to the end of query job names to make this step resilient to retries. This query step creates a temp destination + table to output the query to. The next step would be to export data from this temp table to files, then **delete the temp table+destination** and read from those files. The problem was if a retry happened right after/during temp table+destination deletion, we would need to retry the query job. However, BQ would see the old successful query job name and return as so without actually querying and creating the temp table. The next step, export to files, would always fail because the table we are trying to export from does not exist. The solution introduced by #4869 was written over in a code refactor in #12082 and the random string was lost [here](https://github.com/apache/beam/commit/9af54351ddc31c607d1634382704c8a52aaa78f3#diff-471ca9a2a4c3366613d75c1d98736b7d88c0873d2abc126e01577c937c3759dbL145). This re-introduced the problem of having a deterministic query job name that was vulnerable to retries. This PR just adds back that string to the end of the query job name. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
