mgorsk1 opened a new issue, #41287: URL: https://github.com/apache/beam/issues/41287
### What would you like to happen? BigQuery load jobs can use either the shared slot pool or reserved slots. Currently, when using BigQueryIO, using reserved slots relies on a reservation assignment being configured for the project. This makes it difficult to control reservation usage for individual Beam workloads. Assigning a PIPELINE reservation at the project level affects other applicable jobs in the same project, which might not be desirable when only specific Beam workloads should consume a particular reservation. [The BigQuery API supports specifying a reservation directly at the job level ](https://docs.cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfiguration ) through `JobConfiguration.reservation`. The field accepts a reservation in the following format: `projects/{project}/locations/{location}/reservations/{reservation}`. If it is not specified, BigQuery falls back to its reservation-assignment rules. **Proposed change** Add an optional `withReservation(...)` configuration to BigQueryIO, for example: ``` BigQueryIO.writeTableRows() .to(table) .withMethod(Method.FILE_LOADS) .withReservation("projects/my-project/locations/us/reservations/my-reservation"); ``` When configured, BigQueryIO should propagate this value to `JobConfiguration.reservation` for the BigQuery jobs it creates. When it is not configured, the existing behavior should remain unchanged. This would allow Beam users to route specific BigQuery jobs to a reservation without requiring a project-wide reservation assignment. ### Issue Priority Priority: 2 (default / most feature requests should be filed as P2) ### Issue Components - [ ] Component: Python SDK - [ ] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [x] Component: IO connector - [ ] Component: Beam YAML - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Infrastructure - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Prism Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
