[jira] [Assigned] (BEAM-8023) Allow specifying BigQuery Storage API readOptions at runtime
[ https://issues.apache.org/jira/browse/BEAM-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Beam JIRA Bot reassigned BEAM-8023: --- Assignee: (was: Kenneth Jung) > Allow specifying BigQuery Storage API readOptions at runtime > > > Key: BEAM-8023 > URL: https://issues.apache.org/jira/browse/BEAM-8023 > Project: Beam > Issue Type: Improvement > Components: io-java-gcp >Reporter: Jeff Klukas >Priority: P3 > Labels: stale-assigned > Time Spent: 2h 50m > Remaining Estimate: 0h > > We have support in the Java SDK for using the BigQuery Storage API for reads, > but only the target query or table is supported as a ValueProvider to be > specified at runtime. AFAICT, there is no reason we can't delay specifying > readOptions until runtime as well. > The readOptions are accessed by BigQueryStorageTableSource in getTargetTable; > I believe that's occurring at runtime, but I'd love for someone with deeper > BoundedSource knowledge to confirm that. > I'd advocate for adding new methods > `TypedRead.withSelectedFields(ValueProvider> value)` and > `TypedRead.withRowRestriction(ValueProvider value)`. The existing > `withReadOptions` method would then populate the other two as > StaticValueProviders. Perhaps we'd want to deprecate `withReadOptions` in > favor or specifying individual read options as separate parameters. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (BEAM-8023) Allow specifying BigQuery Storage API readOptions at runtime
[ https://issues.apache.org/jira/browse/BEAM-8023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chamikara Jayalath reassigned BEAM-8023: Assignee: Kenneth Jung > Allow specifying BigQuery Storage API readOptions at runtime > > > Key: BEAM-8023 > URL: https://issues.apache.org/jira/browse/BEAM-8023 > Project: Beam > Issue Type: Improvement > Components: io-java-gcp >Reporter: Jeff Klukas >Assignee: Kenneth Jung >Priority: Minor > Time Spent: 20m > Remaining Estimate: 0h > > We have support in the Java SDK for using the BigQuery Storage API for reads, > but only the target query or table is supported as a ValueProvider to be > specified at runtime. AFAICT, there is no reason we can't delay specifying > readOptions until runtime as well. > The readOptions are accessed by BigQueryStorageTableSource in getTargetTable; > I believe that's occurring at runtime, but I'd love for someone with deeper > BoundedSource knowledge to confirm that. > I'd advocate for adding new methods > `TypedRead.withSelectedFields(ValueProvider> value)` and > `TypedRead.withRowRestriction(ValueProvider value)`. The existing > `withReadOptions` method would then populate the other two as > StaticValueProviders. Perhaps we'd want to deprecate `withReadOptions` in > favor or specifying individual read options as separate parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003)
