pandasanjay opened a new issue, #35196: URL: https://github.com/apache/beam/issues/35196
### Is your feature request related to a problem? A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] ### Describe the solution you'd like Support for a Storage Read API enrichment handler would be beneficial. This approach offers several advantages over traditional BigQuery SQL-based implementations: * **Performance:** The Storage Read API provides a faster way to read data from BigQuery, especially for large datasets, as it streams data directly from storage without the overhead of SQL query execution. * **Cost Efficiency:** By bypassing SQL query execution, it can lead to cost savings, particularly for high-volume read operations. * **Scalability:** The direct stream access is inherently more scalable for high-throughput enrichment scenarios. * **Flexibility:** It allows for more complex, programmatic data manipulation and enrichment logic that might be cumbersome or inefficient to express in SQL. ### Describe alternatives you've considered The current alternative is using traditional BigQuery SQL-based implementations for enrichment. https://github.com/apache/beam/blob/master/sdks/python/apache_beam/transforms/enrichment_handlers/bigquery.py ### Release note <!-- Now that you have thought about the problem, the solution and the alternatives, please provide a release note. This will help us prioritize the feature request. --> <!-- The release note should be written in the past tense and include a brief summary of the feature. --> <!-- It should be suitable for inclusion in the release notes of the next Apache Beam release. --> <!-- Example: Added support for X in Y. --> Added support for a Storage Read API enrichment handler, offering performance and cost benefits over traditional SQL-based BigQuery enrichment. ### Anything else? _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
