[ https://issues.apache.org/jira/browse/BEAM-2532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090599#comment-16090599 ]
Neville Li commented on BEAM-2532: ---------------------------------- Would love to see a fix in the next release. This is a big performance regression for us since we use BigQuery heavily. > BigQueryIO source should avoid expensive JSON schema parsing for every record > ----------------------------------------------------------------------------- > > Key: BEAM-2532 > URL: https://issues.apache.org/jira/browse/BEAM-2532 > Project: Beam > Issue Type: Improvement > Components: sdk-java-gcp > Affects Versions: 2.0.0 > Reporter: Marian Dvorsky > Assignee: Chamikara Jayalath > Priority: Minor > > BigQueryIO source converts the schema from JSON for every input row, here: > https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQuerySourceBase.java#L159 > This is the performance bottleneck in a simple pipeline with BigQueryIO > source. -- This message was sent by Atlassian JIRA (v6.4.14#64029)