These utilities are not a part of Beam public API and can change at any point. Beam transform for writing to BigQuery by default expects a PCollection of TableRow objects as input but can also take a format function that converts a given type into TableRow objects. https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L1489
So please consider either providing an appropriate format function or converting your PCollection<T> into a PCollection<TableRow> using a ParDo before the BigQueryIO.write() step. Thanks, Cham On Fri, Jun 14, 2019 at 12:55 PM Theodore Siu <[email protected]> wrote: > Hi Everyone, > > I am trying to insert Avro data into BigQuery using BigQueryIO. We use > beam because we have additional hydration steps for our avro files and we > use a group by in order to set up multiple load jobs in a window. I see a > direct conversion of Avro into a BigQuery TableRow but they are protected > and only available to Apache Beam Package, (see > https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryAvroUtils.java > ) > > The public class seems to want the user to go from Avro(Generic Record) -> > Beam Row -> TableRow ( > https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java). > I am guessing this is the suggested path of conversion but is there a > specific reason for this method? > > Thank you for your time. > -Theo >
