Is this for a batch pipeline or a streaming pipeline? I don't believe that Python yet has good support for BigQuery in batch jobs at all.
On Wed, Aug 23, 2017 at 6:10 PM, Asha Rostamianfar < [email protected]> wrote: > ah, ok..is there an existing JIRA entry with some approximate ETA? (i.e. > weeks, months, or quarters). > > Thanks again. > > On Wed, Aug 23, 2017 at 8:58 PM, Reuven Lax <[email protected]> > wrote: > > > Not yet, only in Java so far. > > > > On Wed, Aug 23, 2017 at 5:53 PM, Asha Rostamianfar < > > [email protected]> wrote: > > > > > Thanks, Eugene! Is this also supported in the python SDK? > > > > > > On Wed, Aug 23, 2017 at 8:39 PM, Eugene Kirpichov < > > > [email protected]> wrote: > > > > > > > Yes, this is possible using the BigQueryIO.write().to( > > > DynamicDestinations) > > > > API. It allows you to write different values to different tables with > > > > different schemas. > > > > On Wed, Aug 23, 2017, 2:51 PM Asha Rostamianfar > > > > <[email protected]> > > > > wrote: > > > > > > > > > Hi, > > > > > > > > > > I'm wondering whether it's possible to dynamically generate a > > BigQuery > > > > > schema based on input. For instance, the fields would be specified > in > > > one > > > > > or more input files that are read and processed as part of the > > > pipeline. > > > > > > > > > > Thanks, > > > > > Asha > > > > > > > > > > > > > > >
