Cham, Do we have JIRAs tracking these features?
Reuven On Wed, Aug 23, 2017 at 6:26 PM, Asha Rostamianfar < [email protected]> wrote: > It's for a batch job. I'll think about possible workarounds then (perhaps I > can have a prestage process that reads the necessary info from the files > and passes as an argument). Thanks everyone! > > On Wed, Aug 23, 2017 at 9:17 PM, Chamikara Jayalath <[email protected]> > wrote: > > > Ah, actually for this feature I think we need a BigQuery custom sink. > Which > > also doesn't have an ETA currently. > > > > Thanks, > > Cham > > > > On Wed, Aug 23, 2017 at 6:14 PM Chamikara Jayalath <[email protected] > > > > wrote: > > > > > We have to develop a BigQuery BoundedSource for this first. So I expect > > > support for Python SDK to be at least few months away. > > > > > > Thanks, > > > Cham > > > > > > > > > On Wed, Aug 23, 2017 at 6:10 PM Asha Rostamianfar > > > <[email protected]> wrote: > > > > > >> ah, ok..is there an existing JIRA entry with some approximate ETA? > (i.e. > > >> weeks, months, or quarters). > > >> > > >> Thanks again. > > >> > > >> On Wed, Aug 23, 2017 at 8:58 PM, Reuven Lax <[email protected] > > > > >> wrote: > > >> > > >> > Not yet, only in Java so far. > > >> > > > >> > On Wed, Aug 23, 2017 at 5:53 PM, Asha Rostamianfar < > > >> > [email protected]> wrote: > > >> > > > >> > > Thanks, Eugene! Is this also supported in the python SDK? > > >> > > > > >> > > On Wed, Aug 23, 2017 at 8:39 PM, Eugene Kirpichov < > > >> > > [email protected]> wrote: > > >> > > > > >> > > > Yes, this is possible using the BigQueryIO.write().to( > > >> > > DynamicDestinations) > > >> > > > API. It allows you to write different values to different tables > > >> with > > >> > > > different schemas. > > >> > > > On Wed, Aug 23, 2017, 2:51 PM Asha Rostamianfar > > >> > > > <[email protected]> > > >> > > > wrote: > > >> > > > > > >> > > > > Hi, > > >> > > > > > > >> > > > > I'm wondering whether it's possible to dynamically generate a > > >> > BigQuery > > >> > > > > schema based on input. For instance, the fields would be > > >> specified in > > >> > > one > > >> > > > > or more input files that are read and processed as part of the > > >> > > pipeline. > > >> > > > > > > >> > > > > Thanks, > > >> > > > > Asha > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > >
