Can you explain this use case some more? Is this a streaming pipeline? If
so, how are you reading from BigQuery?
On Thu, Feb 23, 2023 at 10:06 PM Sahil Modak via dev <dev@beam.apache.org>
wrote:
> Hi,
>
> We have a requirement wherein we are consuming input from pub/sub
> (PubSubIO) as well as BQ (BQIO)
>
> We want to make sure that we consume the BQ stream first before we start
> consuming the data from pub-sub. Is there a way to achieve this? Can you
> please help with some code samples?
>
> Currently, we read data from big query using BigQueryIO into a PCollection
> & also read data from pubsub using PubsubIO. We then use the flatten
> transform in this manner.
>
> PCollection pubsubKvPairs = reads from pubsub using PubsubIO
> PCollection bigqueryKvPairs = reads from bigquery using BigQueryIO
>
> kvPairs = PCollectionList.of(pubsubKvPairs).and(bigqueryKvPairs).apply("Merge
> Input", Flatten.pCollections());
>
>
> Thanks,
> Sahil
>
>
