Thanks Pablo, our goal is to take advantage of google dataflow to sync mongo and bigquery.
Something like this: https://github.com/snarvaez/Atlas-BQ__ODS-EDW But with a direct approach mongo > bigquery rather than involving Pub/Sub I have found this attempt https://github.com/alexvanboxel/beam-demo/blob/master/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbChangeStreamFn.java And I was wondering if there is any official initiative to build an official connector. On 2021/10/18 21:27:04, Pablo Estrada <pabl...@google.com> wrote: > Hi David, > > Debezium supports reading change streams from MongoDB[1] - you may be able > to use DebeziumIO to consume these [2]. Have you considered that? > Best > -P/ > > [1] https://debezium.io/documentation/reference/connectors/mongodb.html > [2] > https://beam.apache.org/releases/javadoc/current/org/apache/beam/io/debezium/DebeziumIO.html > > On Mon, Oct 18, 2021 at 2:03 PM David Morales de Frias < > david.mora...@syncrtc.com> wrote: > > > Hello everyone, > > > > I would like to read data from change streams to build real-time pipelines > > and I have not found anything related to it. > > > > Is this connector being considered? > > > > Thanks > > >
