No, it only needs to ensure that one record seen on Pubsub has successfully written to a database. So "record by record" is fine, or even "bundle".
*~Vincent* On Wed, Mar 24, 2021 at 9:49 AM Alexey Romanenko <aromanenko....@gmail.com> wrote: > Do you want to wait for ALL records are written for Cassandra and then > write all successfully written records to PubSub or it should be performed > "record by record"? > > On 24 Mar 2021, at 04:58, Vincent Marquez <vincent.marq...@gmail.com> > wrote: > > I have a common use case where my pipeline looks like this: > CassandraIO.readAll -> Aggregate -> CassandraIO.write -> PubSubIO.write > > I do NOT want my pipeline to look like the following: > > CassandraIO.readAll -> Aggregate -> CassandraIO.write > | > -> PubsubIO.write > > Because I need to ensure that only items written to Pubsub have > successfully finished a (quorum) write. > > Since CassandraIO.write is a PTransform<A, PDone> I can't actually use it > here so I often roll my own 'writer', but maybe there is a recommended way > of doing this? > > Thanks in advance for any help. > > *~Vincent* > > >
