I'm building a high-volume Beam pipeline using PubsubIO and running into
some concerns over performance and delivery semantics, prompting me to want
to better understand the implementation. Reading through the library,
PubsubIO appears to be a completely separate implementation of Pubsub
client behavior from Google's own Java client. As a developer trying to
read and understand the implementation, this is a significant hurdle, since
any previous knowledge of the Google library is not applicable and is
potentially at odds with what's in PubsubIO.

Why doesn't beam use the Google clients for PubsubIO, BigQueryIO, etc.? Is
it for historical reasons? Is there difficulty in packaging and integration
of the Google clients? Or are the needs for Beam just substantially
different from what the Google libraries provide?

Reply via email to