Hi,
I agree with Ryan, even if clients might be totally different the
backend technologies are the same so hosting them in the same repo makes
sense. Similar thinking made us put all the Cassandra related connectors
in the same cassandra repo.
Etienne
Le 02/03/2023 à 14:43, Daniel Collins a écrit :
Hello Ryan,
Unfortunately there's not much shared logic between the two- the clients
have to look fundamentally different since the Pub/Sub Lite client exposes
partitions to the split level for repeatable reads.
I have no objection to this living in the same repo as the Pub/Sub
connector, if this is an easier way forward than setting up a new repo,
sounds good to me. The Pub/Sub team is organizationally close to us, and is
looking into providing more support for the flink connector in the near
future.
-Daniel
On Thu, Mar 2, 2023 at 3:26 AM Ryan Skraba <ryan.skr...@aiven.io.invalid>
wrote:
Hello Daniel! Quite a while ago, I started porting the Pub/Sub connector
(from an existing PR) to the new source API in the new
flink-connector-gcp-pubsub repository [PR2]. As Martijn mentioned, there
hasn't been a lot of attention on this connector; any community involvement
would be appreciated!
Instead of considering this a new connector, is there an opportunity here
to offer the two variants (Pub/Sub and Pub/Sub Lite) as different artifacts
in that same repo? Is there much common logic that can be shared between
the two? I'm not as familiar as I should be with Lite, but I do recall
that they share many concepts and _some_ dependencies.
All my best, Ryan
On Wed, Mar 1, 2023 at 11:21 PM Daniel Collins
<dpcoll...@google.com.invalid>
wrote:
Hello all,
I'd like to start an official discuss thread for adding a Pub/Sub Lite
Connector to Flink. We've had requests from our users to add flink
support,
and are willing to maintain and support this connector long term from the
product team.
The proposal is https://cwiki.apache.org/confluence/x/P51bDg, what would
be
people's thoughts on adding this connector?
-Daniel
