[
https://issues.apache.org/jira/browse/FLINK-9311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466409#comment-16466409
]
Richard Deurwaarder edited comment on FLINK-9311 at 5/7/18 8:06 PM:
--------------------------------------------------------------------
PubSub works more like RabbitMQ than Kafka using ack/nack. It has
[atleast-once-delivery|https://cloud.google.com/pubsub/docs/subscriber], but
using a [message
id|http://googleapis.github.io/googleapis/java/all/latest/apidocs/com/google/pubsub/v1/PubsubMessageOrBuilder.html#getMessageId--]
exactly-once can be achived.
Is there an existing connector using message id's for deduplication?
was (Author: xeli):
PubSub works more like RabbitMQ than Kafka using ack/nack. It has
atleast-once-delivery, but using a [message
id|http://googleapis.github.io/googleapis/java/all/latest/apidocs/com/google/pubsub/v1/PubsubMessageOrBuilder.html#getMessageId--]
exactly-once can be achived.
Is there an existing connector using message id's for deduplication?
> PubSub connector
> ----------------
>
> Key: FLINK-9311
> URL: https://issues.apache.org/jira/browse/FLINK-9311
> Project: Flink
> Issue Type: New Feature
> Components: Streaming Connectors
> Reporter: Richard Deurwaarder
> Priority: Minor
>
> I would like start adding some google cloud connectors starting with a PubSub
> Source. I have a basic implementation ready but I want it to be able to:
> * easily scale up (should I have it extend RichParallelSourceFunction for
> this?)
> * Make it easier to provide the google cloud credentials. This would require
> being able to send some json string / ServiceAccount to the nodes when
> starting up this source.
> Could this be something that would be useful for others and added to the
> flink connectors repo?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
