[
https://issues.apache.org/jira/browse/FLINK-23495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392198#comment-17392198
]
Nico Kruber commented on FLINK-23495:
-------------------------------------
If you [look through the
code|https://github.com/apache/flink/blob/99c2a415e9eeefafacf70762b6f54070f7911ceb/flink-connectors/flink-connector-gcp-pubsub/src/main/java/org/apache/flink/streaming/connectors/gcp/pubsub/common/AcknowledgeOnCheckpoint.java#L31],
you will find the reason for requiring checkpoints:
{quote}
The mechanism for this source assumes that messages are identified by a unique
ID. When
messages are taken from the message queue, the message must not be dropped
immediately from the
external system, but must be retained until acknowledged. Messages that are not
acknowledged
within a certain time interval will be served again (to a different connection,
established by
the recovered source).
{quote}
If you do not have checkpoints, it looks like you'd have to acknowledge
messages in a different way, e.g. when consuming them. In that case, the source
would be at-most-once just like the Flink job, so that shouldn't be an issue.
If you just remove the check, then all messages are unacknowledged and should
be delivered again some time - probably not what you want.
> [GCP PubSub] Make checkpoint optional for preview/staging mode
> --------------------------------------------------------------
>
> Key: FLINK-23495
> URL: https://issues.apache.org/jira/browse/FLINK-23495
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Google Cloud PubSub
> Affects Versions: 1.13.0, 1.13.1
> Reporter: Brachi Packter
> Priority: Major
> Labels: pull-request-available
>
> I'm using PubSub connector with Flink sql.
> The issue that I get all the time error that PubSub required checkpoints, My
> question is if I/you can submit a PR that adds a property that can configure
> PubSub to start without checkpoints, and we can describe that it is just for
> preview/staging mode (interactive sql, Jupiter..)
> Other connectors support starting without checkpoints.
> What will be the impact for this change? I tried it locally and it seems to
> work ok.
> That is the code that always fail the source if no checkpoint is configured,
> i want to add some condition here:
> {code:java}
> if (hasNoCheckpointingEnabled(getRuntimeContext())) {
> throw new IllegalArgumentException( "The PubSubSource REQUIRES Checkpointing
> to be enabled and " + "the checkpointing frequency must be MUCH lower than
> the PubSub timeout for it to retry a message.");
> }
>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
