For the UnboundedSource interface, this depends on the Runner.
Generally, close() should be called when no more data will be read from
the Reader. The FlinkRunner calls `close()` on the Readers when it
closes the operator (see UnboundedSourceWrapper).
The best documentation we have for this
Got it, thanks, will double check the PubsubUnboundedSource.
btw, I'd like to learn more about the lifecycle of IO connector(for
example, when will the runner call the reader's close() method?), could you
recommend some documents, thanks!
On Fri, Oct 5, 2018 at 11:01 PM Maximilian Michels
Restoring from a checkpoint is something different. You asked about
acking pending CheckpointMarks.
If you look in the PubsubUnboundedSource, it doesn't close the client
connection if the there are still unacked checkpoints. a) Closing the
reader and b) closing the client connection, these
I've checked the PubsubUnboundedSource, it just throws an exception if it's
a "restored checkpoint".
Actually, for most MQ system, it's no way to ack a message if the
Reader(connection) is closed. So it makes no sense to call the
finalizeCheckpoint() method after closed the Reader.
On Fri, Oct
Hi,
Not sure whether I'm a guru but I'll try to answer your question ;)
Is there any way to ask the runner to call finalizeCheckpoint() method before
it closed the Reader?
Not that I'm aware of.
The point the comment is trying to make is that CheckpointMark should
not depend on the life
Hi Gurus,
I'm building a new IO connector now, and I try to ack messages in the
"UnboundedSource.CheckpointMark.finalizeCheckpoint()" method as MqttIO and
JmsIO did, but I found in the javadoc it said
> It is NOT safe to assume the UnboundedSource.UnboundedReader from which
this checkpoint was
