[
https://issues.apache.org/jira/browse/FLINK-19899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223555#comment-17223555
]
Danny Cranmer commented on FLINK-19899:
---------------------------------------
[~tzulitai] as discussed, I will publish a PR soon.
> [Kinesis][EFO] Optimise error handling to use a separate exception delivery
> mechanism
> -------------------------------------------------------------------------------------
>
> Key: FLINK-19899
> URL: https://issues.apache.org/jira/browse/FLINK-19899
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kinesis
> Reporter: Danny Cranmer
> Priority: Major
> Fix For: 1.12.0
>
>
> *Background*
> There is a queue used to pass events between the network client and consumer
> application. When an error is thrown in the network thread, the queue is
> cleared to make space for the error event. This means that records will be
> thrown away to make space for errors (the records would be subsequently
> reloaded from the shard).
> *Scope*
> Add a new mechanism to pass exceptions between threads, meaning data does not
> need to be discarded. When an error is thrown, the error event will be
> processed by the consumer once all of the records have been processed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
