[
https://issues.apache.org/jira/browse/FLINK-9295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472056#comment-16472056
]
ASF GitHub Bot commented on FLINK-9295:
---------------------------------------
Github user StephanEwen commented on a diff in the pull request:
https://github.com/apache/flink/pull/5977#discussion_r187639708
--- Diff:
flink-connectors/flink-connector-kafka-0.11/src/main/java/org/apache/flink/streaming/connectors/kafka/FlinkKafkaProducer011.java
---
@@ -837,7 +837,7 @@ public void
initializeState(FunctionInitializationContext context) throws Except
nextTransactionalIdHintState =
context.getOperatorStateStore().getUnionListState(
NEXT_TRANSACTIONAL_ID_HINT_DESCRIPTOR);
transactionalIdsGenerator = new TransactionalIdsGenerator(
- getRuntimeContext().getTaskName(),
+ getRuntimeContext().getTaskName() + "-" +
getRuntimeContext().getOperatorUniqueID(),
--- End diff --
You could probably use only the Operator ID here - the task name does not
add to the uniqueness. Unless the purpose of the task name is "human
readability" in log files or metrics.
> FlinkKafkaProducer011 sometimes throws ProducerFencedExceptions when in
> EXACTLY_ONCE mode
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-9295
> URL: https://issues.apache.org/jira/browse/FLINK-9295
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector
> Affects Versions: 1.4.2
> Reporter: Christopher Ng
> Assignee: Piotr Nowojski
> Priority: Major
> Fix For: 1.5.0
>
>
> {{FlinkKafkaProducer011}} can throw {{ProducerFencedExceptions}} if multiple
> sinks are used within the same sub-task. This can happen when
> operator-chaining results in two different sinks in the same topology being
> chained into a task, and thus into each of its sub-tasks.
> The problem is that {{TransactionIdsGenerator}} only takes into account the
> task name, the subtask index, the number of subtasks, and a couple of other
> things. All the attributes are the same between different
> {{FlinkKafkaProducer011s}} within the same sub-task, so they get the same
> transaction ids and one of them ends up failing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
