pnowojski commented on issue #6989: [FLINK-10455][Kafka Tx] Close transactional producers in case of failure and termination URL: https://github.com/apache/flink/pull/6989#issuecomment-435429850 Regarding your question @azagrebin, I think my bug fix for https://issues.apache.org/jira/browse/FLINK-8086 (which added "ignore ProducerFencedException") indeed might turned the @GJL's timeouts code: ``` } catch (final Exception e) { final long elapsedTime = clock.millis() - transactionHolder.transactionStartTime; if (ignoreFailuresAfterTransactionTimeout && elapsedTime > transactionTimeout) { LOG.error("Error while committing transaction {}. " + "Transaction has been open for longer than the transaction timeout ({})." + "Commit will not be attempted again. Data loss might have occurred.", transactionHolder.handle, transactionTimeout, e); } else { throw e; } } ``` into a dead code at the moment. But I think my bug fix is more important (from my commit message): > [FLINK-8086][kafka] Ignore ProducerFencedException > > during recovery ProducerFencedException can happen if we restore twice from the same checkpoint or if we restore from an old savepoint. In both cases transactional.ids that we want to recoverAndCommit have been already committed and reused. Reusing mean that they will be known by Kafka's brokers under newer producerId/epochId, which will result in ProducerFencedException if we try to commit again some old (and already committed) transaction. > > Ignoring this exception might hide some bugs/issues, because instead of failing we might have a semi silent (with a warning) data loss. I think the last sentence answers your question.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
