[ https://issues.apache.org/jira/browse/NIFI-10199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Paul Grey updated NIFI-10199: ----------------------------- Resolution: Fixed Status: Resolved (was: Patch Available) https://github.com/apache/nifi/pull/6185 > PublishKafkaRecord should not flood the logs when a batch times out > ------------------------------------------------------------------- > > Key: NIFI-10199 > URL: https://issues.apache.org/jira/browse/NIFI-10199 > Project: Apache NiFi > Issue Type: Improvement > Reporter: Daniel Urban > Assignee: Daniel Urban > Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > Currently, PublishKafkaRecord reports errors on the record level. This means > that when an error occurs on the batch level, the same error is reported for > each record of said batch. > For example, in case of a delivery timeout, a transactional Kafka producer > logs a message like this: > {code:java} > 2022-06-29 13:16:20,913 INFO > org.apache.kafka.clients.producer.internals.TransactionManager: [Producer > clientId=producer-XYZ, transactionalId=XYZ] Transiting to abortable error > state due to org.apache.kafka.common.errors.TimeoutException: Expiring 63 > record(s) for TOPIC-01:302002 ms has passed since batch creation {code} > Then, following this, we get N lines, where N is the number of records in the > batch. Also note that the TimeoutException is printed twice in this message. > {code:java} > 2022-06-29 13:16:20,913 ERROR > org.apache.nifi.processors.kafka.pubsub.PublishKafkaRecord_2_6: > PublishKafkaRecord_2_6[id=91ba351d-96ad-1d0d-934c-cfd07e0bacd5] Failed to > send > StandardFlowFileRecord[uuid=349b4d38-451d-4e62-8ea9-636f7c666fc1,claim=StandardContentClaim > [resourceClaim=StandardResourceClaim[id=1656453186665-7724448, > container=default, section=416], offset=0, > length=132021252],offset=0,name=NAME,size=132021252] to Kafka: > org.apache.kafka.common.errors.TimeoutException: Expiring 63 record(s) for > PROD_trn_lba_icode_vld_v2-38:302002 ms has passed since batch creation > org.apache.kafka.common.errors.TimeoutException: Expiring 63 record(s) for > PROD_trn_lba_icode_vld_v2-38:302002 ms has passed since batch creation{code} > Experienced this with PublishKafkaRecord_2_6, but other versions might be > affected as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)