[jira] [Commented] (KAFKA-7298) Concurrent DeleteRecords can lead to fatal OutOfSequence error in producer

2018-08-20 Thread Matthias J. Sax (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-7298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16586339#comment-16586339
 ] 

Matthias J. Sax commented on KAFKA-7298:


[~hachikuji] Should this be back ported to older branches, too?

> Concurrent DeleteRecords can lead to fatal OutOfSequence error in producer
> --
>
> Key: KAFKA-7298
> URL: https://issues.apache.org/jira/browse/KAFKA-7298
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: Jason Gustafson
>Priority: Major
> Fix For: 2.0.1
>
>
> We have logic in the producer to handle unknown producer errors. Basically 
> when the producer gets an unknown producer error, it checks whether the log 
> start offset is larger than the last acknowledged offset. If it is, then we 
> know the error is spurious and we reset the sequence number to 0, which the 
> broker will then accept.
> It can happen after a DeleteRecords call, however, that the only record 
> remaining in the log is a transaction marker, which does not have a sequence 
> number. The error we get in this case is OUT_OF_SEQUENCE rather than 
> UNKNOWN_PRODUCER, which is fatal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (KAFKA-7298) Concurrent DeleteRecords can lead to fatal OutOfSequence error in producer

2018-08-16 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/KAFKA-7298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582790#comment-16582790
 ] 

ASF GitHub Bot commented on KAFKA-7298:
---

hachikuji opened a new pull request #5518: KAFKA-7298; Raise 
UnknownProducerIdException if next sequence number is unknown
URL: https://github.com/apache/kafka/pull/5518
 
 
   If the only producer state left in the log is a transaction marker, then we 
do not know the next expected sequence number. This can happen if there is a 
call to DeleteRecords which arrives prior to the writing of the marker. 
Currently we raise an OutOfOrderSequence error when this happens, but this is 
treated as a fatal error by the producer. Raising UnknownProducerId instead 
allows the producer to check for truncation using the last acknowledged 
sequence number and reset if possible.
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


> Concurrent DeleteRecords can lead to fatal OutOfSequence error in producer
> --
>
> Key: KAFKA-7298
> URL: https://issues.apache.org/jira/browse/KAFKA-7298
> Project: Kafka
>  Issue Type: Bug
>Reporter: Jason Gustafson
>Assignee: Jason Gustafson
>Priority: Major
>
> We have logic in the producer to handle unknown producer errors. Basically 
> when the producer gets an unknown producer error, it checks whether the log 
> start offset is larger than the last acknowledged offset. If it is, then we 
> know the error is spurious and we reset the sequence number to 0, which the 
> broker will then accept.
> It can happen after a DeleteRecords call, however, that the only record 
> remaining in the log is a transaction marker, which does not have a sequence 
> number. The error we get in this case is OUT_OF_SEQUENCE rather than 
> UNKNOWN_PRODUCER, which is fatal.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)