[jira] [Commented] (KAFKA-5682) Consumer should include partition in exceptions raised during record parsing/validation
[ https://issues.apache.org/jira/browse/KAFKA-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949648#comment-16949648 ] ASF GitHub Bot commented on KAFKA-5682: --- stanislavkozlovski commented on pull request #5410: KAFKA-5682: Include partitions in exceptions raised during consumer record deserialization/validation URL: https://github.com/apache/kafka/pull/5410 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consumer should include partition in exceptions raised during record > parsing/validation > --- > > Key: KAFKA-5682 > URL: https://issues.apache.org/jira/browse/KAFKA-5682 > Project: Kafka > Issue Type: Improvement > Components: consumer >Reporter: Jason Gustafson >Assignee: Stanislav Kozlovski >Priority: Minor > Labels: needs-kip > > When we encounter an exception when validating a fetched record or when > deserializing it, we raise it to the user and keep the consumer's current > position at the offset of the failed record. The expectation is that the user > will either propagate the exception and shutdown or seek past the failed > record. However, in the latter case, there is no way for the user to know > which topic partition had the failed record. We should consider exposing an > exception type to expose this information which users can catch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-5682) Consumer should include partition in exceptions raised during record parsing/validation
[ https://issues.apache.org/jira/browse/KAFKA-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16949645#comment-16949645 ] ASF GitHub Bot commented on KAFKA-5682: --- stanislavkozlovski commented on pull request #7499: KAFKA-5682: Include partitions in exceptions raised during consumer record deserialization/validation URL: https://github.com/apache/kafka/pull/7499 This patch adds a new RecordDeserializationException which is raised when the user is expected to seek past an offset whose record is corrupted in some way This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consumer should include partition in exceptions raised during record > parsing/validation > --- > > Key: KAFKA-5682 > URL: https://issues.apache.org/jira/browse/KAFKA-5682 > Project: Kafka > Issue Type: Improvement > Components: consumer >Reporter: Jason Gustafson >Assignee: Stanislav Kozlovski >Priority: Minor > Labels: needs-kip > > When we encounter an exception when validating a fetched record or when > deserializing it, we raise it to the user and keep the consumer's current > position at the offset of the failed record. The expectation is that the user > will either propagate the exception and shutdown or seek past the failed > record. However, in the latter case, there is no way for the user to know > which topic partition had the failed record. We should consider exposing an > exception type to expose this information which users can catch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-5682) Consumer should include partition in exceptions raised during record parsing/validation
[ https://issues.apache.org/jira/browse/KAFKA-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770537#comment-16770537 ] Matthias J. Sax commented on KAFKA-5682: Moving all major/minor/trivial tickets that are not merged yet out of 2.2 release. > Consumer should include partition in exceptions raised during record > parsing/validation > --- > > Key: KAFKA-5682 > URL: https://issues.apache.org/jira/browse/KAFKA-5682 > Project: Kafka > Issue Type: Improvement > Components: consumer >Reporter: Jason Gustafson >Assignee: Stanislav Kozlovski >Priority: Minor > Labels: needs-kip > Fix For: 2.2.0 > > > When we encounter an exception when validating a fetched record or when > deserializing it, we raise it to the user and keep the consumer's current > position at the offset of the failed record. The expectation is that the user > will either propagate the exception and shutdown or seek past the failed > record. However, in the latter case, there is no way for the user to know > which topic partition had the failed record. We should consider exposing an > exception type to expose this information which users can catch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-5682) Consumer should include partition in exceptions raised during record parsing/validation
[ https://issues.apache.org/jira/browse/KAFKA-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16551483#comment-16551483 ] ASF GitHub Bot commented on KAFKA-5682: --- stanislavkozlovski opened a new pull request #5410: KAFKA-5682: Include partitions in exceptions raised during consumer record deserialization/validation URL: https://github.com/apache/kafka/pull/5410 KIP: [KIP-334](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=87297793) Mailing Thread: [thread](http://mail-archives.apache.org/mod_mbox/kafka-dev/201807.mbox/%3CCANZZNGx2jq9Xi9nxYQ6tW%2B_orkF3enOmGN13Jqgh9k5DLJLgwA%40mail.gmail.com%3E) * Introduce new Exception - `RecordDeserializationException` - it extends `SerializationException` for backwards compatibility. Throw that exception where appropriate with attached partition/offset. * Introduce new Exception - `CorruptRecordException` - it extends `KafkaException` for backwards compatibility. Throw that exception where appropriate with attached partition/offset. * Make `InvalidRecordException` extend `ApiException`. It previously extended `CorruptRecordException` and thrown only for corrupt record scenarios. (as it's not in a public package). * Add some tests for `KafkaConsumer#poll()` ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consumer should include partition in exceptions raised during record > parsing/validation > --- > > Key: KAFKA-5682 > URL: https://issues.apache.org/jira/browse/KAFKA-5682 > Project: Kafka > Issue Type: Improvement > Components: consumer >Reporter: Jason Gustafson >Assignee: Stanislav Kozlovski >Priority: Minor > Labels: needs-kip > Fix For: 2.1.0 > > > When we encounter an exception when validating a fetched record or when > deserializing it, we raise it to the user and keep the consumer's current > position at the offset of the failed record. The expectation is that the user > will either propagate the exception and shutdown or seek past the failed > record. However, in the latter case, there is no way for the user to know > which topic partition had the failed record. We should consider exposing an > exception type to expose this information which users can catch. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
