[jira] [Commented] (KAFKA-7332) Improve error message when trying to produce message without key for compacted topic
[ https://issues.apache.org/jira/browse/KAFKA-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1662#comment-1662 ] Ismael Juma commented on KAFKA-7332: I think fixing the underlying issue would be good. > Improve error message when trying to produce message without key for > compacted topic > > > Key: KAFKA-7332 > URL: https://issues.apache.org/jira/browse/KAFKA-7332 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 1.1.0 >Reporter: Patrik Kleindl >Assignee: Manikumar >Priority: Trivial > Fix For: 2.1.0 > > > Goal: > Return a specific error message like e.g. "Message without a key is not valid > for a compacted topic" when trying to produce such a message instead of a > CorruptRecordException. > > > Yesterday we had the following exception: > > > > Exception thrown when sending a message with key='null' and payload='...' > > to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: > > This message has failed its CRC checksum, exceeds the valid size, or is > > otherwise corrupt. > > > > The cause was identified with the help of > > > >[https://stackoverflow.com/questions/49098274/kafka-stream-get-corruptrecordexception] > > > > Is it possible / would it makes sense to open an issue to improve the error > > message for this case? > > A simple "Message without a key is not valid for a compacted topic" would > > suffice and point a user in the right direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7332) Improve error message when trying to produce message without key for compacted topic
[ https://issues.apache.org/jira/browse/KAFKA-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621106#comment-16621106 ] James Cheng commented on KAFKA-7332: Thanks [~lindong]. I agree with you that design-wise it makes sense, and practice-wise, I'm not sure. KAFKA-4808 says that you need to exhaust retries or request.timeout.ms expires. I'm not sure how those two interact. Is the request.timeout.ms a per-request thing and so you actually have to wait for (retries * request.timeout.ms)? According to [https://docs.confluent.io/current/streams/developer-guide/config-streams.html,] kafka streams has retries = 10. And in kafka 2.0.0, producer request.timeout.ms = 30 seconds. If you multiple those together, you get 5 minutes until you get a timeout. As you said, it's probably small compared to working through other workflow stuff, but 5 minutes is quite a while to wait to get an error. I would probably end up killing my application repeatedly thinking that something else was wrong, and might not ever get to the 5 minutes. I don't think I've personally run into this. I'm just thinking through what the developer experience would be. [~gwenshap] ran into it recently, but again, not sure how much more widely this happens. > Improve error message when trying to produce message without key for > compacted topic > > > Key: KAFKA-7332 > URL: https://issues.apache.org/jira/browse/KAFKA-7332 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 1.1.0 >Reporter: Patrik Kleindl >Assignee: Manikumar >Priority: Trivial > Fix For: 2.1.0 > > > Goal: > Return a specific error message like e.g. "Message without a key is not valid > for a compacted topic" when trying to produce such a message instead of a > CorruptRecordException. > > > Yesterday we had the following exception: > > > > Exception thrown when sending a message with key='null' and payload='...' > > to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: > > This message has failed its CRC checksum, exceeds the valid size, or is > > otherwise corrupt. > > > > The cause was identified with the help of > > > >[https://stackoverflow.com/questions/49098274/kafka-stream-get-corruptrecordexception] > > > > Is it possible / would it makes sense to open an issue to improve the error > > message for this case? > > A simple "Message without a key is not valid for a compacted topic" would > > suffice and point a user in the right direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7332) Improve error message when trying to produce message without key for compacted topic
[ https://issues.apache.org/jira/browse/KAFKA-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621050#comment-16621050 ] Dong Lin commented on KAFKA-7332: - [~wushujames] The problem described in KAFKA-4808 says that user today has to wait for timeout if they produce to a compacted topic without key and it should be better to throw non-retriable exception immediately without timeout. From design perspective, I think the idea make sense and it is good to address that. In terms of its impact in practice, I personally don't find that issue to be important because in all cases I can think of (please correct me if I am wrong), the issue only happens the first time when user setups a new workflow and tries to produce to a compacted topic with null key. It is probably OK for user to spend a minute or so to receive this first error given that it probably takes much longer time to create and debug a new workflow anyway. Thus it may not be worthwhile to add a new exception to address that problem. So it is nice to address KAFKA-4808 if other developers are interested in pushing progress for KAFKA-4808. > Improve error message when trying to produce message without key for > compacted topic > > > Key: KAFKA-7332 > URL: https://issues.apache.org/jira/browse/KAFKA-7332 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 1.1.0 >Reporter: Patrik Kleindl >Assignee: Manikumar >Priority: Trivial > Fix For: 2.1.0 > > > Goal: > Return a specific error message like e.g. "Message without a key is not valid > for a compacted topic" when trying to produce such a message instead of a > CorruptRecordException. > > > Yesterday we had the following exception: > > > > Exception thrown when sending a message with key='null' and payload='...' > > to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: > > This message has failed its CRC checksum, exceeds the valid size, or is > > otherwise corrupt. > > > > The cause was identified with the help of > > > >[https://stackoverflow.com/questions/49098274/kafka-stream-get-corruptrecordexception] > > > > Is it possible / would it makes sense to open an issue to improve the error > > message for this case? > > A simple "Message without a key is not valid for a compacted topic" would > > suffice and point a user in the right direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7332) Improve error message when trying to produce message without key for compacted topic
[ https://issues.apache.org/jira/browse/KAFKA-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621021#comment-16621021 ] James Cheng commented on KAFKA-7332: [~lindong], this is somewhat related to https://issues.apache.org/jira/browse/KAFKA-4808. While we are making this change, should we also address that JIRA? > Improve error message when trying to produce message without key for > compacted topic > > > Key: KAFKA-7332 > URL: https://issues.apache.org/jira/browse/KAFKA-7332 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 1.1.0 >Reporter: Patrik Kleindl >Assignee: Manikumar >Priority: Trivial > Fix For: 2.1.0 > > > Goal: > Return a specific error message like e.g. "Message without a key is not valid > for a compacted topic" when trying to produce such a message instead of a > CorruptRecordException. > > > Yesterday we had the following exception: > > > > Exception thrown when sending a message with key='null' and payload='...' > > to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: > > This message has failed its CRC checksum, exceeds the valid size, or is > > otherwise corrupt. > > > > The cause was identified with the help of > > > >[https://stackoverflow.com/questions/49098274/kafka-stream-get-corruptrecordexception] > > > > Is it possible / would it makes sense to open an issue to improve the error > > message for this case? > > A simple "Message without a key is not valid for a compacted topic" would > > suffice and point a user in the right direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7332) Improve error message when trying to produce message without key for compacted topic
[ https://issues.apache.org/jira/browse/KAFKA-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16618638#comment-16618638 ] ASF GitHub Bot commented on KAFKA-7332: --- lindong28 closed pull request #5638: KAFKA-7332: Update CORRUPT_MESSAGE exception message description URL: https://github.com/apache/kafka/pull/5638 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/clients/src/main/java/org/apache/kafka/common/protocol/Errors.java b/clients/src/main/java/org/apache/kafka/common/protocol/Errors.java index a796ec912cb..082eff12461 100644 --- a/clients/src/main/java/org/apache/kafka/common/protocol/Errors.java +++ b/clients/src/main/java/org/apache/kafka/common/protocol/Errors.java @@ -116,7 +116,7 @@ NONE(0, null, message -> null), OFFSET_OUT_OF_RANGE(1, "The requested offset is not within the range of offsets maintained by the server.", OffsetOutOfRangeException::new), -CORRUPT_MESSAGE(2, "This message has failed its CRC checksum, exceeds the valid size, or is otherwise corrupt.", +CORRUPT_MESSAGE(2, "This message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt.", CorruptRecordException::new), UNKNOWN_TOPIC_OR_PARTITION(3, "This server does not host this topic-partition.", UnknownTopicOrPartitionException::new), This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Improve error message when trying to produce message without key for > compacted topic > > > Key: KAFKA-7332 > URL: https://issues.apache.org/jira/browse/KAFKA-7332 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 1.1.0 >Reporter: Patrik Kleindl >Assignee: Manikumar >Priority: Trivial > Fix For: 2.1.0 > > > Goal: > Return a specific error message like e.g. "Message without a key is not valid > for a compacted topic" when trying to produce such a message instead of a > CorruptRecordException. > > > Yesterday we had the following exception: > > > > Exception thrown when sending a message with key='null' and payload='...' > > to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: > > This message has failed its CRC checksum, exceeds the valid size, or is > > otherwise corrupt. > > > > The cause was identified with the help of > > > >[https://stackoverflow.com/questions/49098274/kafka-stream-get-corruptrecordexception] > > > > Is it possible / would it makes sense to open an issue to improve the error > > message for this case? > > A simple "Message without a key is not valid for a compacted topic" would > > suffice and point a user in the right direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-7332) Improve error message when trying to produce message without key for compacted topic
[ https://issues.apache.org/jira/browse/KAFKA-7332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610999#comment-16610999 ] ASF GitHub Bot commented on KAFKA-7332: --- omkreddy opened a new pull request #5638: KAFKA-7332: Update CORRUPT_MESSAGE exception message description URL: https://github.com/apache/kafka/pull/5638 ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Improve error message when trying to produce message without key for > compacted topic > > > Key: KAFKA-7332 > URL: https://issues.apache.org/jira/browse/KAFKA-7332 > Project: Kafka > Issue Type: Improvement > Components: producer >Affects Versions: 1.1.0 >Reporter: Patrik Kleindl >Priority: Trivial > > Goal: > Return a specific error message like e.g. "Message without a key is not valid > for a compacted topic" when trying to produce such a message instead of a > CorruptRecordException. > > > Yesterday we had the following exception: > > > > Exception thrown when sending a message with key='null' and payload='...' > > to topic sometopic:: org.apache.kafka.common.errors.CorruptRecordException: > > This message has failed its CRC checksum, exceeds the valid size, or is > > otherwise corrupt. > > > > The cause was identified with the help of > > > >[https://stackoverflow.com/questions/49098274/kafka-stream-get-corruptrecordexception] > > > > Is it possible / would it makes sense to open an issue to improve the error > > message for this case? > > A simple "Message without a key is not valid for a compacted topic" would > > suffice and point a user in the right direction. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
