Greg Harris created KAFKA-10273:
-----------------------------------
Summary: Connect Converters should produce actionable error
messages
Key: KAFKA-10273
URL: https://issues.apache.org/jira/browse/KAFKA-10273
Project: Kafka
Issue Type: Improvement
Components: KafkaConnect
Reporter: Greg Harris
When runtime exceptions are thrown from converters, the current error messages
are descriptive, but not actionable. For example:
{noformat}
Error encountered in task task-0. Executing stage 'VALUE_CONVERTER' with class
'org.apache.kafka.connect.json.JsonConverter', where consumed record is
<snipped>
Caused by: org.apache.kafka.connect.errors.DataException: Converting byte[] to
Kafka Connect data failed due to serialization error:
at
org.apache.kafka.connect.json.JsonConverter.toConnectData(JsonConverter.java:334)
<snipped>
Caused by: org.apache.kafka.common.errors.SerializationException:
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('b' (code
98)): was expecting double-quote to start field name
at [Source: (byte[])"{brokenjson-:"bar 1"}"; line: 1, column: 3]{noformat}
This describes the error accurately, explaining when it happened, and what
caused the exception to be thrown. However, after receiving this error it's
still unclear what remediation step to take next.
Instead, in addition to the existing descriptive context and instigating error
being displayed, these error messages should give users a lead to begin
investigating and remediating the error.
There are a few changes that users may use to remediate exceptions from
converters:
* Reconfigure the converter (change schemas.enable on the JsonConverter to
align with the topic contents)
* Use different data (advancing sink/source offsets manually to skip bad
records, or changing topics)
* Remediate the converter's environment failure (resolving network faults or
reconfiguring external systems)
An error message which suggests manual inspection of the converter
configuration would be user-actionable, and could be generically applied across
converters without knowledge of specific configurations. This would point users
towards the most common remediation step, reconfiguring the converter.
We can also indicate that bad data in a topic may be skipped in a number of
ways, such as changing offsets, ignoring errors, writing to a DLQ, etc. It's
unclear which of these is worth suggesting, and it may change if an alternative
strategy feature exists in the future.
Non-Goals/ Out-of-scope:
The third type of remediation of the converter's environment is outside the
scope of this issue, and requires converter-specific knowledge in order to make
a suggestion.
This change is not attempting to add converter-specific error messaging, and
the framework messaging here needs to be converter-agnostic, and as such can
only point out general classes of errors and remediation steps, not specific
ones. We can programmatically generate lists of relevant configurations, but
can't single any out as being the cause. We can say that the data is
incompatible with the converter, but can't say why exactly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
