I am able to produce Avro from producer for pageview.avsc but not for
userprofile.avsc. Here are two samples below, first one leaves out Avro
optional experience, second one specifies it.
Note: I do not register using REST curl calls schemas to schema-registry as
I assume that kafka-avro-console-producer takes care of that.
First case has Avro disagreeing that experience can be optional with an
exception:
AvroTypeException: Expected field name not found: experience
Second case generates another AvroTypeException complaining about the union
type:
Caused by: org.apache.avro.AvroTypeException: Expected start-union. Got
VALUE_STRING
./bin/kafka-avro-console-producer --broker-list localhost:9092 --topic
UserProfile --property value.schema='{
"type": "record",
"name": "UserProfile",
"fields": [
{"name": "experience", "type": ["string", "null"]},
{"name": "region", "type": "string"}
]
}'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Applications/confluent-3.0.0/share/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Applications/confluent-3.0.0/share/java/confluent-common/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Applications/confluent-3.0.0/share/java/schema-registry/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
{"region":"US"}
org.apache.kafka.common.errors.SerializationException: Error deserializing
json {"region":"US"} to Avro of schema
{"type":"record","name":"UserProfile","fields":[{"name":"experience","type":["string","null"]},{"name":"region","type":"string"}]}
Caused by: org.apache.avro.AvroTypeException: Expected field name not
found: experience
at org.apache.avro.io.JsonDecoder.doAction(JsonDecoder.java:477)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.JsonDecoder.advance(JsonDecoder.java:139)
at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:429)
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155)
at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
at
io.confluent.kafka.formatter.AvroMessageReader.jsonToAvro(AvroMessageReader.java:189)
at
io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:157)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:55)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
./bin/kafka-avro-console-producer --broker-list localhost:9092 --topic
UserProfile --property value.schema='{
"type": "record",
"name": "UserProfile",
"fields": [
{"name": "experience", "type": ["string", "null"]},
{"name": "region", "type": "string"}
]
}'
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/Applications/confluent-3.0.0/share/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Applications/confluent-3.0.0/share/java/confluent-common/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/Applications/confluent-3.0.0/share/java/schema-registry/slf4j-log4j12-1.7.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
{"region":"US","experience":"scala"}
org.apache.kafka.common.errors.SerializationException: Error deserializing
json {"region":"US","experience":"scala"} to Avro of schema
{"type":"record","name":"UserProfile","fields":[{"name":"experience","type":["string","null"]},{"name":"region","type":"string"}]}
Caused by: org.apache.avro.AvroTypeException: Expected start-union. Got
VALUE_STRING
at org.apache.avro.io.JsonDecoder.error(JsonDecoder.java:697)
at org.apache.avro.io.JsonDecoder.readIndex(JsonDecoder.java:441)
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155)
at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
at
io.confluent.kafka.formatter.AvroMessageReader.jsonToAvro(AvroMessageReader.java:189)
at
io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:157)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:55)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
