Are you sure you have not produced any other data into that topic, e.g.
perhaps you were testing the regular kafka-console-producer before? This
would cause it to fail on the non-Avro messages (as Dayong says, because
the initial magic byte mismatches).
Can you try starting the consumer first without --from-beginning and then
running the producer to add more data to the topic? If that works, you can
dump the raw bytes using kafka-console-consumer to try to figure out where
the bad data came from, but I'd guess since it looks like you're just
testing you might also just want to delete the topic to get it back to a
clean state.
-Ewen
On Wed, Nov 23, 2016 at 4:27 AM, Dayong <will...@gmail.com> wrote:
> As I remember this is to complain the fist byte of msg is not x00. I think
> console producer does not support json since it uses string schema.
>
> Thanks,
> Dayong
>
> > On Nov 23, 2016, at 4:28 AM, ZHU Hua B <hua.b....@alcatel-lucent.com>
> wrote:
> >
> > Hi,
> >
> >
> > We tried to produce and consume a AVRO message (zookeeper, broker and
> schema registry have been launched), error "Unknown magic byte" occurred
> while deserializing Avro message, if I missed anything? Thanks!
> >
> >
> > From producer:
> > # bin/kafka-avro-console-producer --broker-list localhost:9092 --topic
> test --property value.schema='{"type":"record"
> ,"name":"myrecord","fields":[{"name":"f1","type":"string"}]}'
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in [jar:file:/root/confluent/shar
> e/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/root/confluent/shar
> e/java/confluent-common/slf4j-log4j12-1.7.6.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/root/confluent/shar
> e/java/schema-registry/slf4j-log4j12-1.7.6.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> > {"f1": "value1"}
> >
> >
> > From consumer:
> > # bin/kafka-avro-console-consumer --topic test --zookeeper
> localhost:2181 --from-beginning
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in [jar:file:/root/confluent/shar
> e/java/kafka-serde-tools/slf4j-log4j12-1.7.6.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/root/confluent/shar
> e/java/confluent-common/slf4j-log4j12-1.7.6.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in [jar:file:/root/confluent/shar
> e/java/schema-registry/slf4j-log4j12-1.7.6.jar!/org/slf4j/
> impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> >
> > [2016-11-23 16:15:11,108] ERROR Unknown error when running consumer:
> (kafka.tools.ConsoleConsumer$:103)
> > org.apache.kafka.common.errors.SerializationException: Error
> deserializing Avro message for id -1
> > Caused by: org.apache.kafka.common.errors.SerializationException:
> Unknown magic byte!
> > [2016-11-23 16:15:11,108] ERROR Unknown error when running consumer:
> (kafka.tools.ConsoleConsumer$:103)
> > org.apache.kafka.common.errors.SerializationException: Error
> deserializing Avro message for id -1
> > Caused by: org.apache.kafka.common.errors.SerializationException:
> Unknown magic byte!
> >
> >
> > Best Regards
> >
> > Johnny
> >
>
--
Thanks,
Ewen
