How are you putting data onto the Topic? The HdfsSink expects that you used the KafkaAvroSerializer (http://docs.confluent.io/1.0/schema-registry/docs/serializer-formatter.html) which prepends a null byte and schema registry id to the front of the serialized avro data. If you just put avro onto the topic you will see this error.
-Dave Dave Tauzell | Senior Software Engineer | Surescripts O: 651.855.3042 | www.surescripts.com | dave.tauz...@surescripts.com Connect with us: Twitter I LinkedIn I Facebook I YouTube -----Original Message----- From: Sungwook Yoon [mailto:sy...@maprtech.com] Sent: Thursday, June 23, 2016 1:46 PM To: users@kafka.apache.org Subject: Failed to deserialize data to Avro: Hi, I am testing kafka connect and got this error, Exception in thread "WorkerSinkTask-local-file-sink-0" org.apache.kafka.connect.errors.DataException: Failed to deserialize data to Avro: at io.confluent.connect.avro.AvroConverter.toConnectData(AvroConverter.java:109) at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:266) at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:175) at org.apache.kafka.connect.runtime.WorkerSinkTaskThread.iteration(WorkerSinkTaskThread.java:90) at org.apache.kafka.connect.runtime.WorkerSinkTaskThread.execute(WorkerSinkTaskThread.java:58) at org.apache.kafka.connect.util.ShutdownableThread.run(ShutdownableThread.java:82) Caused by: org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id -1 Caused by: org.apache.kafka.common.errors.SerializationException: Unknown magic byte! I am using Kafka Connect 2.0.0 Kafka 0.9 with Schema Registry Running for Avro Schema Serving. Why am I getting this error? I believe the schema registry server is running fine, since I checked its running through other means. Any insight would be helpful. Thanks, Sungwook This e-mail and any files transmitted with it are confidential, may contain sensitive information, and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender by reply e-mail immediately and destroy all copies of the e-mail and any attachments.
