[
https://issues.apache.org/jira/browse/KAFKA-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313442#comment-17313442
]
Karim Lamouri commented on KAFKA-8314:
--------------------------------------
We have the same problem on Kafka 2.5.1.
Changing the doc cause issues with Kafka-Connect
{code:java}
org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to
unrecoverable exception.
at
org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:568)
at
org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:326)
at
org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:229)
at
org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:201)
at
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:185)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:235)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.connect.errors.ConnectException:
org.apache.kafka.connect.errors.SchemaProjectorException: Error projecting
auction
at
io.confluent.connect.s3.TopicPartitionWriter.write(TopicPartitionWriter.java:190)
at io.confluent.connect.s3.S3SinkTask.put(S3SinkTask.java:191)
at
org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:546)
... 10 more
Caused by: org.apache.kafka.connect.errors.SchemaProjectorException: Error
projecting auction
at
org.apache.kafka.connect.data.SchemaProjector.projectStruct(SchemaProjector.java:113)
at
org.apache.kafka.connect.data.SchemaProjector.projectRequiredSchema(SchemaProjector.java:93)
at
org.apache.kafka.connect.data.SchemaProjector.project(SchemaProjector.java:73)
at
io.confluent.connect.storage.schema.StorageSchemaCompatibility.projectInternal(StorageSchemaCompatibility.java:395)
at
io.confluent.connect.storage.schema.StorageSchemaCompatibility.projectInternal(StorageSchemaCompatibility.java:383)
at
io.confluent.connect.storage.schema.StorageSchemaCompatibility.project(StorageSchemaCompatibility.java:355)
at
io.confluent.connect.s3.TopicPartitionWriter.checkRotationOrAppend(TopicPartitionWriter.java:269)
at
io.confluent.connect.s3.TopicPartitionWriter.executeState(TopicPartitionWriter.java:219)
at
io.confluent.connect.s3.TopicPartitionWriter.write(TopicPartitionWriter.java:188)
... 12 more
Caused by: org.apache.kafka.connect.errors.SchemaProjectorException: Error
projecting site
at
org.apache.kafka.connect.data.SchemaProjector.projectStruct(SchemaProjector.java:113)
at
org.apache.kafka.connect.data.SchemaProjector.projectRequiredSchema(SchemaProjector.java:93)
at
org.apache.kafka.connect.data.SchemaProjector.project(SchemaProjector.java:73)
at
org.apache.kafka.connect.data.SchemaProjector.projectStruct(SchemaProjector.java:110)
... 20 more
Caused by: org.apache.kafka.connect.errors.SchemaProjectorException: Schema
parameters not equal. source parameters: XXXX
{code}
With the only difference between the source and the target parameters being the
addition of:
{{io.confluent.connect.avro.field.doc.FieldName=new documentation for new
field}}
Only way to fix this situation is to have schema evolution set to NONE in
Kafka-Connect
> Managing the doc field in case of schema projection - kafka connect
> -------------------------------------------------------------------
>
> Key: KAFKA-8314
> URL: https://issues.apache.org/jira/browse/KAFKA-8314
> Project: Kafka
> Issue Type: Bug
> Reporter: kaushik srinivas
> Priority: Major
>
> Doc field change in the schema while writing to hdfs using hdfs sink
> connector via connect framework would cause failures in schema projection.
>
> java.lang.RuntimeException:
> org.apache.kafka.connect.errors.SchemaProjectorException: Schema parameters
> not equal. source parameters: \{connect.record.doc=xxx} and target
> parameters: \{connect.record.doc=yyy}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
