[jira] [Created] (KAFKA-3910) Cyclic schema support in ConnectSchema and SchemaBuilder

[John Hofman (JIRA)]

John Hofman created KAFKA-3910:
----------------------------------

             Summary: Cyclic schema support in ConnectSchema and SchemaBuilder
                 Key: KAFKA-3910
                 URL: https://issues.apache.org/jira/browse/KAFKA-3910
             Project: Kafka
          Issue Type: Improvement
          Components: KafkaConnect
    Affects Versions: 0.10.0.0
            Reporter: John Hofman
            Assignee: Ewen Cheslack-Postava
            Priority: Blocker


Cyclic schema's are not supported by ConnectSchema or SchemaBuilder. 
Subsequently the AvroConverter (confluentinc/schema-registry) hits a stack 
overflow when converting a cyclic avro schema, e.g:

{code}
{"type":"record", 
"name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}
{code}

This is a blocking issue for all connectors running on the connect framework 
and with data containing cyclic references. The AvroConverter cannot support 
cyclic schema's until the underlying ConnectSchema and SchemaBuilder do.

To reproduce the stack-overflow (Confluent-3.0.0):

Produce some cyclic data:
{code}
bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test 
--property value.schema='{"type":"record", 
"name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}'
{"value":1,"next":null} 
{"value":1,"next":{"list":{"value":2,"next":null}}}
{code}

Then try to consume it with connect:

{code:title=connect-console-sink.properties}
name=local-console-sink 
connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector 
tasks.max=1 
topics=test
{code}

{code}
./bin/connect-standalone 
./etc/schema-registry/connect-avro-standalone.properties 
connect-console-sink.properties  
… start up logging … 
java.lang.StackOverflowError 
 at org.apache.avro.JsonProperties.getJsonProp(JsonProperties.java:54) 
 at org.apache.avro.JsonProperties.getProp(JsonProperties.java:45) 
 at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1055) 
 at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103) 
 at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137) 
 at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103) 
 at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
{code}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)