[ https://issues.apache.org/jira/browse/SPARK-25718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-25718: ------------------------------------ Assignee: Apache Spark > Detect recursive reference in Avro schema and throw exception > ------------------------------------------------------------- > > Key: SPARK-25718 > URL: https://issues.apache.org/jira/browse/SPARK-25718 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 2.4.0 > Reporter: Gengliang Wang > Assignee: Apache Spark > Priority: Major > > Avro schema allows recursive reference, e.g. the example schema in > https://avro.apache.org/docs/1.8.2/spec.html#schema_record > ``` > { > "type": "record", > "name": "LongList", > "aliases": ["LinkedLongs"], // old name for this > "fields" : [ > {"name": "value", "type": "long"}, // each element has a long > {"name": "next", "type": ["null", "LongList"]} // optional next element > ] > } > ``` > However, it is impossible to convert the schema as `StructType` in current > Spark SQL. Running `SchemaConverters.toSqlType(avroSchema: Schema)` and we > will get stack overflow exception. > We should detect the recursive reference and throw exception for it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org