Gengliang Wang created SPARK-25718: -------------------------------------- Summary: Detect recursive reference in Avro schema and throw exception Key: SPARK-25718 URL: https://issues.apache.org/jira/browse/SPARK-25718 Project: Spark Issue Type: Sub-task Components: SQL Affects Versions: 2.4.0 Reporter: Gengliang Wang
Avro schema allows recursive reference, e.g. the example schema in https://avro.apache.org/docs/1.8.2/spec.html#schema_record ``` { "type": "record", "name": "LongList", "aliases": ["LinkedLongs"], // old name for this "fields" : [ {"name": "value", "type": "long"}, // each element has a long {"name": "next", "type": ["null", "LongList"]} // optional next element ] } ``` However, it is impossible to convert the schema as `StructType` in current Spark SQL. Running `SchemaConverters.toSqlType(avroSchema: Schema)` and we will get stack overflow exception. We should detect the recursive reference and throw exception for it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org