jacklzg created SPARK-33598: ------------------------------- Summary: Support Java Class with circular references Key: SPARK-33598 URL: https://issues.apache.org/jira/browse/SPARK-33598 Project: Spark Issue Type: Improvement Components: Java API Affects Versions: 2.4.7 Reporter: jacklzg
If the target Java data class has a circular reference, Spark will fail fast from creating the Dataset or running Encoders. For example, with protobuf class, there is a reference with Descriptor, there is no way to build a dataset from the protobuf class. >From this line ``` Encoders.bean(ProtoBuffOuterClass.ProtoBuff.class); ``` It will throw out immediately ``` Exception in thread "main" java.lang.UnsupportedOperationException: Cannot have circular references in bean class, but got the circular reference of class class com.google.protobuf.Descriptors$Descriptor ``` Can we add a parameter, for example, ``` Encoders.bean(Class<T> clas, List<Fields> fieldsToIgnore); ```` or ``` Encoders.bean(Class<T> clas, boolean skipCircularRefField); ```` -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org