[ https://issues.apache.org/jira/browse/SPARK-41007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen resolved SPARK-41007. ---------------------------------- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38500 [https://github.com/apache/spark/pull/38500] > BigInteger Serialization doesn't work with JavaBean Encoder > ----------------------------------------------------------- > > Key: SPARK-41007 > URL: https://issues.apache.org/jira/browse/SPARK-41007 > Project: Spark > Issue Type: Bug > Components: Java API > Affects Versions: 3.3.1 > Reporter: Daniel Fiterma > Assignee: Daniel Fiterma > Priority: Minor > Fix For: 3.4.0 > > > When creating a dataset using the [Java Bean > Encoder|https://spark.apache.org/docs/3.2.0/api/java/org/apache/spark/sql/Encoders.html#bean-java.lang.Class-] > with a bean that contains a field which is a {{java.math.BigInteger}} the > dataset will fail to serialize correctly. When trying to serialize the > dataset, Spark throws the following error: > > {code:java} > Exception in thread "main" org.apache.spark.sql.AnalysisException: Cannot up > cast `bigInteger` from struct<> to decimal(38,18). > {code} > > Reproduction steps: > Using the Java Dataset API: > # Create a Bean with a {{java.math.BigInteger}} field > # Pass said Bean into the Java SparkSession {{createDataset}} function > -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org