[ https://issues.apache.org/jira/browse/SPARK-32486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167614#comment-17167614 ]
Dinesh Kumar commented on SPARK-32486: -------------------------------------- Hey, Its happening in all the higher versions. Can you please reopen? > Issue with deserialization and persist api in latest spark java versions > ------------------------------------------------------------------------ > > Key: SPARK-32486 > URL: https://issues.apache.org/jira/browse/SPARK-32486 > Project: Spark > Issue Type: Bug > Components: Java API > Affects Versions: 2.4.4, 2.4.5, 2.4.6, 3.0.0 > Environment: It's happening on all the os and java8 > Reporter: Dinesh Kumar > Priority: Blocker > > Hey Team, We have class level object instantiations in one of our Classes. > When we want to persist that data into the Dataset of this class Type it's > not persisting the null values instead it's taking class level precedence. > i.e. It's showing as new object. > Eg: > _Test.class has below class level attributes:_ > _private Test1 testNumber = new Test1();_ > _private Test2 testNumber2;_ > > String inputLocation = "src/test/resources/pipeline/test.parquet"; > Dataset<Row> ds = this.session.read().parquet(inputLocation); > ds.printSchema(); > ds.foreach(input->{ > System.out.println(input); // When we verified it's showing testNumber, > testNumber2 as null > }); > Dataset<Test> inputDataSet = ds.as(Encoders.bean(Test.class)); > inputDataSet.foreach(input->{ > System.out.println(input); // When we verified it's showing testNumber as > new Test1(), testNumber2 as null > }); > > > This is the same issue with dataset.persist() call aswell. It is happening > with all 2.4.4 and higher versions. Can you please fix it? > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org