[ https://issues.apache.org/jira/browse/SPARK-28732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-28732: --------------------------------- Priority: Major (was: Blocker) > org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator - failed to > compile: org.codehaus.commons.compiler.CompileException: File > 'generated.java' when storing the result of a count aggregation in an integer > ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-28732 > URL: https://issues.apache.org/jira/browse/SPARK-28732 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.4.0 > Reporter: Alix Métivier > Priority: Major > > I am using agg function on a dataset, and i want to count the number of lines > upon grouping columns. I would like to store the result of this count in an > integer, but it fails with this output : > [ERROR]: org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator - > failed to compile: org.codehaus.commons.compiler.CompileException: File > 'generated.java', Line 89, Column 53: No applicable constructor/method found > for actual parameters "long"; candidates are: "java.lang.Integer(int)", > "java.lang.Integer(java.lang.String)" > Here is the line 89 and a few others to understand : > /* 085 */ long value13 = i.getLong(5); > /* 086 */ argValue4 = value13; > /* 087 */ > /* 088 */ > /* 089 */ final java.lang.Integer value12 = false ? null : new > java.lang.Integer(argValue4); > > As per Integer documentation, there is not constructor for the type Long, so > this is why the generated code fails. > > Here is my code : > org.apache.spark.sql.Dataset<row2Struct> ds_row2 = > ds_conntAggregateRow_1_Out_1 > .groupBy(org.apache.spark.sql.functions.col("n_name").as("n_nameN"), > org.apache.spark.sql.functions.col("o_year").as("o_yearN")) > .agg(org.apache.spark.sql.functions.count("n_name").as("countN"), > .as(org.apache.spark.sql.Encoders.bean(row2Struct.class)); > row2Struct class is composed of n_nameN: String, o_yearN: String, countN: Int > If countN is a Long, code above wont fail > If it is an Int, it works in 1.6 and 2.0, but fails on version 2.1+ > -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org