[ https://issues.apache.org/jira/browse/SPARK-22215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194623#comment-16194623 ]
Apache Spark commented on SPARK-22215: -------------------------------------- User 'mgaido91' has created a pull request for this issue: https://github.com/apache/spark/pull/19447 > Add a configuration parameter to set max size for generated classes > ------------------------------------------------------------------- > > Key: SPARK-22215 > URL: https://issues.apache.org/jira/browse/SPARK-22215 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.2.0 > Reporter: Marco Gaido > > SPARK-18016 introduced an arbitrary threshold for the size of a generated > class > (https://github.com/apache/spark/blob/83488cc3180ca18f829516f550766efb3095881e/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala#L286). > This value is hardcoded. > Since this is just a guess, in some cases making it smaller can help avoiding > the error of exceeding the maximum number of entries in the Constant Pool. > Then, I suggest to introduce a new configuration parameter, which defaults to > the previous value, but it allows to set this to a smaller one if needed. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org