[ https://issues.apache.org/jira/browse/SPARK-23904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493327#comment-16493327 ]
Ruben Berenguel commented on SPARK-23904: ----------------------------------------- I could not (but had no time to dive too deep), but I've seen some issues that may be related as mentioned above. For the time being, can you use [this workaround|https://github.com/graphframes/graphframes/blob/master/src/main/scala/org/graphframes/lib/AggregateMessages.scala#L170] in some part of your query plan? I have used it where I have problems with too large plans and works excellent. But I'll try to carve some time to figure out why this issue is happening > Big execution plan cause OOM > ---------------------------- > > Key: SPARK-23904 > URL: https://issues.apache.org/jira/browse/SPARK-23904 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.2.1 > Reporter: Izek Greenfield > Priority: Major > Labels: SQL, query > > I create a question in > [StackOverflow|https://stackoverflow.com/questions/49508683/spark-physicalplandescription-string-is-to-big] > > Spark create the text representation of query in any case even if I don't > need it. > That causes many garbage object and unneeded GC... > [Gist with code to > reproduce|https://gist.github.com/igreenfield/584c3336f03ba7d63e9026774eaf5e23] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org