[ https://issues.apache.org/jira/browse/SPARK-28618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
angerszhu resolved SPARK-28618. ------------------------------- Resolution: Auto Closed > SparkSQL Plan CollectLimitExec may cause OOM > -------------------------------------------- > > Key: SPARK-28618 > URL: https://issues.apache.org/jira/browse/SPARK-28618 > Project: Spark > Issue Type: Wish > Components: SQL > Affects Versions: 2.4.0 > Reporter: angerszhu > Priority: Major > Attachments: image-2019-08-05-14-02-51-577.png, > image-2019-08-05-14-03-16-112.png > > > When we call a limit in SQL or DataFrame, it will execute like below,if we > call limit(100), it will gather each partition;s first 100 rows data into one > partition. If we call like limit 1000000, it will cause OOM for this singal > partition. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org