[ 
https://issues.apache.org/jira/browse/SPARK-28618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

angerszhu resolved SPARK-28618.
-------------------------------
    Resolution: Auto Closed

> SparkSQL Plan CollectLimitExec may cause OOM
> --------------------------------------------
>
>                 Key: SPARK-28618
>                 URL: https://issues.apache.org/jira/browse/SPARK-28618
>             Project: Spark
>          Issue Type: Wish
>          Components: SQL
>    Affects Versions: 2.4.0
>            Reporter: angerszhu
>            Priority: Major
>         Attachments: image-2019-08-05-14-02-51-577.png, 
> image-2019-08-05-14-03-16-112.png
>
>
> When we call a limit in SQL or DataFrame, it will execute like below,if we 
> call limit(100), it will gather each partition;s first 100 rows data into one 
> partition. If we call like limit 1000000, it will cause OOM for this singal 
> partition. 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to